Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add some example configurations for worker #12492

Merged
merged 3 commits into from
Apr 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/12492.doc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add some example configurations for workers and update architectural diagram.
8 changes: 8 additions & 0 deletions docs/systemd-with-workers/workers/background_worker.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
worker_app: synapse.app.generic_worker
worker_name: background_worker

# The replication listener on the main synapse process.
worker_replication_host: 127.0.0.1
worker_replication_http_port: 9093

worker_log_config: /etc/matrix-synapse/background-worker-log.yaml
23 changes: 23 additions & 0 deletions docs/systemd-with-workers/workers/event_persister.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
worker_app: synapse.app.generic_worker
worker_name: event_persister1

# The replication listener on the main synapse process.
worker_replication_host: 127.0.0.1
worker_replication_http_port: 9093

worker_listeners:
- type: http
port: 8034
resources:
- names: [replication]

# Enable listener if this stream writer handles endpoints for the `typing` or
# `to_device` streams. Uses a different port to the `replication` listener to
# avoid exposing the `replication` listener publicly.
#
#- type: http
# port: 8035
# resources:
# - names: [client]

worker_log_config: /etc/matrix-synapse/event-persister-log.yaml
3 changes: 2 additions & 1 deletion docs/systemd-with-workers/workers/generic_worker.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
worker_app: synapse.app.generic_worker
worker_name: generic_worker1

# The replication listener on the main synapse process.
worker_replication_host: 127.0.0.1
worker_replication_http_port: 9093

worker_listeners:
- type: http
port: 8011
port: 8083
resources:
- names: [client, federation]

Expand Down
45 changes: 22 additions & 23 deletions docs/workers.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,20 +138,7 @@ as the `listeners` option in the shared config.
For example:

```yaml
worker_app: synapse.app.generic_worker
worker_name: worker1

# The replication listener on the main synapse process.
worker_replication_host: 127.0.0.1
worker_replication_http_port: 9093

worker_listeners:
- type: http
port: 8083
resources:
- names: [client, federation]

worker_log_config: /home/matrix/synapse/config/worker1_log_config.yaml
{{#include systemd-with-workers/workers/generic_worker.yaml}}
```

...is a full configuration for a generic worker instance, which will expose a
Expand Down Expand Up @@ -363,6 +350,12 @@ stream_writers:
events: event_persister1
```

An example for a stream writer instance:

```yaml
{{#include systemd-with-workers/workers/event_persister.yaml}}
```

Some of the streams have associated endpoints which, for maximum efficiency, should
be routed to the workers handling that stream. See below for the currently supported
streams and the endpoints associated with them:
Expand Down Expand Up @@ -436,6 +429,12 @@ run_background_tasks_on: background_worker
You might also wish to investigate the `update_user_directory` and
`media_instance_running_background_jobs` settings.

An example for a dedicated background worker instance:

```yaml
{{#include systemd-with-workers/workers/background_worker.yaml}}
```

### `synapse.app.pusher`

Handles sending push notifications to sygnal and email. Doesn't handle any
Expand Down Expand Up @@ -615,14 +614,14 @@ The following shows an example setup using Redis and a reverse proxy:
| Main | | Generic | | Generic | | Event |
| Process | | Worker 1 | | Worker 2 | | Persister |
+--------------+ +--------------+ +--------------+ +--------------+
^ ^ | ^ | | ^ | ^ ^
| | | | | | | | | |
| | | | | HTTP | | | | |
| +----------+<--|---|---------+ | | | |
| | +-------------|-->+----------+ |
| | | |
| | | |
v v v v
====================================================================
^ ^ | ^ | | ^ | | ^ ^
| | | | | | | | | | |
| | | | | HTTP | | | | | |
| +----------+<--|---|---------+<--|---|---------+ | |
| | +-------------|-->+-------------+ |
| | | |
| | | |
v v v v
======================================================================
squahtx marked this conversation as resolved.
Show resolved Hide resolved
Redis pub/sub channel
```