Skip to content

Commit 5e7f8a0

Browse files
author
Cary Huang
committed
updates
1 parent ca24f6a commit 5e7f8a0

File tree

9 files changed

+271
-15
lines changed

9 files changed

+271
-15
lines changed

‎docs/en/monitoring/state_view.md

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,27 @@ Column Details:
2222
| name | the associated connector info name created by `synchdb_add_conninfo()`|
2323
| connector_type | the type of connector (mysql, oracle, sqlserver...etc)|
2424
| pid | the PID of the connector worker process|
25-
| stage | the stage of the connector worker process|
26-
| state | the state of the connector. Possible states are: <br><br><ul><li>stopped - connector is not running</li><li>initializing - connector is initializing</li><li>paused - connector is paused</li><li>syncing - connector is regularly polling change events</li><li>parsing (the connector is parsing a received change event) </li><li>converting - connector is converting a change event to PostgreSQL representation</li><li>executing - connector is applying the converted change event to PostgreSQL</li><li>updating offset - connector is writing a new offset value to Debezium offset management</li><li>restarting - connector is restarting </li><li>dumping memory - connector is dumping JVM memory summary in log file </li><li>unknown</li></ul> |
25+
| stage | the stage of the connector. See below.|
26+
| state | the state of the connector. See below.|
2727
| err | the last error message encountered by the worker which would have caused it to exit. This error could originated from PostgreSQL while processing a change, or originated from Debezium running engine while accessing data from heterogeneous database. |
28-
| last_dbz_offset | the last Debezium offset captured by synchdb. Note that this may not reflect the current and real-time offset value of the connector engine. Rather, this is shown as a checkpoint that we could restart from this offeet point if needed.|
28+
| last_dbz_offset | the last Debezium offset captured by synchdb. Note that this may not reflect the current and real-time offset value of the connector engine. Rather, this is shown as a checkpoint that we could restart from this offeet point if needed.|
29+
30+
**Possible States**:
31+
32+
- 🔴 `stopped` - Inactive
33+
- 🟡 `initializing` - Starting up
34+
- 🟠 `paused` - Temporarily halted
35+
- 🟢 `syncing` - Actively polling
36+
- 🔵 `parsing` - Processing events
37+
- 🟣 `converting` - Transforming data
38+
- ⚪ `executing` - Applying changes
39+
- 🟤 `updating offset` - Updating checkpoint
40+
- 🟨 `restarting` - Reinitializing
41+
- ⚪ `dumping memory` - JVM is prepaaring to dump memory info in log file
42+
- ⚫ `unknown` - Indeterminate state
43+
44+
**Possible Stages**:
45+
46+
- `initial snapshot` - connector is performing initial snapshot (building table schema and optionally the initial data)
47+
- `change data capture` - connector is streaming subsequent table changes (CDC)
48+
- `schema sync` - connector is copying table schema only (no data)

‎docs/en/monitoring/stats_view.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,14 @@ Column Details:
3434
| last_src_ts | the timestamp in nanoseconds when the last batch's last event is produced at the external database |
3535
| last_dbz_ts | the timestamp in nanoseconds when the last batch's last event is processed by Debezium Engine |
3636
| last_pg_ts | the timestamp in nanoseconds when the last batch's last event is applied to PostgreSQL |
37+
38+
39+
**Note: the statistic information is not persistent and will be wiped if PostgreSQL shuts down or restarts**
40+
41+
## **synchdb_reset_stats**
42+
43+
**Purpose**: Resets all statistic information of given connector name
44+
45+
```sql
46+
SELECT synchdb_reset_stats('mysqlconn');
47+
```

‎docs/en/user-guide/secured_connection.md

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,31 @@
1-
# Secured Connection
1+
# Secured Connection
2+
3+
## **Configure Secured Connection**
4+
5+
to secure the connection to remote database, we need to configure additional SSL related parameters to a connector that has been created by `synchdb_add_conninfo`. The SSL certificates and private keys must be packaged as Java keystore file with a passphrase. These information is then passed to SynchDB via synchdb_add_extra_conninfo().
6+
7+
8+
## **synchdb_add_extra_conninfo**
9+
10+
**Purpose**: Configures extra connector parameters to an existing connector created by `synchdb_add_conninfo`
11+
12+
| Parameter | Description | Required | Example | Notes |
13+
|:-:|:-|:-:|:-|:-|
14+
| `name` | Unique identifier for this connector | ✓ | `'mysqlconn'` | Must be unique across all connectors |
15+
| `ssl_mode` | SSL mode | ☐ | `'verify_ca'` | can be one of: <br><ul><li> "disabled" - no SSL is used. </li><li> "preferred" - SSL is used if server supports it. </li><li> "required" - SSL must be used to establish a connection. </li><li> "verify_ca" - connector establishes TLS with the server and will also verify server's TLS certificate against configured truststore. </li><li> "verify_identity" - same behavior as verify_ca but it also checks the server certificate's common name to match the hostname of the system. |
16+
| `ssl_keystore` | keystore path | ☐ | `/path/to/keystore` | path to the keystore file |
17+
| `ssl_keystore_pass` | keystore password | ☐ | `'mykeystorepass'` | password to access the keystore file |
18+
| `ssl_truststore` | trust store path | ☐ | `'/path/to/truststore'` | path to the truststore file |
19+
| `ssl_truststore_pass` | trust store password | ☐ | `'mytruststorepass'` | password to access the truststore file |
20+
21+
22+
```sql
23+
SELECT synchdb_add_extra_conninfo('mysqlconn', 'verify_ca', '/path/to/keystore', 'mykeystorepass', '/path/to/truststore', 'mytruststorepass');
24+
```
25+
26+
## **synchdb_del_extra_conninfo**
27+
28+
**Purpose**: Deletes extra connector paramters created by `synchdb_add_extra_conninfo`
29+
```sql
30+
SELECT synchdb_del_extra_conninfo('mysqlconn');
31+
```

‎docs/en/user-guide/start_stop_connector.md

Lines changed: 69 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,69 @@
1-
# Start / Stop a Connector
1+
# Start / Stop a Connector
2+
3+
## **Control a Connector**
4+
5+
SynchDB provides several utility function to control the behavior and life cycle or a created connector.
6+
7+
## **Start a Connector with Default Snapshot Mode**
8+
9+
synchdb_start_engine_bgw() can be used to start a connector with default snapshot mode called `initial`.
10+
11+
```sql
12+
SELECT synchdb_start_engine_bgw('mysqlconn');
13+
```
14+
15+
## **Start a Connector with Custom Snapshot Mode**
16+
17+
Using the same function synchdb_start_engine_bgw(), it is possible to include snapshot mode to start the connector with, otherwise the `initial` mode will be used by default.
18+
19+
```sql
20+
-- capture table schema and proceed to stream new changes
21+
SELECT synchdb_start_engine_bgw('mysqlconn', 'no_data');
22+
23+
-- always re-build the table schema, existing data and proceed to stream new changes
24+
SELECT synchdb_start_engine_bgw('mysqlconn', 'always');
25+
```
26+
27+
## **Supported Snapshot Modes**
28+
29+
| Mode | Description | Use Case |
30+
|:-:|-|-|
31+
| `always` | Full snapshot on every start | Complete data verification |
32+
| `initial` | First-time snapshot only | Normal operations |
33+
| `initial_only` | One-time snapshot, then stop | Data migration |
34+
| `no_data` | Structure only, no data | Schema synchronization |
35+
| `never` | Skip snapshot, stream only | Real-time updates |
36+
| `recovery` | Rebuilds from source | Disaster recovery |
37+
| `when_needed` | Conditional snapshot | Automatic recovery |
38+
| `schemasync` | Structure only, no data, no CDC | normal operations |
39+
40+
41+
**Refer to the [tutorial](https://docs.synchdb.com/tutorial/selective_table_sync/) on when to use what mode**
42+
43+
## **Pause and Resume a Connector**
44+
45+
Pause a connector with `synchdb_pause_engine`, which temporarily halts a running connector
46+
```sql
47+
SELECT synchdb_pause_engine('mysqlconn');
48+
```
49+
50+
Resume a paused connector with `synchdb_resume_engine`.
51+
```sql
52+
SELECT synchdb_resume_engine('mysqlconn');
53+
```
54+
55+
## **Stop or Restart a Running Connector**
56+
57+
Stops a running connector with `synchdb_stop_engine`
58+
```sql
59+
SELECT synchdb_stop_engine('mysqlconn');
60+
```
61+
62+
Restarts a running connector with `synchdb_restart_connector` with a different snapshot mode.
63+
```sql
64+
-- Restart with specific snapshot mode
65+
SELECT synchdb_restart_connector('mysqlconn', 'initial');
66+
67+
-- Start with specific snapshot mode
68+
SELECT synchdb_start_engine_bgw('mysqlconn', 'always');
69+
```

‎docs/zh/monitoring/state_view.md

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,27 @@ postgres=# select * from synchdb_state_view;
2222
| name | 由 `synchdb_add_conninfo()` 创建的关联连接器信息名称 |
2323
| connector_type | 连接器类型（mysql、oracle、sqlserver 等）|
2424
| pid | 连接器工作进程的 PID |
25-
| stage | 连接器工作进程的阶段 |
26-
| state | 连接器的状态。可能的状态有：<br><br><ul><li>已停止 - 连接器未运行</li><li>正在初始化 - 连接器正在初始化</li><li>已暂停 - 连接器已暂停</li><li>正在同步 - 连接器正在定期轮询变更事件</li><li>正在解析（连接器正在解析收到的变更事件）</li><li>正在转换 - 连接器正在将变更事件转换为 PostgreSQL 表示</li><li>正在执行 - 连接器正在将转换后的变更事件应用到 PostgreSQL</li><li>正在更新偏移量 - 连接器正在将新的偏移量值写入 Debezium 偏移量管理</li><li>正在重启 - 连接器正在重启</li><li>正在转储内存 - 连接器正在将 JVM 内存摘要转储到日志文件中</li><li>未知</li></ul> |
27-
| err | 工作进程遇到的最后一条可能导致其退出的错误消息。此错误可能源于 PostgreSQL 处理更改时，也可能源于 Debezium 运行引擎访问异构数据库数据时。|
28-
| last_dbz_offset | synchdb 捕获的最后一个 Debezium 偏移量。请注意，这可能无法反映连接器引擎的当前和实时偏移量值。相反，它显示为一个检查点，我们可以根据需要从此偏移点重新启动。|
25+
| stage | 连接器的阶段。见下文。|
26+
| state | 连接器的状态。见下文。|
27+
| err | 工作进程遇到的最后一个可能导致其退出的错误消息。此错误可能源于 PostgreSQL 处理更改时，也可能源于 Debezium 运行引擎访问异构数据库数据时。|
28+
| last_dbz_offset | synchdb 捕获的最后一个 Debezium 偏移量。请注意，这可能无法反映连接器引擎的当前实时偏移量值。相反，它显示为一个检查点，我们可以根据需要从此偏移量点重新启动。|
29+
30+
**可能的状态**：
31+
32+
- 🔴 `stopped` - 非活动
33+
- 🟡 `initializing` - 正在启动
34+
- 🟠 `paused` - 暂时停止
35+
- 🟢 `syncing` - 正在主动轮询
36+
- 🔵 `parsing` - 正在处理事件
37+
- 🟣 `converting` - 正在转换数据
38+
- ⚪ `executing` - 正在应用更改
39+
- 🟤 `updating offset` - 正在更新检查点
40+
- 🟨 `restarting` - 正在重新初始化
41+
- ⚪ `dumping memory` - JVM 正在准备将内存信息转储到日志文件
42+
- ⚫ `unknown` - 不确定状态
43+
44+
**可能的状态**：
45+
46+
- `initial snapping` - 连接器正在执行初始快照（构建表结构以及可选的初始数据）
47+
- `change data capture` - 连接器正在流式传输后续表更改 (CDC)
48+
- `schema sync` - 连接器仅复制表架构（无数据）

‎docs/zh/monitoring/stats_view.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,4 +32,14 @@ postgres=# select * from synchdb_stats_view;
3232
| first_pg_ts | 最后一个批次的第一个事件应用到 PostgreSQL 的时间戳（纳秒）|
3333
| last_src_ts | 外部数据库生成最后一个批次的最后一个事件的时间戳（纳秒）|
3434
| last_dbz_ts | Debezium 引擎处理最后一个批次的最后一个事件的时间戳（纳秒）|
35-
| last_pg_ts | 最后一个批次的最后一个事件应用到 PostgreSQL 的时间戳（纳秒）|
35+
| last_pg_ts | 最后一个批次的最后一个事件应用到 PostgreSQL 的时间戳（纳秒）|
36+
37+
**注意：统计信息不是持久化的，如果 PostgreSQL 关闭或重启，统计信息将被清除**
38+
39+
## **synchdb_reset_stats**
40+
41+
**用途**：重置指定连接器名称的所有统计信息
42+
43+
```sql
44+
SELECT synchdb_reset_stats('mysqlconn');
45+
```

‎docs/zh/user-guide/secured_connection.md

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,29 @@
1-
# Secured Connection
1+
# 安全连接
2+
3+
## **配置安全连接**
4+
5+
为了确保与远程数据库的连接安全，我们需要为 `synchdb_add_conninfo` 创建的连接器配置额外的 SSL 相关参数。SSL 证书和私钥必须打包为 Java 密钥库文件，并附带密码。这些信息随后会通过 synchdb_add_extra_conninfo() 传递给 SynchDB。
6+
7+
## **synchdb_add_extra_conninfo**
8+
9+
**用途**：为 `synchdb_add_conninfo` 创建的现有连接器配置额外的连接器参数
10+
11+
| 参数 | 描述 | 必需 | 示例 | 备注 |
12+
|:-:|:-|:-:|:-|:-|
13+
| `name` | 此连接器的唯一标识符 | ✓ | `'mysqlconn'` | 必须在所有连接器中唯一 |
14+
| `ssl_mode` | SSL 模式 | ☐ | `'verify_ca'` |可以是以下之一：<br><ul><li>“disabled”- 不使用 SSL。</li><li>“preferred”- 如果服务器支持，则使用 SSL。</li><li>“required”- 必须使用 SSL 建立连接。</li><li>“verify_ca”- 连接器与服务器建立 TLS，并将根据配置的信任库验证服务器的 TLS 证书。</li><li>“verify_identity”- 与 verify_ca 行为相同，但它还会检查服务器证书的通用名称以匹配系统的主机名。|
15+
| `ssl_keystore` | 密钥库路径 | ☐ | `/path/to/keystore` | 密钥库文件的路径 |
16+
| `ssl_keystore_pass` | 密钥库密码 | ☐ | `'mykeystorepass'` | 访问密钥库文件的密码 |
17+
| `ssl_truststore` | 信任库路径 | ☐ | `'/path/to/truststore'` | 信任库文件路径 |
18+
| `ssl_truststore_pass` | 信任库密码 | ☐ | `'mytruststorepass'` | 访问信任库文件的密码 |
19+
20+
```sql
21+
SELECT synchdb_add_extra_conninfo('mysqlconn', 'verify_ca', '/path/to/keystore', 'mykeystorepass', '/path/to/truststore', 'mytruststorepass');
22+
```
23+
24+
## **synchdb_del_extra_conninfo**
25+
26+
**用途**：删除由 `synchdb_add_extra_conninfo` 创建的额外连接器参数
27+
```sql
28+
SELECT synchdb_del_extra_conninfo('mysqlconn');
29+
```

‎docs/zh/user-guide/start_stop_connector.md

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,68 @@
1-
# 启动/停止连接器
1+
# 启动/停止连接器
2+
3+
## **控制连接器**
4+
5+
SynchDB 提供了几个实用函数来控制已创建连接器的行为和生命周期。
6+
7+
## **以默认快照模式启动连接器**
8+
9+
synchdb_start_engine_bgw() 可用于以名为“initial”的默认快照模式启动连接器。
10+
11+
```sql
12+
SELECT synchdb_start_engine_bgw('mysqlconn');
13+
```
14+
15+
## **以自定义快照模式启动连接器**
16+
17+
使用相同的函数 synchdb_start_engine_bgw()，可以包含快照模式来启动连接器，否则将默认使用“initial”模式。
18+
19+
```sql
20+
-- 捕获表结构并继续流式传输新的更改
21+
SELECT synchdb_start_engine_bgw('mysqlconn', 'no_data');
22+
23+
-- 始终重建表结构和现有数据，并继续流式传输新的更改
24+
SELECT synchdb_start_engine_bgw('mysqlconn', 'always');
25+
```
26+
27+
## **支持的快照模式**
28+
29+
| 模式 | 描述 | 用例 |
30+
|:-:|-|-|
31+
| `always` | 每次启动时都进行完整快照 | 完成数据验证 |
32+
| `initial` | 仅限首次快照 | 正常操作 |
33+
| `initial_only` | 一次性快照，然后停止 | 数据迁移 |
34+
| `no_data` | 仅结构，无数据 | 结构同步 |
35+
| `never` | 跳过快照，仅流式传输 | 实时更新 |
36+
| `recovery` | 从源重建 | 灾难恢复 |
37+
| `when_needed` | 条件快照 | 自动恢复 |
38+
| `schemasync` | 仅结构，无数据，无 CDC | 正常操作 |
39+
40+
**请参阅[教程](https://docs.synchdb.com/zh/tutorial/selective_table_sync/)，了解何时使用何种模式**
41+
42+
## **暂停和恢复连接器**
43+
44+
使用 `synchdb_pause_engine` 暂停连接器，这会暂时停止正在运行的连接器。
45+
```sql
46+
SELECT synchdb_pause_engine('mysqlconn');
47+
```
48+
49+
使用 `synchdb_resume_engine` 恢复已暂停的连接器。
50+
```sql
51+
SELECT synchdb_resume_engine('mysqlconn');
52+
```
53+
54+
## **停止或重启正在运行的连接器**
55+
56+
使用 `synchdb_stop_engine` 停止正在运行的连接器。
57+
```sql
58+
SELECT synchdb_stop_engine('mysqlconn');
59+
```
60+
61+
使用 `synchdb_restart_connector` 以不同的快照模式重启正在运行的连接器。
62+
```sql
63+
-- 使用特定快照模式重启
64+
SELECT synchdb_restart_connector('mysqlconn', 'initial');
65+
66+
-- 使用特定快照模式启动
67+
SELECT synchdb_start_engine_bgw('mysqlconn', 'always');
68+
```

‎mkdocs.yml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ plugins:
7373
Create a Connector: Crear un conector
7474
Statistic View: Vista de estadísticas
7575
State View: Vista de estado
76+
Start/Stop a Connector: Iniciar/Detener un conector
7677
- locale: zh
7778
name: 中文
7879
site_name: SynchDB 文档
@@ -106,6 +107,7 @@ plugins:
106107
Create a Connector: 创建连接器
107108
Statistic View: 统计视图
108109
State View: 状态视图
110+
Start/Stop a Connector: 启动/停止连接器
109111
- mike:
110112
version_selector: true
111113
css_dir: css
@@ -147,16 +149,16 @@ nav:
147149
# - Multi Source Replication: tutorial/multi_source_replication.md
148150
- User guide:
149151
- Create a Connector: user-guide/create_a_connector.md
150-
# - Start/Stop a Connector: user-guide/start_stop_connector.md
152+
- Start/Stop a Connector: user-guide/start_stop_connector.md
151153
- Automatic Connector Launcher: user-guide/connector_auto_launcher.md
152154
- Default Data Type Mapping: user-guide/default_datatype_mapping.md
153155
- Customize Object Mapping: user-guide/object_mapping_rules.md
154156
# - Customize Transformation: user-guide/customize_transform.md
155-
# - Secured Connection: user-guide/secured_connection.md
157+
- Secured Connection: user-guide/secured_connection.md
156158
- Configure Error Strategies: user-guide/configure_error_strategies.md
157159
- DDL Replication: user-guide/ddl_replication.md
158160
- Set Offset: user-guide/set_offset.md
159-
- Utility Functions: user-guide/utility_functions.md
161+
# - Utility Functions: user-guide/utility_functions.md
160162
- Changelog: changelog.md
161163
# - Homepage: https://www.synchdb.com
162164

0 commit comments

Comments
 (0)