You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/en/tutorial/selective_table_sync.md
+9-4Lines changed: 9 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -60,7 +60,7 @@ postgres=#
60
60
Once the snapshot is complete, the `mysqlconn` connector will continue capturing subsequent changes to the `inventory.orders` and `inventory.products` tables.
61
61
62
62
## **Add More Tables to Replicate During Run Time.**
63
-
**
63
+
64
64
The `mysqlconn` from previous section has already completed the initial snapshot and obtained the table schemas of the selected table. If we would like to add more tables to replicate from, we will need to notify the Debezium engine about the updated table section and perform the initial snapshot again. Here's how it is done:
65
65
66
66
1. Update the `synchdb_conninfo` table to include additional tables.
@@ -70,13 +70,18 @@ UPDATE synchdb_conninfo
70
70
SET data = jsonb_set(data, '{table}', '"inventory.orders,inventory.products,inventory.customers"')
71
71
WHERE name ='mysqlconn';
72
72
```
73
-
3. Restart the connector with the snapshot mode set to `always` to perform another initial snapshot:
73
+
3. Configure the snapshot table parameter to include only the new table `inventory.customers` to that SynchDB does not try to rebuild the 2 tables that have already finished the snapshot.
74
+
```sql
75
+
UPDATE synchdb_conninfo
76
+
SET data = jsonb_set(data, '{snapshottable}', '"inventory.customers"')
77
+
WHERE name ='mysqlconn';
78
+
```
79
+
4. Restart the connector with the snapshot mode set to `always` to perform another initial snapshot:
This forces Debezium to re-snapshot all the specified tables, even if two of them already have the data.
83
+
This forces Debezium to re-snapshot only the new table `inventory.customers` while leaving the old tables`inventory.orders` and `inventory.products` untouched. The CDC for all tables will resume once snapshot is complete.
78
84
79
-
Be mindful that if the heterogeneous database type does not support DDL replication (such as SQLServer), we may get a data conflict error when the snapshot is being rebuilt on the 2 tables previously selected for replication. If this is the case, we may need to drop or truncate them before restarting the connector with snapshot mode = 'always'.
Copy file name to clipboardExpand all lines: docs/zh/tutorial/selective_table_sync.md
+37-31Lines changed: 37 additions & 31 deletions
Original file line number
Diff line number
Diff line change
@@ -1,15 +1,15 @@
1
1
---
2
2
weight: 90
3
3
---
4
-
# Selective Table Synchronization
4
+
# 选择性表同步
5
5
6
-
It is possible to select only specific tables from remote heterogeneous database to focus on replication. This could prevent resources being spent on replicating unwanted tables.
6
+
可以从远程异构数据库中仅选择特定表进行同步,从而避免将资源浪费在复制不需要的表上。
7
7
8
-
## **Select Desired Tables and Start it for the First Time**
8
+
## **选择所需表并首次启动**
9
9
10
-
Table selection is done during connector creation phase via `synchdb_add_conninfo()`where we specify a list of tables (expressed in FQN, separated by a comma) to replicate from.
For example, the following command creates a connector that only replicates change from `inventory.orders`and`inventory.products`tables from remote MySQL database.
12
+
例如,以下命令创建一个连接器,该连接器仅从远程 MySQL 数据库的 `inventory.orders`和`inventory.products`表复制更改。
13
13
```sql
14
14
SELECT synchdb_add_conninfo(
15
15
'mysqlconn',
@@ -20,19 +20,20 @@ SELECT synchdb_add_conninfo(
20
20
'inventory',
21
21
'postgres',
22
22
'inventory.orders,inventory.products',
23
+
'null'
23
24
'mysql'
24
25
);
25
26
```
26
27
27
-
Starting this connector for the very first time will trigger an initial snapshot being performed and selected 2 tables' schema and data will be replicated.
28
+
首次启动此连接器将触发执行初始快照,并将复制选定的 2 个表的模式和数据。
28
29
29
30
```sql
30
31
SELECT synchdb_start_engine_bgw('mysqlconn');
31
32
```
32
33
33
-
### **Verify the Connector State and Tables**
34
+
### **验证连接器状态和表**
34
35
35
-
Examine the connector state and the new tables:
36
+
检查连接器状态和新表:
36
37
```sql
37
38
postgres=# Select name, state, err from synchdb_state_view;
38
39
name | state | err
@@ -57,30 +58,35 @@ postgres=# \d
57
58
postgres=#
58
59
```
59
60
60
-
Once the snapshot is complete, the `mysqlconn`connector will continue capturing subsequent changes to the `inventory.orders`and`inventory.products`tables.
## **Add More Tables to Replicate During Run Time.**
63
-
**
64
-
The `mysqlconn` from previous section has already completed the initial snapshot and obtained the table schemas of the selected table. If we would like to add more tables to replicate from, we will need to notify the Debezium engine about the updated table section and perform the initial snapshot again. Here's how it is done:
63
+
## **在运行时添加更多要复制的表**
65
64
66
-
1. Update the `synchdb_conninfo` table to include additional tables.
67
-
2. In this example, we add the `inventory.customers` table to the sync list:
Be mindful that if the heterogeneous database type does not support DDL replication (such as SQLServer), we may get a data conflict error when the snapshot is being rebuilt on the 2 tables previously selected for replication. If this is the case, we may need to drop or truncate them before restarting the connector with snapshot mode = 'always'.
80
86
81
-
### **Verify the Updated Tables**
87
+
### **验证更新后的表**
82
88
83
-
Now, we can examine our tables again:
89
+
现在,我们可以再次检查这些表:
84
90
```sql
85
91
postgres=# SET search_path TO inventory;
86
92
SET
@@ -102,16 +108,16 @@ postgres=#
102
108
103
109
```
104
110
105
-
## **Snapshot Modes**
111
+
## **快照模式**
106
112
107
-
SynchDB offers different snapshot modes, depending on your replication needs:
113
+
SynchDB 提供不同的快照模式,具体取决于您的复制需求:
108
114
109
-
|**setting**|**description**|
115
+
|**设置**|**描述**|
110
116
|:-:|-|
111
-
| always | The connector performs a snapshot every time that it starts. The snapshot includes the structure and data of the captured tables. After the snapshot completes, the connector begins to stream event records for subsequent database changes.|
112
-
| initial (default) | The connector performs a database snapshot if not already done. After the snapshot completes, the connector begins to stream event records for subsequent database changes.|
113
-
| initial_only |The connector performs a database snapshot. After the snapshot completes, the connector stops, and does not stream event records for subsequent database changes.|
114
-
| no_data | The connector captures the structure of all relevant tables, but not the data they contain.|
115
-
| never | When the connector starts, rather than performing a snapshot, it immediately begins to stream event records for subsequent database changes.|
116
-
| recovery | Set this option to restore a database schema history that is lost or corrupted. After a restart, the connector runs a snapshot that rebuilds the topic from the source tables|
117
-
| when_needed | After the connector starts, it performs a snapshot only if it detects one of the following circumstances:<br><ul><li>It cannot detect any topic offsets</li><li>A previously recorded offset specifies a log position that is not available on the server</li></ul> |
0 commit comments