|
1 |
| -# Table Snapshot and Re-snapshot |
| 1 | +# Table Snapshot and Re-snapshot |
| 2 | + |
| 3 | +## Initial Snapshot |
| 4 | +"Initial snapshot" (or table snapshot) in SynchDB means to copy table schema plus initial data for all designated tables. This is similar to the term "table sync" in PostgreSQL logical replication. When a connector is started using the default `initial` mode, it will automatically perform the initial snapshot before going to Change Data Capture (CDC) stage. This can be omitted entirely with mode `never` or partially omitted with mode `no_data`. See [here](https://docs.synchdb.com/user-guide/start_stop_connector/) for all snapshot options. |
| 5 | + |
| 6 | +Once the initial snapshot is completed, the connector will not do it again upon subsequent restarts and will just resume with CDC since the last incomplete offset. This behavior is controled by the metadata files managed by Debezium engine. See [here](https://docs.synchdb.com/architecture/metadata_files/) for more about metadata files. |
| 7 | + |
| 8 | + |
| 9 | +## Re-snapshot |
| 10 | +If for any reason, user needs to perform the initial snapshot again to re-build `all the designated tables` and all of the initial data, we need to use the `always` snapshot mode, which causes the connector to obtain schema and initial data again at the moment the connector starts. You may need to drop all the desginated tables or clears ll the data in them before SynchDB will attempt to create the tables and populate initial data again, which may exist already. |
| 11 | + |
| 12 | +Please be cautious as this may be an aggressive action to perform in your setups. A better alternative would be a selective snapshot where only the selected tables will be re-snapshotted in `always` snapshot mode. See below. |
| 13 | + |
| 14 | + |
| 15 | +## Selective Snapshot |
| 16 | +Selective snapshot can be configured to a connector during creation of changed in run time. This done by specifying a list of tables to perform snapshot in `snapshot table` paramter. For example: |
| 17 | + |
| 18 | + |
| 19 | +**During creation:** |
| 20 | +This example creates a conenctor that perform CDC on `inventory.orders`,`inventory.customers` and `invnetory.produts` tables but will only do initial snapshot again for `inventory.products` if the connector starts in `always` snapshot mode. |
| 21 | + |
| 22 | +```sql |
| 23 | +SELECT synchdb_add_conninfo( |
| 24 | + 'mysqlconn', |
| 25 | + '127.0.0.1', |
| 26 | + 3306, |
| 27 | + 'mysqluser', |
| 28 | + 'mysqlpwd', |
| 29 | + 'inventory', |
| 30 | + 'postgres', |
| 31 | + 'inventory.orders,inventory.customers,invnetory.produts', |
| 32 | + 'inventory.products', |
| 33 | + 'mysql'); |
| 34 | + |
| 35 | +SELECT synchdb_start_engine_bgw('mysqlconn', 'always'); |
| 36 | +``` |
| 37 | + |
| 38 | +**Alter existing connector:** |
| 39 | +This example sets `inventory.products` to snapshot table field. When started in `always` mode, only `inventory.products` table will be re-snapshotted. |
| 40 | + |
| 41 | +```sql |
| 42 | +UPDATE synchdb_conninfo |
| 43 | + SET data = jsonb_set(data, '{snapshottable}', 'inventory.products', true) |
| 44 | + WHERE name = 'mysqlconn'; |
| 45 | + |
| 46 | +SELECT synchdb_start_engine_bgw('mysqlconn', 'always'); |
| 47 | +``` |
0 commit comments