From 4944c01a0837c532c92e5c0bdaa3f4a90088e290 Mon Sep 17 00:00:00 2001 From: "suxiaobin.sxb" Date: Tue, 10 Sep 2024 20:16:49 +0800 Subject: [PATCH] docs: fix docs --- docs/src/en/filter/function.md | 141 ++++++++++++++++++++++++------ docs/src/en/guide/introduction.md | 2 +- docs/src/zh/filter/function.md | 2 - docs/src/zh/guide/introduction.md | 2 +- 4 files changed, 118 insertions(+), 29 deletions(-) diff --git a/docs/src/en/filter/function.md b/docs/src/en/filter/function.md index 579d0eec..355ad882 100644 --- a/docs/src/en/filter/function.md +++ b/docs/src/en/filter/function.md @@ -2,17 +2,17 @@ outline: deep --- -# 什么是 function +# What is function -RedisShake 通过提供 function 功能，实现了的 [ETL(提取-转换-加载)](https://en.wikipedia.org/wiki/Extract,_transform,_load) 中的 `transform` 能力。通过利用 function 可以实现类似功能： -* 更改数据所属的 `db`，比如将源端的 `db 0` 写入到目的端的 `db 1`。 -* 对数据进行筛选，例如，只将 key 以 `user:` 开头的源数据写入到目标端。 -* 改变 Key 的前缀，例如，将源端的 key `prefix_old_key` 写入到目标端的 key `prefix_new_key`。 +RedisShake provides a function feature that implements the `transform` capability in [ETL (Extract-Transform-Load)](https://en.wikipedia.org/wiki/Extract,_transform,_load). By utilizing functions, you can achieve similar functionalities: +* Change the `db` to which data belongs, for example, writing data from source `db 0` to destination `db 1`. +* Filter data, for instance, only writing source data with keys starting with `user:` to the destination. +* Modify key prefixes, such as writing a source key `prefix_old_key` to a destination key `prefix_new_key`. * ... -要使用 function 功能，只需编写一份 lua 脚本。RedisShake 在从源端获取数据后，会将数据转换为 Redis 命令。然后，它会处理这些命令，从中解析出 `KEYS`、`ARGV`、`SLOTS`、`GROUP` 等信息，并将这些信息传递给 lua 脚本。lua 脚本会处理这些数据，并返回处理后的命令。最后，RedisShake 会将处理后的数据写入到目标端。 +To use the function feature, you only need to write a Lua script. After RedisShake retrieves data from the source, it converts the data into Redis commands. Then, it processes these commands, parsing information such as `KEYS`, `ARGV`, `SLOTS`, `GROUP`, and passes this information to the Lua script. The Lua script processes this data and returns the processed commands. Finally, RedisShake writes the processed data to the destination. -以下是一个具体的例子： +Here's a specific example: ```toml function = """ shake.log(DB) @@ -29,28 +29,119 @@ address = "127.0.0.1:6379" [redis_writer] address = "127.0.0.1:6380" ``` -`DB` 是 RedisShake 提供的信息，表示当前数据所属的 db。`shake.log` 用于打印日志，`shake.call` 用于调用 Redis 命令。上述脚本的目的是丢弃源端 `db` 0 的数据，将其他 `db` 的数据写入到目标端。 +`DB` is information provided by RedisShake, indicating the db to which the current data belongs. `shake.log` is used for logging, and `shake.call` is used to call Redis commands. The purpose of the above script is to discard data from source `db 0` and write data from other `db`s to the destination. -除了 `DB`，还有其他信息如 `KEYS`、`ARGV`、`SLOTS`、`GROUP` 等，可供调用的函数有 `shake.log` 和 `shake.call`，具体请参考 [function API](#function-api)。 - -关于更多的示例，可以参考 [最佳实践](./best_practices.md)。 +In addition to `DB`, there is other information such as `KEYS`, `ARGV`, `SLOTS`, `GROUP`, and available functions include `shake.log` and `shake.call`. For details, please refer to [function API](#function-api). ## function API -### 变量 +### Variables -因为有些命令中含有多个 key，比如 `mset` 等命令。所以，`KEYS`、`KEY_INDEXES`、`SLOTS` 这三个变量都是数组类型。如果确认命令只有一个 key，可以直接使用 `KEYS[1]`、`KEY_INDEXES[1]`、`SLOTS[1]`。 +Because some commands contain multiple keys, such as the `mset` command, the variables `KEYS`, `KEY_INDEXES`, and `SLOTS` are all array types. If you are certain that a command has only one key, you can directly use `KEYS[1]`, `KEY_INDEXES[1]`, `SLOTS[1]`. -| 变量 | 类型 | 示例 | 描述 | +| Variable | Type | Example | Description | |-|-|-|-----| -| DB | number | 1 | 命令所属的 `db` | -| GROUP | string | "LIST" | 命令所属的 `group`，符合 [Command key specifications](https://redis.io/docs/reference/key-specs/)，可以在 [commands](https://github.com/tair-opensource/RedisShake/tree/v4/scripts/commands) 中查询每个命令的 `group` 字段 | -| CMD | string | "XGROUP-DELCONSUMER" | 命令的名称 | -| KEYS | table | \{"key1", "key2"\} | 命令的所有 Key | -| KEY_INDEXES | table | \{2, 4\} | 命令的所有 Key 在 `ARGV` 中的索引 | -| SLOTS | table | \{9189, 4998\} | 当前命令的所有 Key 所属的 [slot](https://redis.io/docs/reference/cluster-spec/#key-distribution-model) | -| ARGV | table | \{"mset", "key1", "value1", "key2", "value2"\} | 命令的所有参数 | - -### 函数 -* `shake.call(DB, ARGV)`：返回一个 Redis 命令，RedisShake 会将该命令写入目标端。 -* `shake.log(msg)`：打印日志。 +| DB | number | 1 | The `db` to which the command belongs | +| GROUP | string | "LIST" | The `group` to which the command belongs, conforming to [Command key specifications](https://redis.io/docs/reference/key-specs/). You can check the `group` field for each command in [commands](https://github.com/tair-opensource/RedisShake/tree/v4/scripts/commands) | +| CMD | string | "XGROUP-DELCONSUMER" | The name of the command | +| KEYS | table | {"key1", "key2"} | All keys of the command | +| KEY_INDEXES | table | {2, 4} | The indexes of all keys in `ARGV` | +| SLOTS | table | {9189, 4998} | The [slots](https://redis.io/docs/reference/cluster-spec/#key-distribution-model) to which all keys of the current command belong | +| ARGV | table | {"mset", "key1", "value1", "key2", "value2"} | All parameters of the command | + +### Functions +* `shake.call(DB, ARGV)`: Returns a Redis command that RedisShake will write to the destination. +* `shake.log(msg)`: Prints logs. + +## Best Practices + + +### Filtering Keys + +```lua +local prefix = "user:" +local prefix_len = #prefix + +if string.sub(KEYS[1], 1, prefix_len) ~= prefix then + return +end + +shake.call(DB, ARGV) +``` + +The effect is to only write source data with keys starting with `user:` to the destination. This doesn't consider cases of multi-key commands like `mset`. + +### Filtering DB + +```lua +shake.log(DB) +if DB == 0 +then + return +end +shake.call(DB, ARGV) +``` + +The effect is to discard data from source `db 0` and write data from other `db`s to the destination. + + +### Filtering Certain Data Structures + +You can use the `GROUP` variable to determine the data structure type. Supported data structure types include: `STRING`, `LIST`, `SET`, `ZSET`, `HASH`, `SCRIPTING`, etc. + +#### Filtering Hash Type Data +```lua +if GROUP == "HASH" then + return +end +shake.call(DB, ARGV) +``` + +The effect is to discard `hash` type data from the source and write other data to the destination. + +#### Filtering [LUA Scripts](https://redis.io/docs/interact/programmability/eval-intro/) + +```lua +if GROUP == "SCRIPTING" then + return +end +shake.call(DB, ARGV) +``` + +The effect is to discard `lua` scripts from the source and write other data to the destination. This is common when synchronizing from master-slave to cluster, where there are LUA scripts not supported by the cluster. + +### Modifying Key Prefixes + +```lua +local prefix_old = "prefix_old_" +local prefix_new = "prefix_new_" + +shake.log("old=" .. table.concat(ARGV, " ")) + +for i, index in ipairs(KEY_INDEXES) do + local key = ARGV[index] + if string.sub(key, 1, #prefix_old) == prefix_old then + ARGV[index] = prefix_new .. string.sub(key, #prefix_old + 1) + end +end + +shake.log("new=" .. table.concat(ARGV, " ")) +shake.call(DB, ARGV) +``` +The effect is to write the source key `prefix_old_key` to the destination key `prefix_new_key`. + +### Swapping DBs + +```lua +local db1 = 1 +local db2 = 2 + +if DB == db1 then + DB = db2 +elseif DB == db2 then + DB = db1 +end +shake.call(DB, ARGV) +``` + +The effect is to write source `db 1` to destination `db 2`, write source `db 2` to destination `db 1`, and leave other `db`s unchanged. \ No newline at end of file diff --git a/docs/src/en/guide/introduction.md b/docs/src/en/guide/introduction.md index 40e13675..8db56284 100644 --- a/docs/src/en/guide/introduction.md +++ b/docs/src/en/guide/introduction.md @@ -32,7 +32,7 @@ Versions (configurations are not interchangeable between different versions): - The [RedisShake 2.x](https://github.com/tair-opensource/RedisShake/tree/v2) version brought a series of improvements and updates, enhancing its overall stability and performance. - The [RedisShake 3.x](https://github.com/tair-opensource/RedisShake/tree/v3) version was a significant milestone, with the entire codebase being completely rewritten and optimized for better efficiency and availability. -- The [RedisShake 4.x](https://github.com/tair-opensource/RedisShake/tree/v4) version further enhanced features related to the [Reader](../reader/scan_reader.md), configuration, observability, and [function](../function/introduction.md). +- The [RedisShake 4.x](https://github.com/tair-opensource/RedisShake/tree/v4) version further enhanced features related to the [Reader](../reader/scan_reader.md), configuration, observability, and `function`. ## License diff --git a/docs/src/zh/filter/function.md b/docs/src/zh/filter/function.md index 52676f7f..069ad562 100644 --- a/docs/src/zh/filter/function.md +++ b/docs/src/zh/filter/function.md @@ -33,8 +33,6 @@ address = "127.0.0.1:6380" 除了 `DB`，还有其他信息如 `KEYS`、`ARGV`、`SLOTS`、`GROUP` 等，可供调用的函数有 `shake.log` 和 `shake.call`，具体请参考 [function API](#function-api)。 -关于更多的示例，可以参考 [最佳实践](./best_practices.md)。 - ## function API ### 变量 diff --git a/docs/src/zh/guide/introduction.md b/docs/src/zh/guide/introduction.md index feaaee96..4367471c 100644 --- a/docs/src/zh/guide/introduction.md +++ b/docs/src/zh/guide/introduction.md @@ -36,7 +36,7 @@ RedisShake 是阿里云 [Tair 团队](https://github.com/tair-opensource) - [RedisShake 2.x](https://github.com/tair-opensource/RedisShake/tree/v2) 版本带来了一系列的改进和更新，提高了其整体稳定性和性能。 - [RedisShake 3.x](https://github.com/tair-opensource/RedisShake/tree/v3) 版本是一个重要的里程碑，整个代码库被完全重写和优化，具有更好的效率和可用性。 - [RedisShake 4.x](https://github.com/tair-opensource/RedisShake/tree/v4) 版本 - ，进一步增强了 [Reader](../reader/scan_reader.md)、配置、可观察性和 [function](../function/introduction.md) 相关的特性。 + ，进一步增强了 [Reader](../reader/scan_reader.md)、配置、可观察性和 function 相关的特性。 ## 许可证