Skip to content

Commit 5a9802c

Browse files
author
Cary Huang
committed
updates
1 parent f81bc67 commit 5a9802c

File tree

6 files changed

+235
-2
lines changed

6 files changed

+235
-2
lines changed

‎docs/en/monitoring/jmx_exporter.md

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# JMX Exporter
2+
3+
## What is JMX Exporter
4+
5+
JMX Exporter (also known as the JMX Prometheus Java Agent) is a Java agent that exposes JMX metrics of a Java application in a format Prometheus can scrape and monitor. It is a tool created by the Prometheus community that allows:
6+
7+
* Access JVM internal metrics (like memory usage, thread count, GC stats, etc.)
8+
* Expose custom application metrics exposed via MBeans
9+
* Export these metrics via an HTTP endpoint (e.g., http://localhost:9404/metrics)
10+
* Integrate Java applications (like Kafka, Cassandra, or your own apps) into a Prometheus monitoring setupx
11+
12+
This tool unlocks Prometheus + Graphana based monitoring for a SynchDB Connector
13+
14+
![img](/images/prom.png)
15+
16+
## Obtain JMX Exporter
17+
18+
Pre-compiled JMX Exporter (.jar) are available on official [JMX Exporter release page](https://github.com/prometheus/jmx_exporter/releases). We are interested in the "jmx_prometheus_javaagent" tool such as:
19+
20+
```
21+
jmx_prometheus_javaagent-1.3.0.jar
22+
23+
```
24+
25+
Please download it to the same machine where SynchDB connector is running.
26+
27+
## Write a JMX Exporter Conf File
28+
29+
JMX exporter requires a config file to define the behavior of exposing metrics. Below is a basic configuration template to get started.
30+
31+
```
32+
startDelaySeconds: 0
33+
ssl: false
34+
lowercaseOutputName: true
35+
lowercaseOutputLabelNames: true
36+
37+
rules:
38+
- pattern: ".*"
39+
40+
```
41+
42+
Refer to the [official prometheus documentation](https://prometheus.github.io/jmx_exporter/1.3.0/http-mode/rules/) for more advanced configuration parameters and their usage.
43+
44+
## Configure JMX Exporter to a SynchDB Connector
45+
46+
The synchdb_add_jmx_exporter_conninfo() and synchdb_del_jmx_exporter_conninfo() functions adds or deletes JMX exporter configuration to or from an existing connector. This enables runtime monitoring and diagnostics via tools like Prometheus and Graphana.
47+
48+
**Function Signature**
49+
50+
```
51+
synchdb_add_jmx_exporter_conninfo(
52+
connector_name TEXT,
53+
exporter_jar_path TEXT,
54+
exporter_port INTEGER,
55+
config_file_path TEXT
56+
);
57+
58+
synchdb_del_jmx_exporter_conninfo(
59+
name text
60+
)
61+
62+
```
63+
64+
| Parameter | Type | Description |
65+
| ------------------- | ------ | --------------------------------------------------------------------------- |
66+
| `connector_name` | `TEXT` | Name of the existing connector you want to attach the JMX Exporter to. |
67+
| `exporter_jar_path` | `TEXT` | Absolute path to the `jmx_prometheus_javaagent.jar` file. |
68+
| `exporter_port` | `INT` | Port on which the JMX Exporter HTTP server will expose metrics (e.g. 9404). |
69+
| `config_file_path` | `TEXT` | Path to the JMX Exporter's YAML configuration file we defined above. |
70+
71+
72+
```sql
73+
SELECT synchdb_add_jmx_exporter_conninfo(
74+
'mysqlconn', -- existing connector name
75+
'/path/to/jmx_exporter/jar' -- path to JMX exporter java agent jar
76+
9404, -- JMX exporter running port
77+
'/path/to/jmx/conf'); -- path to JMX exporter conf file
78+
79+
```
80+
81+
## Obtain Metrics via HTTP
82+
83+
when the connector starts with JMX exporter settings, it will expose metrics at:
84+
85+
```
86+
http://<host>:9404/metrics
87+
```
88+
89+
we can test it by:
90+
91+
```
92+
curl http://<host>:9404/metrics
93+
94+
# HELP debezium_mysql_connector_metrics_binlogposition debezium.mysql:name=null,type=connector-metrics,attribute=BinlogPosition
95+
# TYPE debezium_mysql_connector_metrics_binlogposition untyped
96+
debezium_mysql_connector_metrics_binlogposition{context="streaming",server="synchdb-connector"} 1500.0
97+
# HELP debezium_mysql_connector_metrics_changesapplied debezium.mysql:name=null,type=connector-metrics,attribute=ChangesApplied
98+
# TYPE debezium_mysql_connector_metrics_changesapplied untyped
99+
debezium_mysql_connector_metrics_changesapplied{context="schema-history",server="synchdb-connector"} 39.0
100+
# HELP debezium_mysql_connector_metrics_changesrecovered debezium.mysql:name=null,type=connector-metrics,attribute=ChangesRecovered
101+
# TYPE debezium_mysql_connector_metrics_changesrecovered untyped
102+
debezium_mysql_connector_metrics_changesrecovered{context="schema-history",server="synchdb-connector"} 26.0
103+
# HELP debezium_mysql_connector_metrics_connected debezium.mysql:name=null,type=connector-metrics,attribute=Connected
104+
# TYPE debezium_mysql_connector_metrics_connected untyped
105+
debezium_mysql_connector_metrics_connected{context="streaming",server="synchdb-connector"} 1.0
106+
107+
...
108+
...
109+
...
110+
```
111+
112+
## Prometheus and Graphana
113+
114+
Once we have confirmed the metrics can be obtained via HTTP, then we can configure this endpoint to prometheus system and have it to `scrape` them all. Then we can create a prometheus data source in graphana and create a dashboard out of it. Please refer to prometheus and graphana [tutorial](https://grafana.com/docs/grafana/latest/getting-started/get-started-grafana-prometheus/) on how to do so.

‎docs/en/monitoring/jmx_monitor.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Debezium connectors expose metrics via the MBean name for the connector with MBe
1010

1111
## **Enable JMX on a Connector with synchdb_add_jmx_conninfo() or disable with synchdb_del_jmx_conninfo()**
1212

13-
The synchdb_add_jmx_conninfo() function adds JMX monitoring configuration to an existing connector. This enables runtime monitoring and diagnostics via tools like JConsole or Prometheus JMX Exporter.synchdb_del_jmx_conninfo() can be used to disable JMX.
13+
The synchdb_add_jmx_conninfo() and synchdb_del_jmx_conninfo() functions adds or deletes JMX monitoring configuration to or from an existing connector. This enables runtime monitoring and diagnostics via tools like JConsole.
1414

1515
**Function Signature**
1616

‎docs/images/prom.png

30.2 KB
Loading

‎docs/zh/monitoring/jmx_exporter.md

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# JMX Exporter
2+
3+
## 什么是 JMX Exporter
4+
5+
JMX Exporter（也称为 JMX Prometheus Java Agent）是一个 Java 代理，它以 Prometheus 可以抓取和监控的格式公开 Java 应用程序的 JMX 指标。它是由 Prometheus 社区创建的一款工具，支持以下功能：
6+
7+
* 访问 JVM 内部指标（例如内存使用情况、线程数、GC 统计信息等）
8+
* 通过 MBean 公开自定义应用程序指标
9+
* 通过 HTTP 端点导出这些指标（例如 http://localhost:9404/metrics）
10+
* 将 Java 应用程序（例如 Kafka、Cassandra 或您自己的应用程序）集成到 Prometheus 监控设置中
11+
12+
此工具可解锁基于 Prometheus + Graphana 的 SynchDB 连接器监控功能
13+
14+
![img](/images/prom.png)
15+
16+
## 获取 JMX 导出器
17+
18+
预编译的 JMX 导出器 (.jar) 可在官方 [JMX 导出器发布页面](https://github.com/prometheus/jmx_exporter/releases) 获取。我们感兴趣的是“jmx_prometheus_javaagent”工具，例如：
19+
20+
```
21+
jmx_prometheus_javaagent-1.3.0.jar
22+
23+
```
24+
25+
请将其下载到运行 SynchDB 连接器的同一台机器上。
26+
27+
## 编写 JMX 导出器配置文件
28+
29+
JMX 导出器需要一个配置文件来定义暴露指标的行为。以下是一个基本的配置模板，可帮助您入门。
30+
31+
```
32+
startDelaySeconds: 0
33+
ssl: false
34+
lowercaseOutputName: true
35+
lowercaseOutputLabelNames: true
36+
37+
rules:
38+
- pattern: ".*"
39+
40+
```
41+
42+
有关更多高级配置参数及其使用方法，请参阅[官方 prometheus 文档](https://prometheus.github.io/jmx_exporter/1.3.0/http-mode/rules/)。
43+
44+
## 将 JMX 导出器配置到 SynchDB 连接器
45+
46+
synchdb_add_jmx_exporter_conninfo() 和 synchdb_del_jmx_exporter_conninfo() 函数用于向现有连接器添加或删除 JMX 导出器配置。这支持通过 Prometheus 和 Graphana 等工具进行运行时监控和诊断。
47+
48+
**函数签名**
49+
50+
```
51+
synchdb_add_jmx_exporter_conninfo(
52+
connector_name TEXT,
53+
exporter_jar_path TEXT,
54+
exporter_port INTEGER,
55+
config_file_path TEXT
56+
);
57+
58+
synchdb_del_jmx_exporter_conninfo(
59+
name text
60+
)
61+
62+
```
63+
64+
| Parameter | Type | Description |
65+
| ------------------- | ------ | --------------------------------------------------------------------------- |
66+
| `connector_name` | `TEXT` | Name of the existing connector you want to attach the JMX Exporter to. |
67+
| `exporter_jar_path` | `TEXT` | Absolute path to the `jmx_prometheus_javaagent.jar` file. |
68+
| `exporter_port` | `INT` | Port on which the JMX Exporter HTTP server will expose metrics (e.g. 9404). |
69+
| `config_file_path` | `TEXT` | Path to the JMX Exporter's YAML configuration file we defined above. |
70+
71+
```sql
72+
SELECT synchdb_add_jmx_exporter_conninfo(
73+
'mysqlconn', -- existing connector name
74+
'/path/to/jmx_exporter/jar' -- path to JMX exporter java agent jar
75+
9404, -- JMX exporter running port
76+
'/path/to/jmx/conf'); -- path to JMX exporter conf file
77+
78+
```
79+
80+
## 通过 HTTP 获取指标
81+
82+
当连接器使用 JMX 导出器设置启动时，它将在以下位置公开指标：
83+
84+
```
85+
http://<host>:9404/metrics
86+
```
87+
88+
我们可以通过以下方式测试：
89+
```
90+
curl http://<host>:9404/metrics
91+
92+
# HELP debezium_mysql_connector_metrics_binlogposition debezium.mysql:name=null,type=connector-metrics,attribute=BinlogPosition
93+
# TYPE debezium_mysql_connector_metrics_binlogposition untyped
94+
debezium_mysql_connector_metrics_binlogposition{context="streaming",server="synchdb-connector"} 1500.0
95+
# HELP debezium_mysql_connector_metrics_changesapplied debezium.mysql:name=null,type=connector-metrics,attribute=ChangesApplied
96+
# TYPE debezium_mysql_connector_metrics_changesapplied untyped
97+
debezium_mysql_connector_metrics_changesapplied{context="schema-history",server="synchdb-connector"} 39.0
98+
# HELP debezium_mysql_connector_metrics_changesrecovered debezium.mysql:name=null,type=connector-metrics,attribute=ChangesRecovered
99+
# TYPE debezium_mysql_connector_metrics_changesrecovered untyped
100+
debezium_mysql_connector_metrics_changesrecovered{context="schema-history",server="synchdb-connector"} 26.0
101+
# HELP debezium_mysql_connector_metrics_connected debezium.mysql:name=null,type=connector-metrics,attribute=Connected
102+
# TYPE debezium_mysql_connector_metrics_connected untyped
103+
debezium_mysql_connector_metrics_connected{context="streaming",server="synchdb-connector"} 1.0
104+
105+
...
106+
...
107+
...
108+
```
109+
110+
## Prometheus 和 Graphana
111+
112+
一旦我们确认可以通过 HTTP 获取指标，就可以将此端点配置到 Prometheus 系统，并让其“抓取”所有指标。然后，我们可以在 graphana 中创建一个 Prometheus 数据源，并使用它创建一个仪表板。请参阅 Prometheus 和 graphana [教程](https://grafana.com/docs/grafana/latest/getting-started/get-started-grafana-prometheus/) 了解如何操作。

‎docs/zh/monitoring/jmx_monitor.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ JMX，即 Java 管理扩展，是一种 Java 技术，提供用于管理和监
88
Debezium 连接器通过连接器的 MBean 名称（MBean 标签等于连接器名称）公开指标。这些指标特定于每个连接器实例，提供有关连接器快照、流式传输和模式历史记录进程行为的数据。
99

1010
## **使用 synchdb_add_jmx_conninfo() 在连接器上启用 JMX，或使用 synchdb_del_jmx_conninfo() 禁用**
11-
synchdb_add_jmx_conninfo() 函数将 JMX 监控配置添加到现有连接器。这可以通过 JConsole 或 Prometheus JMX Exporter 等工具启用运行时监控和诊断。synchdb_del_jmx_conninfo() 可用于禁用 JMX。
11+
synchdb_add_jmx_conninfo() 和 synchdb_del_jmx_conninfo() 函数用于向现有连接器添加或删除 JMX 监控配置。这允许通过 JConsole 等工具进行运行时监控和诊断。
1212

1313
**函数签名**
1414

‎mkdocs.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,9 @@ plugins:
8181
Oracle CDC to PostgreSQL: CDC de Oracle a PostgreSQL
8282
Table Snapshot and Re-snapshot: Instantánea de tabla y nueva instantánea
8383
JMX Monitor Settings: Configuración del monitor JMX
84+
JVM Memory Usage: Uso de memoria de JVM
85+
Attribute View: Vista de atributos
86+
JMX Exporter Settings: Configuración del exportador JMX
8487
- locale: zh
8588
name: 中文
8689
site_name: SynchDB 文档
@@ -121,6 +124,9 @@ plugins:
121124
Oracle CDC to PostgreSQL: Oracle CDC 到 PostgreSQL
122125
Table Snapshot and Re-snapshot: 表快照和重新快照
123126
JMX Monitor Settings: JMX 监视器设置
127+
JVM Memory Usage: JVM内存使用情况
128+
Attribute View: 属性视图
129+
JMX Exporter Settings: JMX exporter 设置：
124130
- mike:
125131
version_selector: true
126132
css_dir: css
@@ -153,6 +159,7 @@ nav:
153159
- Attribute View: monitoring/attr_view.md
154160
- JVM Memory Usage: monitoring/jvm_mem.md
155161
- JMX Monitor Settings: monitoring/jmx_monitor.md
162+
- JMX Exporter Settings: monitoring/jmx_exporter.md
156163
- Tutorial:
157164
- MySQL CDC to PostgreSQL: tutorial/mysql_cdc_to_postgresql.md
158165
- SQL Server CDC to PostgreSQL: tutorial/sqlserver_cdc_to_postgresql.md

0 commit comments

Comments
 (0)