Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

canal-1.1.6启动后instance一直刷日志Skipping unrecognized binlog event Unknown type:41 #4225

Closed
Eat-Sleep-HitBean opened this issue Jun 13, 2022 · 2 comments
Assignees
Labels
Milestone

Comments

@Eat-Sleep-HitBean
Copy link

Eat-Sleep-HitBean commented Jun 13, 2022

canal版本:1.1.6

mysql版本:8.0.29

日志信息

2022-06-13 03:40:54.806 [MultiStageCoprocessor-other-example-0] WARN  com.taobao.tddl.dbsync.binlog.LogDecoder - Skipping unrecognized binlog event Unknown type:41 from: binlog.000005:114150465
2022-06-13 03:41:39.967 [MultiStageCoprocessor-other-example-0] WARN  com.taobao.tddl.dbsync.binlog.LogDecoder - Skipping unrecognized binlog event Unknown type:41 from: binlog.000005:114155694

查询MySQL事件信息

show binlog events in 'binlog.000005' from 114155694 limit 5;
Log_name Pos Event_type Server_id End_log_pos Info
binlog.000005 114155694 Anonymous_Gtid 1 114155773 SET @@SESSION.GTID_NEXT= 'ANONYMOUS'
binlog.000005 114155773 Query 1 114155856 BEGIN
binlog.000005 114155856 Table_map 1 114155931 table_id: 107 (tag.qrtz_scheduler_state)
binlog.000005 114155931 Update_rows 1 114156115 table_id: 107 flags: STMT_END_F
binlog.000005 114156115 Xid 1 114156146 COMMIT /* xid=2835988 */

排查过程

查看LogEvent类中的代码,存在对事件类型41的定义

    /* mysql 8.0.20 */
    public static final int    TRANSACTION_PAYLOAD_EVENT                = 40;

    public static final int    MYSQL_ENUM_END_EVENT                     = 41;

查询MySQL手册:https://dev.mysql.com/doc/internals/en/event-classes-and-types.html
手册中有类似名称的事件类型(ENUM_END_EVENT),但是没有解释具体含义

  PREVIOUS_GTIDS_LOG_EVENT= 35, 
  ENUM_END_EVENT 

疑问

项目本身使用了quartz框架,猜测是quartz框架自动查询数据库表的时候,MySQL生成了一个41类型的事件,事件内容为分配下一个全局事务ID为“匿名的”,本身不清楚MySQL这块儿的处理逻辑,这个日志输出可以忽略吗?

@jimmy384
Copy link

我之前canal-1.1.5 + mysql-8.0.28也遇到了, 只是建了schema,没有做任何sql操作,也会时不时出现这个警告日志的
我是通过改源码编译替换jar包临时规避了这个问题,应该不影响同步

@agapple agapple self-assigned this Sep 1, 2022
@agapple agapple added bug feature and removed bug labels Sep 1, 2022
@agapple agapple added this to the v1.1.7 milestone Sep 1, 2022
@agapple
Copy link
Member

agapple commented Sep 1, 2022

@agapple agapple closed this as completed Sep 1, 2022
rewerma added a commit that referenced this issue Dec 6, 2022
* optimize YAML config loader (#4332)

* fix bug BASE TABLE as table name (#4217)

* fix issues#4328 (#4329)

* docs: add nodejs canal client support (#4260)

Co-authored-by: zhangxunwei <zhangxunwei@dxy.cn>

* fix destination not encoded (#4279)

* 修复Canal指定时间戳启动失效,总是从最新的点位开始同步问题 (#4348)

* 支持用户自定义的CanalAlarmHandler

* RowsLogEvent增加对TableMapLogEvent判空检查,防止NPE异常

* 修复Canal指定时间戳启动失效,总是从最新的点位开始同步问题, issue: #4347

Co-authored-by: 云时 <mingya.wmy@alibaba-inc.com>

* fixed 4334 , support jdk8/jdk11

* fixed issue #4266 , typo

* fixed issue #4243 , support auto register for cluster = null

* fixed issue #4225 , support mysql version >= 8.0.26 heartbeat v2

* fixed issue #4308 , support query_log_event for maraiadb 10.10.1

* ignore compression event

* support jdk11

* support druid 1.2.12

* fixed mariadb 10.x

* sync canal-template.properties

* update fastjson & druid version (#4406)

* 修复升级2.0.4导致兼容的问题

* update druid & fastjson version

* update fastjson version

* update druid & fastjson version

* meta.dat文件数据丢失 (#4397)

* update fastjson & druid version (#4438)

* 局部变量线程安全,优先使用StringBuilder替换StringBuffer (#4472)

Co-authored-by: 夏亮 <xialiang@newerabc.com>

* 1. CanalController stop 需要同时将 embededCanalServer.stop (#4477)

2. ServerRunningMonitor 线程池未正常回收,线程池管理与 start/stop保持一致

* vuln-fix: Use HTTPS instead of HTTP to resolve dependencies (#4437)

This fixes a security vulnerability in this project where the `build.gradle`
files were configuring Gradle to resolve dependencies over HTTP instead of
HTTPS.

Weakness: CWE-829: Inclusion of Functionality from Untrusted Control Sphere
Severity: High
CVSSS: 8.1
Detection: OpenRewrite

Reported-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>

Bug-tracker: JLLeitschuh/security-research#9


Co-authored-by: Moderne <team@moderne.io>

Co-authored-by: Moderne <team@moderne.io>

* fix_ETL同步mysql关键字报错 (#4346)

Co-authored-by: foleyang <foleyang@cogobuy.com>

* optimize code

* use compact BigDecimal

* performance optimize , 1. cache string names 2. cache Charset

* performance optimize ,cache integer/long valueof

* 测试类报错

Co-authored-by: gongchangyou <gongchangyou@gmail.com>
Co-authored-by: ChanaLii <316529035@qq.com>
Co-authored-by: zhangxunwei <zhangxunweia@gmail.com>
Co-authored-by: zhangxunwei <zhangxunwei@dxy.cn>
Co-authored-by: tianpeidong <39491687+tianpeidong@users.noreply.github.com>
Co-authored-by: dataccs <dataccs@163.com>
Co-authored-by: 云时 <mingya.wmy@alibaba-inc.com>
Co-authored-by: jianghang.loujh <jianghang.loujh@alibaba-inc.com>
Co-authored-by: 温绍锦 <shaojin.wensj@alibaba-inc.com>
Co-authored-by: noaso <noasoso@gmail.com>
Co-authored-by: HumanPassenger <40585855+HumanPassenger@users.noreply.github.com>
Co-authored-by: 夏亮 <xialiang@newerabc.com>
Co-authored-by: 华仔 <591327356@qq.com>
Co-authored-by: Jonathan Leitschuh <jonathan.leitschuh@gmail.com>
Co-authored-by: Moderne <team@moderne.io>
Co-authored-by: 杰锅不是锅 <1105568074@qq.com>
Co-authored-by: foleyang <foleyang@cogobuy.com>
rewerma added a commit that referenced this issue Dec 6, 2022
* 修复测试类报错 (#4516)

* optimize YAML config loader (#4332)

* fix bug BASE TABLE as table name (#4217)

* fix issues#4328 (#4329)

* docs: add nodejs canal client support (#4260)

Co-authored-by: zhangxunwei <zhangxunwei@dxy.cn>

* fix destination not encoded (#4279)

* 修复Canal指定时间戳启动失效,总是从最新的点位开始同步问题 (#4348)

* 支持用户自定义的CanalAlarmHandler

* RowsLogEvent增加对TableMapLogEvent判空检查,防止NPE异常

* 修复Canal指定时间戳启动失效,总是从最新的点位开始同步问题, issue: #4347

Co-authored-by: 云时 <mingya.wmy@alibaba-inc.com>

* fixed 4334 , support jdk8/jdk11

* fixed issue #4266 , typo

* fixed issue #4243 , support auto register for cluster = null

* fixed issue #4225 , support mysql version >= 8.0.26 heartbeat v2

* fixed issue #4308 , support query_log_event for maraiadb 10.10.1

* ignore compression event

* support jdk11

* support druid 1.2.12

* fixed mariadb 10.x

* sync canal-template.properties

* update fastjson & druid version (#4406)

* 修复升级2.0.4导致兼容的问题

* update druid & fastjson version

* update fastjson version

* update druid & fastjson version

* meta.dat文件数据丢失 (#4397)

* update fastjson & druid version (#4438)

* 局部变量线程安全,优先使用StringBuilder替换StringBuffer (#4472)

Co-authored-by: 夏亮 <xialiang@newerabc.com>

* 1. CanalController stop 需要同时将 embededCanalServer.stop (#4477)

2. ServerRunningMonitor 线程池未正常回收,线程池管理与 start/stop保持一致

* vuln-fix: Use HTTPS instead of HTTP to resolve dependencies (#4437)

This fixes a security vulnerability in this project where the `build.gradle`
files were configuring Gradle to resolve dependencies over HTTP instead of
HTTPS.

Weakness: CWE-829: Inclusion of Functionality from Untrusted Control Sphere
Severity: High
CVSSS: 8.1
Detection: OpenRewrite

Reported-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>

Bug-tracker: JLLeitschuh/security-research#9


Co-authored-by: Moderne <team@moderne.io>

Co-authored-by: Moderne <team@moderne.io>

* fix_ETL同步mysql关键字报错 (#4346)

Co-authored-by: foleyang <foleyang@cogobuy.com>

* optimize code

* use compact BigDecimal

* performance optimize , 1. cache string names 2. cache Charset

* performance optimize ,cache integer/long valueof

* 测试类报错

Co-authored-by: gongchangyou <gongchangyou@gmail.com>
Co-authored-by: ChanaLii <316529035@qq.com>
Co-authored-by: zhangxunwei <zhangxunweia@gmail.com>
Co-authored-by: zhangxunwei <zhangxunwei@dxy.cn>
Co-authored-by: tianpeidong <39491687+tianpeidong@users.noreply.github.com>
Co-authored-by: dataccs <dataccs@163.com>
Co-authored-by: 云时 <mingya.wmy@alibaba-inc.com>
Co-authored-by: jianghang.loujh <jianghang.loujh@alibaba-inc.com>
Co-authored-by: 温绍锦 <shaojin.wensj@alibaba-inc.com>
Co-authored-by: noaso <noasoso@gmail.com>
Co-authored-by: HumanPassenger <40585855+HumanPassenger@users.noreply.github.com>
Co-authored-by: 夏亮 <xialiang@newerabc.com>
Co-authored-by: 华仔 <591327356@qq.com>
Co-authored-by: Jonathan Leitschuh <jonathan.leitschuh@gmail.com>
Co-authored-by: Moderne <team@moderne.io>
Co-authored-by: 杰锅不是锅 <1105568074@qq.com>
Co-authored-by: foleyang <foleyang@cogobuy.com>

* Revert "修复测试类报错 (#4516)"

This reverts commit d899981.

Co-authored-by: gongchangyou <gongchangyou@gmail.com>
Co-authored-by: ChanaLii <316529035@qq.com>
Co-authored-by: zhangxunwei <zhangxunweia@gmail.com>
Co-authored-by: zhangxunwei <zhangxunwei@dxy.cn>
Co-authored-by: tianpeidong <39491687+tianpeidong@users.noreply.github.com>
Co-authored-by: dataccs <dataccs@163.com>
Co-authored-by: 云时 <mingya.wmy@alibaba-inc.com>
Co-authored-by: jianghang.loujh <jianghang.loujh@alibaba-inc.com>
Co-authored-by: 温绍锦 <shaojin.wensj@alibaba-inc.com>
Co-authored-by: noaso <noasoso@gmail.com>
Co-authored-by: HumanPassenger <40585855+HumanPassenger@users.noreply.github.com>
Co-authored-by: 夏亮 <xialiang@newerabc.com>
Co-authored-by: 华仔 <591327356@qq.com>
Co-authored-by: Jonathan Leitschuh <jonathan.leitschuh@gmail.com>
Co-authored-by: Moderne <team@moderne.io>
Co-authored-by: 杰锅不是锅 <1105568074@qq.com>
Co-authored-by: foleyang <foleyang@cogobuy.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants