Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

是否考虑过多线程接收binlog以优化性能 #4285

Closed
Daviedme opened this issue Jul 7, 2022 · 1 comment
Closed

是否考虑过多线程接收binlog以优化性能 #4285

Daviedme opened this issue Jul 7, 2022 · 1 comment

Comments

@Daviedme
Copy link

Daviedme commented Jul 7, 2022

问题

看到项目在1.1.x 版本做了很大的性能优化,整体性能提升了150%( #726 参考: Performance
但还是有个疑问,目前来看,单线程接收binlog的上限性能为117MB/s,不知道你们有没有考虑或者尝试实现过用多个线程来接收binlog日志,以此来突破117MB/s这个瓶颈。理论上来说,mysql协议支持通过指定的位点来获取binlog日志,我们假设mysql server写入日志的速度高于单线程接收binlog日志的能力,那么我们可以通过多个线程并行的获取binlog(每个线程获取一段),然后再并行处理,最后在一个线程将最终的结果合并,做有序化处理。当然,这只是我个人的想法,想知道你们有没有做过这方面的尝试或者对这种思路可行性的看法是怎么样的?

@agapple
Copy link
Member

agapple commented Aug 24, 2022

有多线程的处理,117MB/s是在千兆网卡下跑的,如果有万兆网卡机器可以跑下看看

@agapple agapple closed this as completed Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants