Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Segment replication stats APIs to support pull based architecture #15534

Open
Tracked by #15306
mch2 opened this issue Aug 30, 2024 · 1 comment
Open
Tracked by #15306

Update Segment replication stats APIs to support pull based architecture #15534

mch2 opened this issue Aug 30, 2024 · 1 comment
Labels
v2.18.0 Issues and PRs related to version 2.18.0

Comments

@mch2
Copy link
Member

mch2 commented Aug 30, 2024

With #4577 replicas will sync directly with their source of replication rather than pushing updates from the primary.

Today stats are collected at the primary level to support APIs and enforce backpressure. With the rw split we will ensure primaries do not have any direct communication with replicas and will not be able to collect these stats.

To fix this we can update our stats APIs to fetch the latest checkpoint from each replica directly and compute the required stats at the coordinator, eliminating the need for primaries to capture these stats.

@mch2 mch2 changed the title Update Segment replication stats APIs and backpressure. Update Segment replication stats APIs. Aug 30, 2024
@mch2 mch2 changed the title Update Segment replication stats APIs. Update Segment replication stats APIs to support pull based architecture Aug 30, 2024
@mch2 mch2 added v2.18.0 Issues and PRs related to version 2.18.0 and removed untriaged labels Sep 16, 2024
@vinaykpud
Copy link

Hi @mch2 can I take this task?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v2.18.0 Issues and PRs related to version 2.18.0
Projects
Status: Todo
Development

No branches or pull requests

2 participants