Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support per-fragment vnode count #18444

Open
wants to merge 22 commits into
base: main
Choose a base branch
from

Conversation

BugenZhao
Copy link
Member

@BugenZhao BugenZhao commented Sep 6, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This is a progress towards #15900.

In previous PRs, we mainly focused on refactoring codes from different modules to avoid directly referencing the global constant of VirtualNode::COUNT, but infer it from the context or directly accept it as a parameter.

In this PR, we introduce the vnode_count field for Table and StreamFragmentGraph, update the logic for its maintenance. Many TODOs of directly using VirtualNode::COUNT are fixed by reading the values from Fragment or TableDesc.

Note that there's still no user-facing changes in this PR, but we are now quite close to that. However, there are still some unaddressed TODOs that may require changes in behavior. We leave this in future PRs.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@BugenZhao BugenZhao force-pushed the bz/var-vnode-user-facing-local branch from 3292eed to c4772c9 Compare September 9, 2024 06:02
@BugenZhao BugenZhao force-pushed the bz/var-vnode-user-facing-local branch from c4772c9 to 166d631 Compare September 9, 2024 09:03
@BugenZhao BugenZhao force-pushed the bz/var-vnode-user-facing-local branch from 166d631 to 31e2d77 Compare September 9, 2024 09:10
@BugenZhao BugenZhao force-pushed the bz/var-vnode-user-facing-local branch 3 times, most recently from a5afcdf to fce41b8 Compare September 11, 2024 06:17
@BugenZhao BugenZhao changed the title feat: specify vnode count for jobs via session config feat: support per-fragment vnode count Sep 11, 2024
@BugenZhao BugenZhao marked this pull request as ready for review September 11, 2024 06:55
@BugenZhao BugenZhao force-pushed the bz/var-vnode-user-facing-local branch 2 times, most recently from fa288c5 to e4ef75f Compare September 12, 2024 08:51
@BugenZhao BugenZhao force-pushed the bz/var-vnode-user-facing-local branch 2 times, most recently from 640e7d8 to fe25c89 Compare September 13, 2024 06:13
Base automatically changed from bz/var-vnode-storage to main September 20, 2024 10:09
@graphite-app graphite-app bot requested a review from a team September 20, 2024 10:09
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant