Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revision auto compaction should have configurable timer instead of fixed 5 minute timer #18471

Open
jrcichra opened this issue Aug 19, 2024 · 4 comments
Open

Revision auto compaction should have configurable timer instead of fixed 5 minute timer #18471

jrcichra opened this issue Aug 19, 2024 · 4 comments
Labels
stage/triaged type/feature

Comments

@jrcichra
Copy link

What would you like to be added?

Today the revision auto compaction has a constant 5 minute compaction. Change this to be a command-line / environment variable configurable option.

Why is this needed?

With enough key revisions per second, the constant 5 minute interval on revision compaction isn't fast enough to keep up with growth. Consider making the interval configurable, which will override the default behavior of 5 minutes.
jrcichra added a commit to jrcichra/etcd that referenced this issue Aug 19, 2024
@jrcichra
etcdserver: add auto compaction interval option
480bd92 
With enough key revisions per second, the constant 5 minute interval on
revision compaction isn't fast enough to keep up with growth. Make the
interval configurable, which will override the default behavior.

Also pass the configuration through to periodic compaction for finer
control of when compaction happens based on time.

Fixes:
etcd-io#18471

Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
@jrcichra jrcichra mentioned this issue Aug 19, 2024
Open
@parthlaw
Copy link

I would like to work on this issue.

@jrcichra
Copy link
Author

I would like to work on this issue.

Hey @parthlaw , I have an implementation here I made along with the issue: #18472. Let me know what you think!

@parthlaw
Copy link

Thanks for the update @jrcichra . I am looking into the implementation. This is a great learning opportunity for me—Thanks for the guidance!

jrcichra added a commit to jrcichra/etcd that referenced this issue Aug 21, 2024
@jrcichra
etcdserver: add auto compaction interval option
5c0fda1 
With enough key revisions per second, the constant 5 minute interval on
revision compaction isn't fast enough to keep up with growth. Make the
interval configurable, which will override the default behavior.

Also pass the configuration through to periodic compaction for finer
control of when compaction happens based on time.

Fixes:
etcd-io#18471

Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
jrcichra added a commit to jrcichra/etcd that referenced this issue Aug 21, 2024
@jrcichra
etcdserver: add auto compaction interval option
28c3f6d 
With enough key revisions per second, the constant 5 minute interval on
revision compaction isn't fast enough to keep up with growth. Make the
interval configurable, which will override the default behavior.

Also pass the configuration through to periodic compaction for finer
control of when compaction happens based on time.

Fixes:
etcd-io#18471

Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
jrcichra added a commit to jrcichra/etcd that referenced this issue Aug 21, 2024
@jrcichra
etcdserver: add auto compaction interval option
523e17e 
With enough key revisions per second, the constant 5 minute interval on
revision compaction isn't fast enough to keep up with growth. Make the
interval configurable, which will override the default behavior.

Also pass the configuration through to periodic compaction for finer
control of when compaction happens based on time.

Fixes:
etcd-io#18471

Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
jrcichra added a commit to jrcichra/etcd that referenced this issue Aug 21, 2024
@jrcichra
etcdserver: add auto compaction interval option
9437d43 
With enough key revisions per second, the constant 5 minute interval on
revision compaction isn't fast enough to keep up with growth. Make the
interval configurable, which will override the default behavior.

Also pass the configuration through to periodic compaction for finer
control of when compaction happens based on time.

Fixes:
etcd-io#18471

Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
jrcichra added a commit to jrcichra/etcd that referenced this issue Aug 21, 2024
@jrcichra
etcdserver: add auto compaction interval option
5167a62 
With enough key revisions per second, the constant 5 minute interval on
revision compaction isn't fast enough to keep up with growth. Make the
interval configurable, which will override the default behavior.

Also pass the configuration through to periodic compaction for finer
control of when compaction happens based on time.

Fixes:
etcd-io#18471

Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
jrcichra added a commit to jrcichra/etcd that referenced this issue Aug 26, 2024
@jrcichra
etcdserver: add auto compaction interval option
3a824ea 
With enough key revisions per second, the constant 5 minute interval on
revision compaction isn't fast enough to keep up with growth. Make the
interval configurable, which will override the default behavior.

Also pass the configuration through to periodic compaction for finer
control of when compaction happens based on time.

Fixes:
etcd-io#18471

Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
@ivanvc
Copy link
Member

ivanvc commented Aug 29, 2024

Discussed during the fortnightly triage meeting. @moficodes to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stage/triaged type/feature
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ivanvc @jrcichra @parthlaw