Switch compactor to new BasicLifecycler with AutoForgetDelegate #1178
+137
−51
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
This PR switches the compactor ring from
Lifecycler
to the new pluggableBasicLifeCycler
, and addsAutoForgetDelegate
to automatically forget unhealthy entries.Notes:
Autoforget defaults to 2xHeartbeatTimeout=2M, which is a middle ground between other components with auto-forget loki ingester (1x) and cortex storegateway (10x). Would like feedback on if this should be exposed as a config option, and preferences on different default behavior, if any.
There is slightly more leg-work required to use
BasicLifecycler
such creating the KV client, and the newOnRingInstance*
callbacks. Considered locating parts of this intempo/pkg/ring
for reuse, but the numerous small differences in how each component interacts with the ring (ingester vs compactor, for example), make it seem not very beneficial. Can revisit later if/when more components are switched toBasicLifecycler
.There are a few other changes to make the compactor startup leverage existing ring functionality, for waiting for ring activation and topology stabilization.
Which issue(s) this PR fixes:
Fixes #1081
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]