Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blocklist never prunes missing tenants #285

Closed
joe-elliott opened this issue Oct 29, 2020 · 1 comment · Fixed by #314
Closed

Blocklist never prunes missing tenants #285

joe-elliott opened this issue Oct 29, 2020 · 1 comment · Fixed by #314

Comments

@joe-elliott
Copy link
Member

Describe the bug
If a tenant disappears (say by deleting all of its blocks manually from the backend) the internal in memory blocklist will never be updated to indicate this.

To Reproduce
Steps to reproduce the behavior:

  1. Send spans to tempo
  2. Let some blocks build up
  3. Stop sending spans to tempo
  4. Delete the tenant level folder on the backend
  5. Note that the tempodb_blocklist_length value doesn't go to 0.

Expected behavior
Blocks should be pruned from the in memory blocklist.

Environment:

Additional Context
This is b/c the pollBlocklist method in tempodb updates the blocklist of any tenant it finds, but if it fails to find the tenant it does nothing.

@dgzlopes
Copy link
Member

dgzlopes commented Nov 2, 2020

Hey :)

The blocks should be pruned from the in-memory compactedBlockLists too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants