Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate transactions into compaction #4824

Closed
1 of 2 tasks
rystsov opened this issue May 19, 2022 · 2 comments · Fixed by #6664
Closed
1 of 2 tasks

Integrate transactions into compaction #4824

rystsov opened this issue May 19, 2022 · 2 comments · Fixed by #6664
Assignees
@bharathv @rystsov
Labels
area/transactions kind/enhance New feature or request

Comments

@rystsov
Copy link
Contributor

rystsov commented May 19, 2022 •
edited by ajfabbri
Loading

  • rm_stm should implement a compaction_controller interface, register within consensus / log and use LSO to block the compaction
  • compaction_controller interface should contain a method to check if a given record is aborted or committed. compaction logic should treat the aborted records as if they belong to a separate batch type to prevent an aborted record overwriting a committed record
@rystsov rystsov added the kind/enhance New feature or request label May 19, 2022
@ajfabbri ajfabbri self-assigned this Jun 29, 2022
@rystsov rystsov changed the title Integrate aborted transactions into compaction Integrate transactions into compaction Jun 29, 2022
@rystsov rystsov mentioned this issue Sep 6, 2022
Merged
@mmedenjak
Copy link
Contributor

It's possible that the second point in the issue description is a part of #4826. To be confirmed with @mmaslankaprv @bharathv and @rystsov

@rystsov
Copy link
Contributor Author

rystsov commented Sep 26, 2022

it is related but isn't a duplicate, the question whether or not the the aborted records should be deleted (4826) is orthogonal to ignoring the aborted records during compaction (4824)

@bharathv bharathv mentioned this issue Oct 10, 2022
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bharathv bharathv

@rystsov rystsov

Labels
area/transactions kind/enhance New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[transactions]: Support transactions on compaction enabled topics bharathv/redpanda
4 participants
@bharathv @ajfabbri @rystsov @mmedenjak