Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not allow decommissioned node rejoining the cluster #8547

Merged
merged 6 commits into from
Feb 1, 2023

Conversation

mmaslankaprv
Copy link
Member

@mmaslankaprv mmaslankaprv commented Feb 1, 2023

Added a piece of logic that doesn't allow decommissioned node to rejoin the cluster after it is restarted. This way a node that was decommissioned will always require a data wipe to be able to join the cluster again. It will make restarting decommissioned safe.

Fixes: #8404
Fixes: #8362

Backports Required

  • none - not a bug fix
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v22.3.x
  • v22.2.x
  • v22.1.x

UX Changes

Release Notes

Improvements

  • Made it safe to restart decommissioned node without a worry that it would rejoin the cluster

Signed-off-by: Michal Maslanka <michal@redpanda.com>
When new simple configuration is replicated after leaving joint
consensus we still may send a request to follower that is going to be
removed after the transition from joint to simple state. This way there
is a chance that the follower will receive an information about it being
removed from the cluster. The change introduced here is just an
optimization and it does not influence correctness or configuration
change semantics. It just allows the removed follower to discover its
removal.

The last configuration change  delivery isn't guaranteed and it may
happen that the follower will not receive it so we should not treat it
as reliable mechanism to discover removal.

Signed-off-by: Michal Maslanka <michal@redpanda.com>
Signed-off-by: Michal Maslanka <michal@redpanda.com>
Added a piece of logic that doesn't allow decommissioned node to rejoin
the cluster after it is restarted. This way a node that was
decommissioned will always require a data wipe to be able to join the
cluster again. It will make restarting decommissioned safe.

Fixes: redpanda-data#8404

Signed-off-by: Michal Maslanka <michal@redpanda.com>
…arts

When Redpanda is started it may be the case that it will not be able to
join the cluster but its process can still be up and running. To test
this behavior we may need to start Redpanda without waiting for it to be
fully operational.

Signed-off-by: Michal Maslanka <michal@redpanda.com>
Signed-off-by: Michal Maslanka <michal@redpanda.com>
@mmaslankaprv mmaslankaprv changed the title Do not allow rejoins Do not allow decommissioned node rejoining the cluster Feb 1, 2023
@mmaslankaprv
Copy link
Member Author

/backport v22.3.x

@vshtokman
Copy link
Contributor

/backport v22.2.x

@vbotbuildovich
Copy link
Collaborator

Failed to run cherry-pick command. I executed the below command:

git cherry-pick -x 9db0e31126aea87e2bb0a261b0dd694b369f56a6 76a9c4e525ef69b04cd24ba1825b9e7f21d14f4c 313e471a6e1d8ad2e0dc6a60cb3b9cdded31a003 fbe17b40c2eb3e8a821fe9cb6343894df95c5090 a5a1b8309b17d187882b5d07d6821ec7c13c26a1 6e568613c3fc0a4462903a2551e676a796afda9c

Workflow run logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants