-
Notifications
You must be signed in to change notification settings - Fork 9
TopicRebalance
Jimmy Bonds edited this page May 29, 2019
·
2 revisions
- Describe the existing topic, examine current Leaders and Replicas:
# > kafkactl describe topic -x my.topic.one
TOPIC PART OFFSET LEADER REPLICAS ISRs OFFLINE
my.topic.one 0 565060 5 [5 2 3] [3 2 5] []
my.topic.one 1 565346 4 [4 3 5] [5 3 4] []
my.topic.one 2 564768 1 [1 5 4] [5 4 1] []
my.topic.one 3 566025 2 [2 4 1] [1 2 4] []
my.topic.one 4 565027 3 [3 1 2] [1 3 2] []
my.topic.one 5 564926 5 [5 3 4] [5 4 3] []
my.topic.one 6 564827 4 [4 5 1] [5 1 4] []
my.topic.one 7 565435 1 [1 4 2] [4 2 1] []
my.topic.one 8 564624 2 [2 1 3] [1 3 2] []
my.topic.one 9 564809 3 [3 2 5] [5 2 3] []
my.topic.one 10 564917 5 [5 4 1] [5 4 1] []
my.topic.one 11 566616 4 [4 1 2] [1 4 2] []
my.topic.one 12 566058 1 [1 2 3] [2 1 3] []
my.topic.one 13 564785 2 [2 3 5] [5 2 3] []
my.topic.one 14 564372 3 [3 5 4] [5 4 3] []
- Initiate the rebalance across all available nodes. The new targeted Brokers will be output upon success:
# > kafkactl admin rebalance -x my.topic.one
TOPIC PARTITION BROKERS
my.topic.one 0 [1 8 9]
my.topic.one 1 [2 7 5]
my.topic.one 2 [3 8 9]
my.topic.one 3 [4 7 2]
my.topic.one 4 [5 4 1]
my.topic.one 5 [6 3 2]
my.topic.one 6 [7 6 1]
my.topic.one 7 [8 3 4]
my.topic.one 8 [9 6 7]
my.topic.one 9 [1 9 8]
my.topic.one 10 [2 5 7]
my.topic.one 11 [3 9 8]
my.topic.one 12 [4 6 3]
my.topic.one 13 [5 4 1]
my.topic.one 14 [6 5 2]
- Describe the topic again and note the activity. Get broker details and note the current migrating replicas::
# > kafkactl describe topic -x my.topic.one
TOPIC PART OFFSET LEADER REPLICAS ISRs OFFLINE
my.topic.one 0 565068 5 [5 1 9 2 3 8] [3 2 5] []
my.topic.one 1 565355 4 [5 2 7 3 4] [5 3 4 7] []
my.topic.one 2 564783 1 [5 1 9 3 8 4] [5 4 1] []
my.topic.one 3 566029 2 [4 7 2 1] [1 2 4] []
my.topic.one 4 565037 3 [5 1 2 3 4] [1 3 2] []
my.topic.one 5 564935 5 [5 6 2 3 4] [5 4 3] []
my.topic.one 6 564834 4 [5 1 6 7 4] [5 1 4] []
my.topic.one 7 565445 1 [1 2 3 8 4] [4 2 1] []
my.topic.one 8 564632 2 [1 6 9 2 7 3] [1 3 2] []
my.topic.one 9 564820 3 [5 1 9 2 3 8] [5 2 3] []
my.topic.one 10 564922 5 [5 1 2 7 4] [5 4 1 7] []
my.topic.one 11 566625 4 [1 9 2 3 8 4] [1 4 2] []
my.topic.one 12 566062 1 [1 6 2 3 4] [2 1 3] []
my.topic.one 13 564796 2 [5 1 2 3 4] [5 2 3] []
my.topic.one 14 564378 3 [5 6 2 3 4] [5 4 3] []
# > kafkactl get brokers
BROKER ID GRPs LDR.REPLICAS PEER.REPLICAS TOTAL.REPLICAS MIGRATING.REPLICAS OVERLOAD
broker01.example.io:9092 1 4 5820 11652 17472 0 55
broker02.example.io:9092 2 7 5815 11642 17457 16 78
broker03.example.io:9092 3 9 5830 11672 17502 0 78
broker04.example.io:9092 4 6 5434 10880 16314 13 81
broker05.example.io:9092 5 7 5793 11598 17391 11 77
broker06.example.io:9092 6 12 454 918 1372 62 0
broker07.example.io:9092 7 8 460 930 1390 68 1
broker08.example.io:9092 8 7 467 944 1411 72 1
broker09.example.io:9092 9 8 507 1024 1531 36 1
- Once complete, describe the topic for the results and new Partition Leaders. Note a Preferred Replica Election is needed:
# > kafkactl describe topic -x my.topic.one
TOPIC PART OFFSET LEADER REPLICAS ISRs OFFLINE
my.topic.one 0 565072 1 [1 8 9] [1 9 8] []
my.topic.one 1 565357 2 [2 7 5] [5 2 7] []
my.topic.one 2 564786 3 [3 8 9] [9 3 8] []
my.topic.one 3 566031 2 [4 7 2] [2 4 7] []
my.topic.one 4 565042 5 [5 4 1] [5 1 4] []
my.topic.one 5 564936 6 [6 3 2] [6 2 3] []
my.topic.one 6 564835 7 [7 6 1] [1 6 7] []
my.topic.one 7 565448 8 [8 3 4] [3 8 4] []
my.topic.one 8 564632 9 [9 6 7] [6 9 7] []
my.topic.one 9 564823 1 [1 9 8] [1 9 8] []
my.topic.one 10 564924 5 [2 5 7] [5 2 7] []
my.topic.one 11 566628 3 [3 9 8] [9 3 8] []
my.topic.one 12 566064 4 [4 6 3] [6 3 4] []
my.topic.one 13 564804 5 [5 4 1] [5 1 4] []
my.topic.one 14 564381 6 [6 5 2] [5 6 2] []
- Run a Preferred Replica Election and describe the topic once again for final results:
# > kafkactl admin set pre
Preferred Replica Election Successfully Sent.
TOPICS.NEED.PRE PARTITIONS.NEED.PRE
1 2
# > kafkactl describe topic my.topic.one
TOPIC PART OFFSET LEADER REPLICAS ISRs OFFLINE
my.topic.one 0 566201 1 [1 8 9] [1 9 8] []
my.topic.one 1 566497 2 [2 7 5] [5 2 7] []
my.topic.one 2 565955 3 [3 8 9] [9 3 8] []
my.topic.one 3 567207 4 [4 7 2] [2 4 7] []
my.topic.one 4 566177 5 [5 4 1] [5 1 4] []
my.topic.one 5 566053 6 [6 3 2] [6 2 3] []
my.topic.one 6 565987 7 [7 6 1] [1 6 7] []
my.topic.one 7 566636 8 [8 3 4] [3 8 4] []
my.topic.one 8 565797 9 [9 6 7] [6 9 7] []
my.topic.one 9 566000 1 [1 9 8] [1 9 8] []
my.topic.one 10 566041 2 [2 5 7] [5 2 7] []
my.topic.one 11 567756 3 [3 9 8] [9 3 8] []
my.topic.one 12 567261 4 [4 6 3] [6 3 4] []
my.topic.one 13 566031 5 [5 4 1] [5 1 4] []
my.topic.one 14 565536 6 [6 5 2] [5 6 2] []