What does “Rebalancing” mean in Apache Kafka context?

When a new consumer joins a consumer group the set of consumers attempt to “rebalance” the load to assign partitions to each consumer. If the set of consumers changes while this assignment is taking place the rebalance will fail and retry. This setting controls the maximum number of attempts before giving up.

the command for this is: rebalance.max.retries and is set to 4 by default.

also, it might be happening if the following is true:

ZooKeeper session timeout. If the consumer fails to send a heartbeat to ZooKeeper for this period of time it is considered dead and a rebalance will occur.

Hope this helps!

Leave a Comment