Consumer group partition rebalancing
Your task
Design the partition-assignment + rebalancing coordination for a consumer-group system processing a 2,000-partition event stream across an autoscaling pool of consumers (5–100 nodes). Each partition must be owned by exactly one consumer at a time, ownership must rebalance when nodes join/leave, and rebalancing must not cause a long global stop-the-world pause where no partitions are being processed. How do you assign partitions and minimize disruption during rebalances?
It covers
- Partition assignment
- Rebalancing
- Consumer groups