Load balancing with Kafka is a straightforward process and is handled by the Kafka producers by default.  While it isn’t traditional load balancing, it does spread out the message load between partitions while preserving message ordering.

Round-robin approach:  By default, producers choose the partition assignment for each incoming message within a topic, following a round-robin approach.

Parallelization:  Partitions allow users to parallelize topics.  This means that data for any topic can be divided over multiple machines, allowing multiple consumers to read topics in parallel.  The system of partitions is part of what makes Kafka ideal for high message throughput use cases.

Specifying exact partition:  If your use case would benefit from an approach other than the round-robin parsing of messages into different partitions, you’re in luck. With Kafka, users can specify the exact partition for a message. If such a specification is made, then the message will be guaranteed to go the specified partition.

Rebalancing:  If a consumer drops off or a new partition is added, then the consumer group rebalances the workload by dividing ownership of the partitions between the remaining consumers.  Rebalancing of one consumer group does not have an effect on other consumer groups.  

For more information on consumer groups, check out our post detailing how Kafka uses consumer groups for scaling event streaming.

And check out this article to learn more about Kafka partitions’ role in guaranteeing message order.