Event ordering in Kafka: partition-level ordering vs global ordering
Fatima Kumar
·13 views
Kafka's default behavior provides ordering guarantees at the partition level. If you partition your events by `userId`, all events for a specific user will be processed in order. This works great for many scenarios. However, we're encountering situations where we need global ordering for certain types of events. For example, an admin disabling an account might have an event that needs to be globally ordered relative to other system-wide changes, not just relative to that specific user's partition.
Achieving global ordering in Kafka is notoriously difficult, often requiring a single partition or some complex re-sequencing logic in consumers, which defeats the purpose of Kafka's distributed nature. We've explored re-partitioning strategies or using a dedicated 'global' topic for critical events, but each adds significant complexity and potential performance bottlenecks. How do you balance the need for partition-level ordering with occasional requirements for system-wide, globally ordered events without creating a single point of contention?
6 comments