Kafka vs RabbitMQ vs SQS: how to choose for an event-driven system
Omar Reyes
·577 views
We're building out a new event-driven system, and the choice of message queue is a big debate right now: Kafka, RabbitMQ, or SQS. We expect around 10,000 messages per second, need at-least-once delivery, the ability to replay messages for debugging or new consumers, and about 5 different consumers potentially reading from the same topic.
Kafka is the obvious choice for high-throughput, ordered events and replayability, but it comes with a significant operational overhead for self-hosting. RabbitMQ is simpler to operate and great for general-purpose messaging, but its replayability features aren't as strong as Kafka's. SQS is fully managed, which is attractive, but has limitations on message ordering and replay depending on the queue type, and generally isn't designed for pub-sub fan-out to many consumers with persistent logs in the same way. What are the key decision points you use when choosing between these for a critical event-driven system, especially when throughput and replay are important?
18 comments