Menu
Back to Discussions

Message queues for async job processing: is Kafka overkill?

Rina Kim
Rina Kim
·110 views
our cto is pushing to use kafka for literally everything, including sending emails and generating reports, citing standardization benefits. while i agree kafka is powerful for high-throughput data streams and event sourcing, for simple async job processing like sending a few thousand emails or generating daily pdf reports, it feels like overkill. solutions like bullmq, celery, or even just sqs for simple queueing seem more appropriate, easier to manage, and better suited for smaller, independent tasks that might not require the full durability or ordering guarantees of kafka. are we missing something by not standardizing everything on kafka, or is it a valid architectural choice to use different queueing systems for different job processing needs? i'm trying to find the right balance between standardization and using the right tool for the job.
6 comments

Comments

Sign in to join the conversation.

Loading comments...