Menu
Back to Discussions

Message queues for async job processing: is Kafka overkill?

Ivan Sato
Ivan Sato
·289 views
our cto is pushing to use kafka for pretty much everything, including what i feel are simple async job processing tasks like sending email notifications or generating daily reports. while kafka is great for high-throughput stream processing, it feels like overkill for these fire-and-forget types of jobs. i've used tools like bullmq, celery, or even just plain old sqs for these in previous roles, and they seem much simpler to operate and reason about. i get the desire to standardize on a single messaging backbone, but is the operational complexity and resource consumption of kafka justified for simple task queues? i'm trying to make a case for using the right tool for the job, rather than a one-size-fits-all approach. has anyone successfully argued against using kafka for simpler async tasks?
22 comments

Comments

Sign in to join the conversation.

Loading comments...