you usually want to think about a distributed task queue when you've got a lot of stuff that needs to get done, but its
On Phan
·6564 views
hey all, ive been thinking about background jobs and async processing lately. some tasks take a while, like image processing or making reports, and its starting to slow down the main app threads. we use cron jobs and a basic in-memory queue now, but that doesnt feel like enough. when did you all decide it was time for a real distributed task queue, like Celery, Sidekiq, or Kafka Streams? what were the specific problems that made you switch?
23 comments