Menu
Back to Discussions

How do you handle lots of background jobs when things get busy?

Qhama Mthethwa
Qhama Mthethwa
·4871 views
Hey everyone, I'm trying to figure out how to scale up our background job processing. Right now, we're just using a basic queue system, and it's getting a bit overwhelmed when things get busy. I'm wondering how other people manage this in their production setups. Are you guys using specific job queues like Celery or Sidekiq, or did you build your own? What are the biggest problems you've hit, particularly with retries, dead-letter queues, and keeping an eye on things?
31 comments

Comments

Loading comments...