Async Job Queues: When Do They Become a Bottleneck?

So, I've been giving a lot of thought to how we use async job queues, you know, like Celery or Sidekiq, for background stuff. They're great for taking work off the main thread and making things feel faster. But, I'm starting to wonder if there's a point where the queue itself, or the workers handling it, can become a real problem. When have you guys seen this actually happen? And what did you do to figure out it was the queue, and then how did you fix it? If a job queue just can't handle the traffic, what else can we do?