how to manage distributed task queues well?

I've been struggling with managing distributed task queues lately, especially when workloads jump around and I need to make sure tasks get done right. I've checked out things like Celery and Kafka, but they both seem to have their own downsides when it comes to how complicated they are, how well they handle failures, and how fast they can process stuff. What strategies or tools have you found work best for building task processing systems that can scale up and don't break easily? Any war stories from using these in real life?