So, when does RabbitMQ actually slow things down in busy systems?

RabbitMQ gets a lot of love for being reliable and good at routing in microservices, and yeah, it deserves it. It really does make sure messages get where they need to go, even with tricky routing. But I'm wondering, where does it start to hit a wall? Like, with how many messages or how much complexity does RabbitMQ actually slow things down? Have you found specific setups or ways of using it that let you push it further? Or are there times you just know it's better to switch to something like Kafka because of how much scale you need? I'm not just looking for "it scales" answers, more like real-world numbers and stories from the trenches.