When should you start thinking about using a distributed pub/sub system?

Lately, I've been building some real-time apps and it got me thinking about how to get services talking to each other. For smaller setups, just using HTTP or a simple message queue is usually okay. But when you have a lot more services and they all need to chat with each other, it really feels like you should be using a proper pub/sub system, you know, something like Kafka, Pulsar, or NATS. So, what makes you decide to switch to a distributed pub/sub setup? Is it how many things are listening, how many messages are flying around, or is it something else?