when do message queues start to hurt more than help

I've been using message queues for a bit now, you know, to split up services and deal with tasks that don't need to happen right away. They've worked really well. But I'm starting to wonder, is there a point where it just gets too complicated? Like, managing the queue itself, the risk of losing messages if you mess up the setup, and the extra delay all that adds. Does that stuff start to outweigh the good parts? Has anyone else run into this issue, and what did you do about it?