When does caching actually become a bottleneck?

Hey everyone, I've been mulling over caching strategies recently, and when the caching layer itself starts causing more headaches than it helps. Caching is obviously great for speed, but I've run into situations where the logic for invalidating the cache gets incredibly messy, or the cache becomes a bottleneck when traffic spikes. Are there usual ways or particular numbers you watch to figure out if it's time to rethink your caching setup, or maybe even scale it back? I'd love to hear your experiences and what worked best for you.