when to cache versus when to recompute

We all know caching is great for speeding things up and reducing server load. The common advice is to cache API responses. But I'm curious about the tipping point. When does the complexity and potential staleness of a cache outweigh the benefits of simply recomputing the data when needed? Are there specific scenarios where avoiding caching altogether is a better design choice, even if it means slightly slower response times initially?