when does redis become a bottleneck in caching

We've been scaling our read-heavy app, and Redis has been our go-to for caching. Lately though, we're hitting some latency spikes. It looks like it's happening when Redis gets close to its memory or connection limits. I'm wondering what other teams have run into. When did Redis start becoming a problem for you, and what did you do about it besides just making the server bigger? I'm thinking about things like sharding Redis itself, or maybe adding more caching layers in front of it.