When does in memory cache become a performance bottleneck

hey all, i've been thinking about the sweet spot for in-memory caching. we all know it's fast, but when does it start causing more problems than it solves? are there specific patterns or scales where holding everything in memory becomes a major bottleneck, perhaps due to size, contention, or even just the overhead of managing it? curious to hear your experiences and any specific scenarios where you've had to move beyond simple in-memory solutions. thanks