Menu
Back to Discussions

Multi-level caching: L1 (in-process) + L2 (Redis) + L3 (CDN) — is it worth it?

Pierre Watanabe
Pierre Watanabe
·319 views
we're exploring a multi-level caching strategy for some of our ml inference results: an l1 cache (in-process memory) for the hottest keys, backed by an l2 cache (redis cluster), and then potentially an l3 (cdn) for static assets. the motivation is that even redis hits still incur 1-2ms of network latency, and for our highest-volume endpoints, that adds up. adding an in-process l1 cache could bring response times down to microseconds. the main challenge is cache coherence across 20+ instances of our service. we'd likely use redis pub/sub for l1 invalidation events. is this kind of multi-level caching worth the complexity? what pitfalls have you encountered trying to keep in-process caches consistent across many service instances, especially concerning invalidation latency and avoiding false positives?
0 comments

Comments

Sign in to join the conversation.

Loading comments...