Optimizing CDN Cache Storage with Zstandard Compression in Pingora
Cloudflare explored using Zstandard compression within their Pingora-based CDN cache to significantly reduce storage costs and inter-data center bandwidth. This "Cache Transcoding" approach encodes eligible assets upon entry into the cache and decodes them only when served to clients, trading a minor CPU increase for substantial space and network efficiency. The architectural decision focuses on optimizing resource usage in a massively distributed storage product.
Read original on Cloudflare BlogThe Challenge of Cache Efficiency in Distributed Systems
Massively distributed storage products, like Cloudflare's CDN, face increasing memory costs (RAM and disk). Efficient utilization of deployed memory is crucial to maintain service levels while managing expenses. This article details Cloudflare's prototype, "Cache Transcoding," to expand effective cache capacity by leveraging Zstandard compression.
Cache Transcoding Architecture
Cache Transcoding integrates Zstandard (zstd) compression directly into the caching layer. When an eligible HTTP response enters the cache via the Pingora proxy, it's encoded with zstd before being written to disk. The compressed form is maintained within the cache and across data centers (via Tiered Cache) until it's decoded just before being served to the client. This "compress once, decode many" strategy amortizes the compression cost.
- Encoding: Occurs on a cache miss when fetching from origin. The body is zstd encoded before disk write.
- Decoding: Occurs on a cache hit or after transfer from an upper tier, just before serving to the client.
- Tiered Cache Interaction: Compressed assets are transferred between cache tiers in their compressed state, reducing inter-data center bandwidth.
Trade-offs: CPU vs. Storage/Bandwidth
The core trade-off is a minor increase in CPU utilization for significant storage and cross-data center bandwidth savings. Initial tests showed eligible assets shrinking to 1/3 of their original size on average. The encoding cost is paid once per asset, while storage and bandwidth savings are realized on every subsequent reuse. Zstd level 3 was chosen for its balance of compression ratio and speed, minimizing CPU bottlenecking during cache fills.
Eligibility Criteria for Compression
Not all content is suitable for re-compression. Images, videos, and fonts are typically already compressed and re-compressing them would waste CPU. Cache Transcoding targets uncompressed, compressible text content (HTML, JSON, CSS, JavaScript). Specific eligibility checks include: 200 OK response, unset `Content-Encoding`, compressible `Content-Type`, and a `Content-Length` of at least 4 KiB. This threshold avoids overhead for tiny objects while capturing most benefits.