Optimizing Tiered Caching for Public Cloud Anycast Origins
Cloudflare's Smart Tiered Cache optimizes cache efficiency by routing traffic through a single optimal upper-tier data center to an origin. This article discusses how Cloudflare enhanced Smart Tiered Cache to address challenges posed by public cloud origins behind anycast or regional unicast networks, which previously hindered efficient single-tier routing due to ambiguous latency probes.
Read original on Cloudflare BlogCloudflare's Smart Tiered Cache aims to improve cache hit ratios and reduce origin load by funneling all cache misses through a single, latency-optimal upper-tier data center. This works by continuously probing origin IPs to identify the data center with the lowest latency. The core idea is to concentrate traffic for a given origin at one Cloudflare PoP, thereby maximizing the chances of serving content from cache rather than forwarding requests to the origin.
The Challenge of Anycast Origins in Public Clouds
Public cloud providers frequently use anycast or regional unicast networking for their front-end services and load balancers. When an origin is behind such an IP, it appears 'close' to many Cloudflare data centers simultaneously because the IP represents a cloud provider's edge network, not a single physical origin location. This ambiguity makes it difficult for Smart Tiered Cache to confidently select a single best upper-tier, leading to potential hairpinning where traffic traverses continents unnecessarily, significantly increasing latency.
Hairpinning Example
If an origin in Singapore is behind an anycast IP, Cloudflare's Chicago data center might register the lowest probe latency. Requests from Asia would then route to Chicago, which then fetches data from Singapore, resulting in two cross-ocean trips instead of one direct path.
Smart Tiered Cache for Public Cloud Regions Solution
To solve this, Cloudflare implemented several improvements: 1. Anycast Detection: They use a physics-based approach, comparing probe latencies from multiple global checkpoints. If combined latencies between two points are faster than light can physically travel, it indicates the origin is answering from multiple locations, confirming it's an anycast setup. 2. Region Hinting: Customers can now provide a cloud region hint (e.g., `aws:us-east-1`) via the Cloudflare dashboard or API. This explicit hint allows Cloudflare to map the origin to its true cloud region. 3. Dynamic Upper-Tier Selection: Every few hours, Cloudflare fetches IP range files from major cloud providers. These subnets are matched against their upper-tier database, which is built from continuous latency probing. For each cloud region, subnets contribute weighted votes to determine the primary and fallback upper-tiers, ensuring they come from different PoPs for resilience. If probe data is insufficient, it falls back to a geographic guess, refining as real data accrues.
- Improved Cache Efficiency: By correctly identifying the origin's cloud region, Smart Tiered Cache can select a truly optimal upper tier, boosting cache hit ratios and reducing origin load.
- Reduced Latency: Eliminates hairpinning by ensuring traffic is routed to the closest Cloudflare PoP to the actual cloud region.
- Automatic Adaptability: The system automatically learns and adapts to changes in cloud provider IP ranges and origin setups, minimizing manual configuration.
- Resilience: Primary and fallback upper tiers are always in different PoPs, providing failover capabilities.
This evolution of Smart Tiered Cache demonstrates a sophisticated approach to optimizing content delivery networks in a dynamic public cloud environment, balancing automated intelligence with user-provided hints to achieve optimal performance and efficiency.