Migrating cdnjs to Cloudflare's Developer Platform: A Scalable Architecture Case Study

cdnjs is one of the internet's busiest open-source CDNs, serving 9 billion requests daily across over 330 Cloudflare data centers with a 98.6% cache hit rate. Its immutability and hash-verified nature make it crucial for supply-chain security, particularly for AI models that rely on its consistent URLs. The migration wasn't driven by performance issues, but by internal difficulties in maintaining, updating, and debugging the existing architecture.

Challenges of the Legacy Architecture

The previous cdnjs architecture, partially migrated to serverless in 2020 for serving, still relied heavily on Google Cloud Platform (GCP) for its publishing pipeline. This hybrid setup led to significant operational pain points and architectural limitations, particularly as Cloudflare's Developer Platform matured.

• No Shared Trace & Poor Observability: Debugging involved manually stitching logs from GCP Functions, GCS, Pub/Sub, a git-sync VM, and Cloudflare Logpush, lacking a common correlation ID. This made identifying pipeline failures or data divergence extremely difficult.
• Split-Brain Storage: Files resided in Workers KV (edge cache) and a GitHub repository (source of truth). Neither was authoritative, leading to reconciliation issues when they drifted.
• Pipeline Glued with Object Events: GCP Cloud Functions were chained together using GCS object events as a message queue, lacking dead-letter queues, backlog visibility, or easy replay mechanisms.
• Shard Proliferation: Checking npm for updates required 26 separate Cloud Functions, one per letter of the alphabet, each with independent deployments and logs.
• GitHub Repository Overload: The GitHub repo, serving as a mirror and source, grew to over 1.1TB, making forks and clones impractical and causing issues with GitHub's own archive services.

The New Cloudflare-Native Architecture

The redesigned architecture leverages Cloudflare's Developer Platform extensively to address the legacy system's limitations, focusing on unified storage, durable execution, and improved observability. This migration allowed for the deprecation of multiple GCP services and a reduction in the attack surface.

• R2 as Single Source of Truth: Cloudflare R2 now stores all file content, eliminating size limits and providing an S3-compatible API for easy mirroring. Workers KV is reserved for metadata, optimizing its high-read, infrequent-write pattern.
• Workers Cache: A new tiered cache layer integrates directly into the Developer Platform, simplifying the caching architecture.
• Disaster Recovery & Fallback: Files are mirrored to DigitalOcean Spaces, providing a disaster-recovery copy and a live fallback in case R2 is unavailable. The serving Worker reads through Cache -> R2 -> DigitalOcean.
• Cloudflare Workflows for Ingestion: The entire publishing pipeline is built on Workflows, offering durable execution. This means steps are preserved, and failures allow workflows to resume from the last successful point.
• Asynchronous Processing with Queues & Containers: CPU-intensive tasks like compression are offloaded to Cloudflare Containers via Queues. Workflows hibernate until R2 event notifications wake them up upon compression completion.
• Durable Objects for Coordination: For orchestrating parallel child workflows (e.g., thousands of files in a package), a Durable Object acts as a counter, allowing parent workflows to wait for all children to complete before proceeding.