Perplexity's CobbleDB: Building a Custom Key-Value Store for Search Traffic
Perplexity replaced DynamoDB with CobbleDB, a custom key-value store built in Rust, to achieve lower latency and reduce costs for its search infrastructure. This article details the architectural decisions behind separating storage tiers, optimizing for read performance, and the trade-offs of building a bespoke database for specific operational needs.
Read original on The New StackPerplexity faced significant challenges with DynamoDB, primarily around read performance control and escalating costs for its high-volume search traffic. Each search request involves fetching 100-120 page keys in batches, with items averaging 50KB. DynamoDB's lack of control over replica behavior meant a slow replica could bottleneck an entire batch read, leading to high p99 latencies. Additionally, the cost model for large reads and writes generated by crawling, reprocessing, and live search became unsustainable as traffic and data corpus grew.
CobbleDB: A Custom Key-Value Solution
To address these issues, Perplexity developed CobbleDB, a 40,000-line Rust key-value store. It now handles a portion of their production search traffic, demonstrating significant improvements: median batch-read latency dropped from 31.4 ms to 5.6 ms, and p99 latency decreased from 123 ms to 24.2 ms. The company projects at least 20% cost savings compared to DynamoDB. This highlights a common dilemma in system design: leveraging managed services for speed versus building custom solutions for tailored performance and cost optimization.
Custom vs. Managed Database Trade-offs
Building a custom database, while offering granular control and potential cost savings, introduces significant operational overhead and long-term maintenance costs. These engineering efforts must be weighed against the benefits, especially for core infrastructure components.
Three-Tiered Storage Architecture
Perplexity redesigned its storage stack into three distinct tiers to optimize for durability, updates, and live serving:
- Pillar: Durable document state in YTsaurus on HDDs for versioned metadata, chunks, and embeddings.
- Lorry: Packages updates into partition-specific batches and moves them via S3 to CobbleDB.
- CobbleDB: Stores processed page data across three replicas per partition. It uses hashed URLs as keys, with RocksDB caching frequently accessed data in memory and the rest on local NVMe drives. Reads prioritize the same availability zone and can failover to other replicas to prevent batch holdups. Updates are applied independently, allowing replicas to catch up without blocking others.
This tiered approach separates concerns, allowing each layer to be optimized for its specific role – archival storage, data propagation, and low-latency serving.