Leveraging Napkin Math for Robust System Design and Infrastructure at Scale
This article highlights the critical role of "napkin math" in identifying performance bottlenecks and optimizing system costs, drawing insights from Simon Eskildsen's experience at Shopify and turbopuffer. It emphasizes the importance of understanding theoretical limits of compute operations and applying quick calculations to challenge existing architectural decisions and drive significant improvements in large-scale infrastructure.
Read original on The Pragmatic EngineerThe Power of Napkin Math in System Design
Simon Eskildsen's "napkin math" approach is a cornerstone for robust system design. It involves performing quick, back-of-the-envelope calculations to estimate performance, identify bottlenecks, and evaluate the cost-effectiveness of architectural choices. This method helps engineers understand the theoretical limits of hardware and software operations, enabling them to challenge existing systems and propose more efficient solutions. For instance, at turbopuffer, Simon used napkin math to uncover that existing search solutions were significantly more expensive than necessary, leading to the development of a cost-optimized alternative.
Applying Napkin Math
When designing or troubleshooting systems, always start with simple calculations. What are the expected QPS? What's the latency budget? How much data needs to be stored and transferred? Comparing these estimates against actual performance or cost can quickly highlight areas for optimization or reveal fundamental design flaws.
Key Infrastructure Learnings from Shopify
Simon's almost decade-long tenure at Shopify provided extensive experience in scaling infrastructure for a massive Ruby on Rails monolith. Key challenges and architectural decisions included:
- Database Sharding and Cut-overs: Moving from a single database to a sharded architecture to handle immense write loads, even performing high-stakes cut-overs just before peak traffic events like Black Friday.
- Multi-Data Center Expansion: Extending database footprints across multiple data centers to enhance resilience and availability.
- Splitting Monolithic Services: Decomposing critical, oversized components (like a 128GB Redis instance) into smaller, manageable services to improve fault isolation and maintainability.
Toxiproxy: A Tool for Chaos Engineering
To ensure system resilience during partial outages, Simon developed toxiproxy. This tool acts as a TCP proxy that allows injecting various network failures (latency, downtime, packet loss) between application components and databases. It enables engineers to write comprehensive, systems-level integration tests to validate how an application behaves under adverse network conditions, rather than relying solely on mocking low-level drivers. This is a practical application of chaos engineering principles to build more robust distributed systems.