Fundamentals of High-Scale System Design

Core System Design Concepts

Understanding fundamental distinctions is paramount in system design. The article highlights several critical pairs:

• Performance vs. Scalability: Performance measures how fast a system is at a given load, while scalability assesses its ability to handle increased load without significant performance degradation. A system that scales well maintains performance as user traffic grows.
• Latency vs. Throughput: Latency is the time taken for a single request to complete (the 'waiting' time), whereas throughput is the number of requests processed per unit of time. High latency does not always mean low throughput, especially in parallel processing systems.
• Availability vs. Consistency (CAP Theorem): In distributed systems, a critical trade-off exists between availability (system remains operational and responds to all requests), consistency (all nodes see the same data at the same time), and partition tolerance (system continues to operate despite network failures). Most distributed systems choose between AP (Available, Partition tolerant) or CP (Consistent, Partition tolerant), with eventual consistency often adopted for AP systems.

Practical Scaling Scenarios and Solutions

The article presents several common system design challenges and their architectural solutions:

1. Handling Traffic Spikes: To scale purely with infrastructure, options include vertical scaling (adding resources to an existing server), horizontal scaling (adding more servers behind a load balancer), and partitioning (sharding or regional routing to distribute workload).
2. Database Bottlenecks: For high read-intensive workloads like URL shorteners (Bitly example), the database quickly becomes a bottleneck due to lookup intensity. Implementing Redis caching for frequently accessed data is a common solution to offload the database.
3. Feed Generation (Push vs. Pull): For social media feeds, a hybrid approach combining push (fan-out-on-write) for regular users and pull (fan-out-on-read) for high-follower accounts (celebrities) optimizes both read and write performance. Pure push can overwhelm during celebrity posts, while pure pull can overload databases during user login for many follows.
4. Distributed Counters: For rapidly incrementing counters (e.g., likes on a viral tweet), directly updating a database row (`UPDATE likes = likes + 1`) leads to row locking and write contention. A more scalable approach involves storing each like as a new row (append-only) and using a distributed counter in a key-value store like Redis. For extreme scale, sharded counters distribute increments across multiple keys to prevent hotkey issues.