Beyond Basics: Deep Dive into System Design Interview Concepts

Many candidates can define system design concepts like caching or sharding, but often struggle with follow-up questions that probe deeper into their practical implications, failure modes, and trade-offs. This article focuses on addressing these deeper questions, which are crucial for designing robust and scalable systems.

Caching: The Invalidation Problem

Caches significantly improve read performance by storing hot data closer to the user (e.g., in-memory stores like Redis). The cache-aside pattern is common, where the application first checks the cache, then the database on a miss, and updates the cache. However, the critical and often overlooked aspect is cache invalidation.

def get_user(user_id: str):
  cached = redis.get(f"user:{user_id}")
  if cached: return json.loads(cached) # Cache hit ✓

  user = db.find_one({"_id": user_id})
  redis.setex(f"user:{user_id}", ttl=3600, value=json.dumps(user))
  return user
  # ⚠️ The hidden danger: what if the user updates their profile?
  # If you forget: redis.delete(f"user:{user_id}")
  # ...they'll see stale data for up to an hour.

Sharding: When and Why, Not Just How

Sharding distributes a database across multiple servers, improving scalability and throughput. However, it introduces significant complexity (cross-shard joins, distributed transactions, debugging). A common mistake is to prematurely introduce sharding without exhausting simpler scaling options first. The article emphasizes a progression of scaling techniques:

1. Vertical Scaling: Upgrading a single server's resources.
2. Read Replicas: Distributing read load across multiple database copies.
3. Caching: Reducing database load for frequently accessed data.
4. Sharding: Only when simpler methods genuinely cannot keep up.

# ❌ Bad shard key — timestamp creates a "hot shard"
def get_shard(created_at: datetime) -> int:
  return hash(created_at) % NUM_SHARDS
# All new writes hit the same shard. Others sit idle.

# ✅ Good shard key — user_id distributes evenly
def get_shard(user_id: str) -> int:
  return hash(user_id) % NUM_SHARDS
# Load spreads evenly. No single shard gets hammered.

Consistent Hashing: Resilient Distributed Caching

Naive modulo hashing (e.g., `key % N_SERVERS`) for distributing cache keys across servers leads to a catastrophic cache cold-start when servers are added or removed. Most keys remap, causing a database stampede.

Consistent hashing solves this by mapping both keys and servers onto a circular ring. When a server is added or removed, only a small fraction of keys (approximately `1/N`) are remapped, preserving most of the cache. This principle is fundamental to distributed systems like Redis Cluster, Cassandra, and DynamoDB for maintaining availability and performance during scaling events.