Applying Redis and Kafka in Production: Common Interview Questions & System Design Patterns

The article uses a PSI (POS + inventory microsystem), comprising Spring Boot and Go services, as a practical example to demonstrate robust uses of Redis for caching and Kafka for asynchronous events. The author emphasizes a 'boring architecture' approach, prioritizing reliability and timely delivery over novel complexities.

Redis's Single-Threaded Nature and Performance

Redis's speed, despite being single-threaded, is attributed to fundamental optimizations that eliminate common bottlenecks in multi-threaded environments. The single-threaded model inherently avoids:

• Lock contention on data structures, simplifying concurrent access.
• Context switches between threads, reducing latency spikes.
• Cache line bouncing across CPU cores, improving cache coherency.

Mitigating Common Cache Problems with Redis

The article details practical solutions for three common caching issues:

1. Cache Penetration: When requests for non-existent data repeatedly hit the backend database. Solved by caching 'negative' results (e.g., a `NULL_TOKEN`) with a short TTL.
2. Cache Breakdown (Dogpile Effect): When a hot key expires, and many requests simultaneously try to re-fetch the data, overwhelming the database. Addressed using a distributed lock (e.g., Redis `SET NX EX`) to ensure only one request repopulates the cache.
3. Cache Avalanche: When many keys expire simultaneously, causing a mass expiry and a sudden load on the database. Mitigated by adding random jitter to TTL values (`baseTtl + jitter`), spreading out expiry times.

// 1. Penetration 
 Sku sku = redis.get("sku:" + code);
 if (sku == null) {
  sku = db.query(code);
  if (sku == null) {
   redis.setex("sku:" + code, 300, "NULL_TOKEN"); // 5-min null cache
   return null;
  }
  redis.setex("sku:" + code, 3600, JsonUtil.toJson(sku));
 }

// 2. Breakdown 
 Sku cached = redis.get("sku:" + code);
 if (cached != null) return cached;
 String lockKey = "lock:sku:" + code;
 if (redis.set(lockKey, "1", "NX", "EX", 5) != null) { // got the lock
  try {
   Sku fresh = db.query(code);
   redis.setex("sku:" + code, 3600, JsonUtil.toJson(fresh));
   return fresh;
  } finally {
   redis.del(lockKey);
  }
 }
 Thread.sleep(50); // wait + retry

// 3. Avalanche 
 long baseTtl = 3600;
 long jitter = ThreadLocalRandom.current().nextLong(0, 600);
 redis.setex("sku:" + code, baseTtl + jitter, JsonUtil.toJson(sku));

When Not to Use Redis

While Redis is powerful, it's not a universal solution. The article highlights scenarios where local caching combined with a publish-subscribe invalidation mechanism might be more appropriate. For example, storing a permission tree that changes infrequently and requires local access across many nodes can benefit from a local cache (like Caffeine) with Redis Pub/Sub used only for invalidation, reducing network overhead for reads.