Reliability, Availability & Fault Tolerance

Reliability vs Availability

These terms are often used interchangeably in casual conversation, but they measure distinct properties. Getting them right in an interview demonstrates precision.

The Nines of Availability

Availability is typically expressed as a percentage of uptime. The industry uses a shorthand of 'nines' to describe these percentages:

Each additional nine is approximately 10x harder and more expensive to achieve. Going from three nines to four nines is not an incremental improvement — it requires fundamental architectural changes: redundancy at every layer, automatic failover, and very disciplined operational practices.

Why Dependencies Reduce Availability

When a request must pass through multiple services in sequence (in series), overall availability is the product of each component's availability:

# System availability for components in series:
Overall = A1 × A2 × A3 × ...

# Example: 3 services each at 99.9%
Overall = 0.999 × 0.999 × 0.999 = 0.997 = 99.7%

# Microservices at scale: 10 services each at 99.9%
Overall = 0.999^10 = 0.990 = 99.0%  ← only two nines!

# For components in parallel (redundant):
Overall = 1 - (1-A)^N
# Two services each at 99%, running in parallel:
Overall = 1 - (0.01 × 0.01) = 1 - 0.0001 = 99.99%  ← four nines!

This math has profound implications. A microservices architecture with 50 services in a critical path, each at 99.9%, would have an overall availability of 95.1% — worse than two nines. This is why high-availability systems are designed with redundancy (parallel paths) and why minimizing synchronous dependencies is crucial.

Fault Tolerance Patterns

Fault tolerance is achieved through a combination of architectural patterns. Understanding these patterns and when to apply them is central to system design interviews.

Redundancy and Replication

The most fundamental fault tolerance technique: run multiple copies of a component so that if one fails, others take over. This can be active-passive (one leader handles traffic, replica takes over on failure) or active-active (multiple nodes handle traffic simultaneously, any can fail without impact).

Circuit Breaker

Named after the electrical safety device, a circuit breaker monitors calls to a dependency. If failures exceed a threshold, it 'opens' the circuit — subsequent calls fail immediately without attempting the downstream service. After a timeout, it enters a 'half-open' state and allows a probe request through. If that succeeds, the circuit 'closes' and normal operation resumes.

Netflix's Hystrix library (now largely superseded by Resilience4j) popularized circuit breakers in microservices. When Netflix's recommendation service has an outage, the circuit breaker ensures that streaming requests fail fast rather than waiting for a timeout — preserving threads and allowing the core playback experience to continue.

Bulkhead Isolation

Borrowed from ship design (watertight compartments that prevent a single hull breach from sinking the vessel), the bulkhead pattern partitions resources to isolate failures. Different services or tenants get separate thread pools, connection pools, or even separate infrastructure. A spike in one area cannot exhaust resources needed by another.

Timeouts and Retries

• Timeouts — Every network call must have a timeout. A slow dependency without a timeout will hold threads indefinitely, cascading into a full system failure.
• Retry with exponential backoff — On a transient failure, retry after a short delay. Double the delay on each subsequent retry (backoff). Add jitter (randomness) to prevent all retrying clients from hammering the service simultaneously (thundering herd).
• Idempotency — Retries are only safe if the operation is idempotent (repeated calls produce the same result). Payment systems must be especially careful: use idempotency keys to prevent duplicate charges.

Designing for Failure

The most resilient systems assume that every component will fail, and design for graceful degradation. Amazon's approach, formalized in their Chaos Engineering practices, involves deliberately injecting failures in production to verify that the system handles them correctly. Netflix's Chaos Monkey randomly terminates EC2 instances during business hours to ensure services never depend on a single node being alive.