Implementing Distributed Rate Limiting with Redis for Microservices
This article explores the critical architectural decision of state management when migrating from a monolithic application to microservices, specifically in the context of implementing a rate limiter. It highlights why in-memory rate limiters fail in distributed environments and advocates for centralizing shared state in a fast, external data store like Redis to ensure accurate and scalable rate limiting.
Read original on Dev.to #systemdesignThe Challenge of Rate Limiting in Microservices
Rate limiting is crucial for protecting systems from traffic spikes and abuse. While implementing a rate limiter in a monolithic application is straightforward (using in-memory data structures like token buckets or sliding window counters), distributing this functionality across multiple microservice instances introduces significant complexity. The core problem lies in state sharing.
Why In-Memory Limiters Fail in Distributed Systems
In a monolithic architecture, all requests for a given key (e.g., user ID, IP address) typically hit the same process or can access a shared in-memory state. However, when a service scales out to multiple instances, each instance maintains its own isolated in-memory state. This leads to inaccurate rate limiting, as each instance independently tracks usage, effectively multiplying the allowed request rate by the number of instances.
The Core Insight: Centralize Shared State
To correctly implement rate limiting in a microservices environment, any state that needs to be consistent across service instances must be moved out of individual service processes and into a centralized, highly available, and fast data store. Redis is a common and effective choice for this purpose due to its in-memory nature and atomic operations.
Implementing a Redis-Backed Token Bucket Rate Limiter
The article demonstrates using Redis with Lua scripting to implement an atomic token bucket algorithm. Lua scripts executed via Redis' `EVAL` command run as a single atomic operation, preventing race conditions that could occur if read and write operations were performed separately. This ensures the rate limit is enforced accurately across all service instances.
import redis
import time
r = redis.Redis(host='redis-cache', port=6379, db=0)
LUA_SCRIPT = """
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local bucket = redis.call('HMGET', key, 'tokens', 'last')
local tokens = tonumber(bucket[1]) or capacity
local last = tonumber(bucket[2]) or now
-- refill
local delta = math.max(0, now - last)
tokens = math.min(capacity, tokens + delta * refill_rate)
local allowed = 0
if tokens >= 1 then
tokens = tokens - 1
allowed = 1
end
redis.call('HMSET', key, 'tokens', tokens, 'last', now)
redis.call('EXPIRE', key, 3600) -- auto-clean after 1h of inactivity
return allowed
"""
def allow_request(key: str, capacity: int = 10, refill_rate: float = 5.0) -> bool:
now = time.time()
allowed = r.eval(LUA_SCRIPT, 1, key, capacity, refill_rate, now)
return bool(allowed)
- Atomic Operations: The Lua script guarantees that reading and updating the token bucket state is atomic within Redis, preventing race conditions.
- Single Source of Truth: All service instances consult the same Redis cluster, ensuring consistent rate limit enforcement.
- Low Latency: Redis' in-memory nature provides sub-millisecond response times for rate limit checks.
- Operational Simplicity: Centralized state simplifies monitoring, tuning, and debugging of the rate limiting mechanism.
- Automatic Cleanup: The `EXPIRE` command ensures that inactive keys are automatically removed, managing memory usage.