Designing a Distributed Rate Limiter with Redis and Lua

The Challenge: Flawed In-Memory Rate Limiting

Initially, the rate limiter was implemented with per-instance in-memory counters. While simple for single-node applications, this approach quickly failed in a distributed production environment. Key issues included lack of global state, leading to inconsistent rate limits across different service instances, and race conditions where multiple threads could bypass the limit due to non-atomic check-then-increment operations. This highlighted the necessity for a centralized, globally consistent mechanism.

Leveraging Redis for Global Consistency and Atomicity

The core insight was to centralize the rate limiting logic in a system that guarantees atomicity and provides a single source of truth. Redis, with its single-threaded event loop, is an excellent candidate for this. By executing the increment, expiry-set, and comparison operations within a Lua script on Redis, the entire sequence becomes an atomic transaction. This prevents race conditions and ensures that all API gateway instances operate against the same, consistent view of the current rate limit for a given user or key.

-- rate_limiter.lua
-- KEYS[1] = redis key for this user (e.g., "rate:user:123")
-- ARGV[1] = limit (e.g., 100)
-- ARGV[2] = window size in seconds (e.g., 60)
local current = redis.call('GET', KEYS[1])
if current == false then
  -- first request in this window
  redis.call('SET', KEYS[1], 1)
  redis.call('EXPIRE', KEYS[1], ARGV[2])
  return 1 -- allowed
end
if tonumber(current) >= tonumber(ARGV[1]) then
  return 0 -- deny
end
local newval = redis.call('INCR', KEYS[1])
return newval -- allowed (new count)

Trade-offs of the Redis-based Approach

• Latency vs. Consistency: Introducing a Redis round-trip for every request adds a small amount of network latency. However, this is a necessary trade-off for achieving global consistency and preventing unfair blocking or excessive traffic.
• Complexity: Using Lua scripts might seem like an added layer of complexity, but it's a one-time development cost that significantly improves correctness and reliability.
• Throughput: Redis is highly performant (hundreds of thousands of operations/sec), sufficient for most API rate limiting needs. For extreme cases requiring millions of operations, further scaling like sharding Redis or specialized rate-limiting services might be considered.
• Scalability: The system scales horizontally by adding more API gateway instances; Redis acts as the centralized, consistent counter.