Replacing Database Sequences at Scale: A Cached, Distributed ID Generation System
This article details Coupang's journey to replace traditional database sequences with a highly scalable, available, and low-latency distributed ID generation system. It highlights critical design decisions, such as prioritizing eventual consistency and local caching over strict global ordering and network calls, to support over 100 services and facilitate a seamless migration from relational databases to NoSQL.
Read original on InfoQ ArchitectureThe Challenge: Database Sequences in a Microservices World
When migrating from relational databases to NoSQL at scale, a common but often overlooked dependency is the database sequence. Many services rely on these for unique, monotonically increasing primary keys. The article discusses how Coupang faced this challenge with over 100 services and 10,000 distinct counters, where traditional solutions like UUIDs or Snowflake IDs didn't fit due to schema compatibility, ordering requirements, or operational complexity.
Key Design Principles for a Drop-in Replacement
- Minimize Coordination: Avoid distributed locks and consensus protocols to reduce latency and failure modes.
- Tolerate Gaps: Accept that unused sequences may be lost, simplifying the distributed coordination problem.
- Push Caching to the Edges: Implement aggressive caching both server-side and client-side to minimize network round-trips.
- Keep Architecture Legible: Design for operational clarity, allowing on-call engineers to easily understand and debug the system.
- Preserve Backward Compatibility: Ensure the new system is a drop-in replacement, requiring minimal code changes for existing services.
Revalidating Requirements: Global Ordering vs. Uniqueness
Validate Core Requirements
A crucial insight was realizing that most teams did not require *strict global ordering* or *gap-free sequences*, only uniqueness and local monotonicity. This fundamental shift allowed for a much simpler, highly performant distributed design, avoiding complex consensus protocols. Always question assumptions about requirements; what seems critical might be negotiable.