Meta Engineering·September 3, 2026

ZGateway: A Proxy Layer for Meta's ZippyDB

This article details ZGateway, a proxy layer introduced in front of Meta's ZippyDB key-value store to manage a vast and diverse client population. It highlights how ZGateway centralizes critical functionalities like connection management, request batching, admission control, and traffic routing, which were previously fragmented across millions of client binaries. The implementation addresses scalability and reliability challenges arising from a direct-access model, transforming an unmanageable many-to-many connection mesh into a controlled, efficient two-hop architecture.

Distributed Systems Performance & Scaling Cloud & Infrastructure

Read original on Meta Engineering

Meta's ZGateway serves as a crucial intermediary layer for ZippyDB, a globally distributed key-value store handling billions of operations per second. The introduction of ZGateway was driven by the challenges of managing a rapidly growing and diverse client fleet, where direct client-to-database connections led to significant inefficiencies and reliability concerns. Proxies are a common pattern in distributed systems, especially when dealing with a large, diverse client population communicating with a shared backend. They offer a centralized point for managing traffic and implementing shared services.

Why a Proxy was Essential for ZippyDB

Initially, ZippyDB clients directly connected to every database host they needed, creating a dense many-to-many mesh of TLS connections. This direct-access model exhibited several problems at Meta's scale:

Resource Waste: Each connection consumed memory, CPU, and file descriptors on both client and server, often while idle.
Fragility: Connection storms, often triggered by client restarts or deployments, could lead to database host crashes due to resource exhaustion (e.g., file descriptor limits, OOMs).
Operational Complexity: Fixing client-side issues was challenging due to the large, diverse, and independently evolving client fleet. Implementing features like connection pooling or retries consistently across millions of client binaries was impractical.

The ZGateway Solution: Bounding Complexity

ZGateway decouples clients from ZippyDB servers, transforming the unbounded many-to-many mesh into two bounded hops: clients connect to ZGateway, and ZGateway connects to ZippyDB servers. This architecture significantly reduces per-host connection counts and isolates the database fleet from client-side issues. The proxy layer allows for centralized control over key aspects of traffic management, security, and performance. Importantly, ZGateway is stateless, runs as regional tiers, and uses Meta's thick C++ client internally, essentially acting as a managed service running a ZippyDB client.

Key Capabilities Enabled by ZGateway

Connection Management: ZGateway consolidates connections, reducing the total persistent connections and eliminating client-induced connection storms. It takes ownership of connection pooling and failover logic.
Request Batching and Coalescing: By observing traffic from many clients, ZGateway can batch multiple requests for the same destination (shard) into a single backend RPC. It also coalesces requests for the same key, fetching the data once and fanning out the result to all waiting clients. This amortizes overhead, reduces QPS to the backend, and protects against hot key stampedes.
Admission Control and Tenant Isolation: ZGateway implements Discriminant Load Shedding (DLS) to isolate misbehaving tenants. If a tenant floods the system, only its requests are shed, ensuring other tenants remain unaffected. This is crucial for a shared service serving hundreds of use cases.
Traffic Routing and Safe Migration: ZGateway provides configurable routing logic, enabling incremental, reversible, and scoped migrations of client traffic. This allows for controlled rollouts with instant rollback capabilities.
Security and Observability: Centralizing traffic allows for unified TLS termination, authorization, validation, per-use-case metrics, traces, and quota usage recording.

ℹ️

System Design Takeaway

Introducing an intelligent proxy layer is a powerful pattern for managing large-scale distributed systems, especially when dealing with diverse, uncontrolled client populations. It centralizes shared concerns, improves reliability, enhances observability, and allows for controlled evolution of both client and backend systems without tight coupling. The trade-off is an additional hop and tier to operate, which is justified when the benefits of control and efficiency are significant.

proxykey-value storeconnection managementbatchingcoalescingadmission controlload balancingreliability

Comments

Loading comments...

Architecture Design

Design this yourself

Design a globally distributed key-value store that supports billions of operations per second with a large and diverse client base. Include a detailed architecture for a proxy layer that sits in front of the key-value store, addressing connection management, request batching and coalescing, admission control for multi-tenancy, traffic routing, and cross-region resilience. Discuss the trade-offs of introducing such a proxy.

Practice Interview

Focus: distributed proxy for a key-value store with connection management, request batching, coalescing, and admission control

Other design angles

· Design only the proxy layer for an existing key-value store, focusing on its internal architecture, connection pooling, and request processing flow.· Design a rate limiter and admission control system for a shared backend service that handles traffic from hundreds of internal teams, ensuring fair usage and preventing noisy neighbors.· Evaluate the architectural decision of direct client-to-database access versus an intermediary proxy layer for a high-traffic microservice, considering factors like operational overhead, development complexity, and reliability at scale.