Uber's GitFarm: Git Operations as a Service for Large-Scale Monorepos
Uber developed GitFarm, a "Git as a Service" platform, to manage Git operations across its massive monorepos. This centralized service significantly reduces client-side resource utilization and cold-start times by eliminating the need for local repository clones and providing pre-warmed checkouts. It leverages gRPC for high-performance API access and sandboxed command execution, demonstrating an effective approach to scaling developer infrastructure for large organizations.
Read original on InfoQ ArchitectureThe Challenge of Monorepos at Scale
Uber's diverse engineering organization, spanning Go, Java, Python, Web, Android, and iOS, operates within large-scale monorepos. Traditionally, individual services required full repository checkouts for various Git operations, leading to substantial resource consumption and slow cold-start times. For instance, cloning Uber's Go monorepo could take 15 minutes, consume 6 CPU cores, 32 GB of memory, and over 40 GB of disk space. This overhead became a significant bottleneck for developer workflows and automation systems, which collectively invoke Git millions of times daily.
GitFarm: Git Operations as a Service
GitFarm addresses these challenges by centralizing Git operations. It is not a source control management system but acts as a high-performance, shared Git client. Client systems no longer maintain local clones; instead, they interact with GitFarm via a gRPC API to execute standard Git commands. This approach has drastically reduced client-side resource utilization and eliminated lengthy cold-start delays, providing full Git checkouts in under 500 milliseconds.
Key System Design Principle: Centralization for Efficiency
GitFarm exemplifies the principle of centralizing a frequently performed, resource-intensive operation (Git clone and sync) into a dedicated service. This pattern is effective when the overhead of local execution becomes prohibitive across a distributed fleet of clients. By abstracting away the complexity and resource demands, the service optimizes performance and consistency for consumers.
Architectural Overview
- gRPC API Gateway: Authenticates and authorizes requests before routing them to backend clusters.
- Backend Clusters: Maintain bare repository clones, synchronized with upstream repositories via push-based updates and periodic fetches.
- Isolated, Ephemeral Sandboxes: Git commands execute within these sandboxes for security and resource isolation.
- Repository Checkout Pools: To achieve sub-second checkout times, GitFarm maintains pools of pre-warmed repository checkouts and sandbox containers. When a request arrives, a pre-warmed checkout is mounted into an available sandbox, bypassing the need for a fresh clone.
- Bidirectional gRPC Streaming: Supports multi-command workflows, allowing sequential execution against the same checkout (e.g., fetch, calculate merge base, push) without re-initializing repository state.
This architecture not only improved individual client performance but also offered flexibility. Workloads requiring the absolute latest state can explicitly trigger `git fetch`, while those tolerating bounded staleness can utilize the backend's continuously synchronized state.
Impact and Future
The adoption of GitFarm led to significant improvements. A code ownership service reduced CPU consumption from over 70 cores to 16 and memory from 400 GB to 32 GB across six hosts, with startup times falling from 15-20 minutes to less than one minute. A compliance auditing service saw median latency drop from 110-160 seconds to 20-30 seconds. GitFarm's roadmap includes features like streaming Git output, sparse checkouts, bare workspaces, and longer-lived sessions, indicating continued evolution for advanced use cases like coding agents.