Block's Monorepo Migration: Tackling Dependency Drift in JVM Ecosystems
Block, Inc. migrated approximately 450 JVM repositories into a monorepo for Cash App and Square to address significant dependency management and coordination challenges inherent in a polyrepo architecture. This shift aimed to simplify cross-service development, improve dependency visibility, and reduce operational friction, ultimately enhancing developer experience and CI/CD efficiency for their distributed systems. The article details the motivations, implementation strategies, and resulting benefits and challenges of this large-scale architectural change.
Read original on InfoQ ArchitectureMotivation for Monorepo Adoption
Block faced increasing coordination and dependency management challenges with its polyrepo architecture, particularly across its large JVM-based backend systems. Key issues included dependency version drift, duplicated upgrade efforts across numerous repositories, and a high risk of runtime incompatibilities and "diamond dependency surprises" in their distributed services. Rolling out library or API changes became a "heroic effort" due to the extensive coordination required.
Architectural Shift and Key Benefits
The migration consolidated around 450 JVM repositories into a single monorepo. This change enabled atomic updates across services in a single commit, resolving shared dependencies directly from source rather than through independently versioned internal libraries. This drastically improved dependency consistency and reduced the risk of runtime failures, leading to a "step-change in developer experience" with a more cohesive codebase.
Monorepo vs. Polyrepo
While polyrepos offer autonomy to individual teams, they can introduce significant operational overhead in large, interconnected systems due to versioning complexities and coordination challenges. Monorepos centralize code, simplifying dependency management and enabling atomic changes, but require substantial investment in tooling and platform teams to scale effectively. The choice depends on organizational structure, project size, and investment capacity for infrastructure.
Engineering Challenges and Solutions
- Build Performance & Scoping: Block implemented dependency graph-based build scoping to determine which projects needed to be built and how, based on modified files. This allowed for selective CI checks, dropping unnecessary steps for indirect changes.
- IDE Workflow: To manage the scale within IDEs, Block developed a custom IntelliJ plugin that loads only specified projects, dynamically swapping project references with published JARs for dependent projects. This kept IDEs fast and manageable.
- CI Stability & Throughput: To maintain a reliably green main branch under high commit volume (8,800 builds/week), merge queues were adopted. Investment in shared Gradle plugins and Git performance tuning (e.g., sparse checkouts) were also crucial.
- Ownership & Boundaries: Repo ownership tooling and strong Gradle convention plugins defined common module types (e.g., proto, service, library modules) to enforce dependencies and roll out improvements consistently.
Lessons Learned and Recommendations
The migration emphasized the critical need for a properly funded platform team to support a monorepo. Without such investment, a monorepo can suffer greatly from neglect, leading to poor developer experience and performance issues. Monorepos are also most beneficial for projects with similar languages, frameworks, and styles; diverse projects might not see significant gains to justify the investment.
Example: Scalable Monorepo CI/CD
Imagine a monorepo with hundreds of microservices. When a change is committed, a CI system uses a dependency graph to identify only the affected services and their transitive dependencies. It then selectively runs tests and builds for only those components, leveraging caching for unchanged artifacts. A merge queue ensures that only code passing all checks is merged to the main branch, maintaining stability even with high commit rates.