Automating Feature Flag Cleanup with Multi-Agent LLMs at DoorDash
DoorDash developed a multi-agent LLM system to automate the cleanup of stale feature flags across its vast codebase. This system integrates with live experimentation data, uses isolated Git worktrees for safe code modification, and includes automated validation, significantly reducing manual effort and improving developer efficiency in managing a large-scale feature flagging infrastructure.
Read original on InfoQ ArchitectureManaging tens of thousands of feature flags in a large-scale distributed system like DoorDash presents significant operational overhead. Stale flags accumulate, creating technical debt, increasing code complexity, and potentially introducing subtle bugs. Manually identifying, cleaning up, and validating changes across hundreds of repositories and a complex dependency injection pattern is time-consuming and error-prone.
The Challenge of Stale Feature Flags
- DoorDash manages over 60,000 feature flags across 623 repositories, with 2,300 new flags monthly.
- Over 1,000 stale flags identified (not modified for 90 days, referenced in code, not archived/retired).
- Complexity due to dependency injection: a single Boolean flag change can span 5-20 files, including tests.
Multi-Agent LLM System Architecture
DoorDash's solution leverages a multi-agent LLM system, built on Google's Agent Development Kit, to streamline the cleanup process. This architecture focuses on safety, efficiency, and scalability, addressing the limitations of syntax-based approaches like Uber's Piranha for their specific dependency injection patterns.
Two-Phase Workflow
- Orchestration Phase: An orchestrator (Claude Sonnet) retrieves stale flag tickets from Jira, searches relevant repositories, and queries the experimentation platform (via Model Context Protocol - MCP) for metadata. An engineer reviews this report to confirm the target value before code changes begin.
- Cleanup Phase: Multiple Claude Opus cleanup agents operate concurrently in isolated Git worktrees. Each agent identifies flag references, determines cleanup strategy, modifies source code and tests, and runs builds, tests, JaCoCo coverage, and static analysis. Pull requests are opened only after all validation checks pass. Agents have a one-hour timeout, and Gradle runs without its daemon to prevent state sharing.
Architectural Insight
The use of isolated Git worktrees is a critical design decision for safety and concurrency. It allows multiple agents to operate in parallel without interfering with each other's state or the main codebase, ensuring atomicity and preventing side effects until changes are fully validated. This pattern is essential for any automated code modification system in a large monorepo or multi-repo environment.
The system demonstrated significant efficiency gains, producing usable pull requests for 45 out of 50 evaluated stale flags, averaging 13.8 minutes and $4.79 per cleanup, compared to 1-2 hours for manual cleanup. This approach showcases how AI can be integrated into developer experience and DevOps workflows to manage technical debt at scale.