Anthropic's Approach to Building AI Platform Infrastructure
This article provides insights into how Anthropic approaches software engineering, particularly for complex AI platform projects like Claude Managed Agents. It highlights architectural decisions, such as migrating from Python to Rust for performance, and practical engineering processes, including internal dogfooding and iterative re-architecture, that are relevant to designing scalable AI infrastructure.
Read original on The Pragmatic EngineerThe Evolving Landscape of Software Engineering at Anthropic
The article explores how AI tooling is influencing software development at Anthropic, focusing on both changes and continuities in engineering practices. It delves into the development of Claude Managed Agents, a complex infrastructure project, and touches upon the rapid re-writing of Bun to Rust, demonstrating the impact of AI-assisted development.
Claude Platform: API and Infrastructure Layer
Anthropic's Claude Platform team is responsible for the API and underlying infrastructure that powers Claude models. This layer handles critical tasks like tokenization, safeguards, and billing. Initially developed in Python for rapid iteration and researcher familiarity, the platform is undergoing a migration to Rust to address performance bottlenecks and single-threaded limitations under high load.
Token Hot Path
The Claude Platform operates on the "token hot path," meaning it processes user prompts, tokenizes them, and applies safeguards and billing logic before model inference. This critical path demands high performance and reliability.
Designing Claude Managed Agents: Lessons Learned
The development of Claude Managed Agents, a pre-built harness for production agents, took six months. Key architectural lessons from this project include:
- Upfront Planning is Crucial for Complexity: For complex projects like Managed Agents, extensive upfront architecture and planning, including detailed Product Requirements Documents (PRDs) and cross-team alignment, remain essential, even in an AI-first environment.
- Internal Dogfooding for Validation: The team built an internal version for Claude Code first to stress-test the architecture and gather feedback, proving invaluable for surfacing hard problems related to reliability, scalability, credentials, and access control.
- Iterative Re-architecture: Based on learnings from internal use, the architecture was re-designed to decouple components like the "brain" (Claude and its harness), "hands" (sandboxes and tools), and "session" (event log) into loosely coupled interfaces.
- Secure Credential Handling: A significant architectural decision involved abstracting vaults and credentials. Calls using credentials are made via a proxy with a session token, fetching credentials from a vault. Credentials are only injected at the egress boundary, never seen by the agent, sandbox, or session.
Impact of AI on Development Practices
While some core engineering principles endure, AI is significantly altering others. Prototyping has become more fluid, and verification is increasingly time-consuming relative to implementation. AI-assisted code review and testing are becoming more prevalent. This suggests a shift towards optimizing the verification and iteration cycles with AI, while core design principles for complex systems remain foundational.