Scaling AI Agents on Kubernetes with Agent Substrate: Decoupling Workload Lifecycle from Compute
This article introduces Agent Substrate, an open-source project that rethinks how AI agents and similar bursty, stateful workloads are managed on Kubernetes. It proposes decoupling the logical lifecycle of an agent from the physical compute allocated to it, allowing for significantly higher density and faster activation of stateful sessions by suspending and resuming agents across a shared pool of worker Pods.
Read original on DZone MicroservicesThe Challenge with Traditional Kubernetes Pods for AI Agents
Traditional Kubernetes deployments often couple a workload's logical lifecycle directly to a Pod's runtime lifecycle. For applications like AI agents, which are often stateful, sandboxed, and predominantly idle, this one-agent-per-Pod model leads to inefficiencies. Compute resources are unnecessarily consumed for idle workloads, and the overhead of creating and initializing new Pods for short, active periods can introduce noticeable latency.
Agent Substrate's Core Thesis: Make Running Optional
Agent Substrate's central idea is that an agent (termed an "actor") doesn't need to perpetually occupy compute resources. It introduces a system that can suspend an actor into a snapshot (preserving process memory and filesystem state), freeing the worker Pod. When a request arrives, the actor is restored onto an available worker, and the request is routed. This allows a smaller pool of "warm" worker Pods to serve a much larger population of actors over time, with state surviving worker reassignment.
- Warm Capacity, Not Per-Session Capacity: A shared pool of ready workers serves numerous actors.
- State Survives Reassignment: Actors can be restored to any available worker without losing state.
- Request-Initiated Activation: The router can hold requests while a suspended actor is brought online.
Decoupling Lifecycles: Infrastructure, Workload, and Execution
A key architectural departure is the separation of three distinct lifecycles: infrastructure, workload, and execution. Kubernetes continues to manage infrastructure capacity (worker Pods). An agent-specific control plane manages the logical identity, placement, and lifecycle of actors. Snapshot and sandbox mechanisms handle the preservation and reconstitution of execution state. This transforms a worker Pod into a reusable execution slot rather than the sole identity of the workload itself.
Three Planes of State
| State Type | Where it Lives | Why |
|---|
Architectural Insight
By separating high-churn runtime state (actors, workers, assignments) from low-frequency desired state (Kubernetes CRDs), Agent Substrate avoids overwhelming the Kubernetes API machinery with frequent lifecycle operations, using a dedicated low-latency control-plane store (like Redis/Valkey) instead of etcd for dynamic state management.