Netflix's Single-Tenant Titus: VM-level Isolation on a Multi-Tenant Container Platform
This article discusses Netflix's architectural decision to introduce single-tenant virtual machine (VM) isolation within their multi-tenant Titus container platform. It explains the limitations of shared-kernel isolation for critical, performance-sensitive workloads and the trade-offs considered (microVMs, gVisor, EC2) before opting for dedicated EC2 instances per task, managed seamlessly by Titus.
Read original on Netflix Tech BlogNetflix's Titus is a multi-tenant container platform built on Kubernetes, running millions of containers weekly across tens of thousands of EC2 instances. While multi-tenancy offers efficiency by packing many containers onto shared hosts, certain demanding, real-time workloads face significant challenges due to 'noisy neighbor' issues arising from shared physical hardware resources.
Limitations of Shared-Kernel Isolation
Traditional container isolation mechanisms (cgroups, namespaces, seccomp) are effective for partitioning CPU, memory, and disk at the kernel level. However, they cannot fully isolate shared hardware resources such as last-level cache, memory bandwidth, network bandwidth, and storage I/O. Contention for these resources can lead to performance degradation, increased tail latency, and operational incidents for sensitive applications, even when containers adhere to their assigned quotas. Netflix has previously attempted to mitigate these issues with predictive CPU isolation and eBPF-based noisy neighbor detection, but the problem persists due to the fundamental nature of shared physical resources.
Hardware Contention is Hard to Fence
A crucial takeaway is that while software-defined quotas (like CPU shares) work well for logical resource allocation, physical hardware resources such as CPU cache, memory bus, and network interface queues are inherently shared and cannot be fully partitioned at the kernel level. This leads to performance indeterminacy and 'noisy neighbor' issues that impact tail latency for critical workloads.
Evaluating Isolation Alternatives
- Continued shared-kernel improvements: Offers general benefits but cannot fully resolve hardware-level contention.
- MicroVMs (Firecracker, Kata Containers): Provide excellent isolation by emulating minimal hardware. However, they were not feasible due to reliance on bare-metal or unsupported nested virtualization, high operational overhead of managing a hypervisor, and performance overhead/lack of GPU support.
- gVisor: A user-space kernel offering sandbox-grade isolation without a hypervisor. Suffered from significant performance overhead on I/O-heavy workloads and vendor lock-in concerns.
- Moving workloads to EC2: Provides full VM isolation but shifts the operational burden (patching, scaling, instance selection) back to application teams, negating the benefits of a managed container platform like Titus.
The Solution: Single-Tenant Titus with Dedicated VMs
Netflix decided to offer single-tenant VMs within the existing Titus framework. This approach assigns a dedicated EC2 instance, sized appropriately, to a single critical task. The key innovation is that the Compute/Platform team still owns and manages these EC2 instances, abstracting away the operational complexity from application teams. This provides the desired VM-level performance isolation without changing the developer experience (same Titus API, deployment path, and container packaging). This solution effectively addresses resource and performance isolation needs (rather than security isolation for mutually distrusting tenants) by leveraging the managed nature of AWS EC2 instances under the Titus control plane.