Netflix TechBlog | Medium
Follow
Single-tenant Titus: VM-level isolation on a multi-tenant container platform
Netflix utilizes Titus, a Kubernetes-based platform, to host millions of containers weekly across numerous AWS regions. Titus offers a managed container experience, handling host, cluster, and kernel maintenance. It is designed for multi-tenancy, efficiently sharing hosts among many containers from different teams. However, this shared-kernel model presents challenges for demanding real-time systems due to hardware resource contention.These issues arise because containers share physical resources like last-level caches and memory bandwidth, which cgroups cannot fully isolate. Noisy neighbors can exhaust these shared resources, leading to performance degradation and increased tail latency. Network bandwidth and packet processing, storage IOPS, and various kernel-level operations also contribute to interference. Stateful, high-throughput streaming jobs are particularly vulnerable to these intermittent interferences, which can cause significant downtime.To address this, Netflix explored several isolation strategies, including continued shared-kernel tuning, microVMs, gVisor, and running containers directly on EC2. Shared-kernel tuning offers incremental improvements but doesn't fully solve the structural problem of shared hardware. MicroVMs and gVisor provided stronger isolation but came with performance overhead, operational complexity, and hardware constraints making them less appealing. Running containers directly on EC2 would shift operational burden back to application teams, negating Titus's benefits.Ultimately, Netflix opted to integrate single-tenant Titus, offering dedicated VMs for specific workloads, into the existing Titus platform. This approach provides full VM isolation while maintaining the familiar Titus API and managed experience. The Compute/Platform team still owns and patches the underlying EC2 instance, ensuring a consistent operational model. This solution delivers the necessary resource and performance isolation without imposing unnecessary security isolation features or significant operational overhead.