How Netflix Simplified Batch C... Note

How Netflix Simplified Batch Compute with Kueue

Netflix is transitioning its compute infrastructure to a Kubernetes-native model, integrating components like Kueue into its Titus platform. Kueue, a cloud-native job queueing system, has largely replaced the custom logic of their homegrown batch solution, Compute Managed Batch (CMB). Motivations for this migration included CMB's age, the evolution of Kubernetes, and the growing difficulty of adding features like preemption to the existing system. Kueue was chosen over alternatives due to its compatibility with existing Titus scheduling profiles, its adoption momentum, and its native support for features like preemption.The migration, known as Netflix Batch, aimed for zero user impact and no reduction in throughput. The process involved rerouting job submissions to Kueue-enabled Titus cells, with Titus federation handling job distribution. Operator adjustments were minimal, primarily involving a simple UI toggle to enroll tenants into Kueue. This enrollment translates CMB's existing tenant hierarchy and capacity configurations into Kueue's concepts of Cohorts, ClusterQueues, and LocalQueues.Key lessons learned include maintaining API parity with the old system to ensure a smooth user experience and migrating complex use cases early to build confidence. It was also crucial to load test Kueue to ensure it could handle Netflix's high throughput requirements, necessitating adjustments to default configurations. Kueue is now fully operational, managing millions of batch workloads, and has enabled enhanced fair sharing and preemption features. These improvements allow for better utilization of reserved capacity, reduced job starvation, and faster turnaround for critical workloads, significantly increasing average resource utilization.
CdXz5zHNQW_CNU9jjDeMi.png