Netflix TechBlog | Medium
Follow
The Human Infrastructure: How Netflix Built the Operations Layer Behind Live at Scale
Netflix's live streaming operations have rapidly expanded since their first live show, now streaming multiple events daily. Initial live shows relied on engineers and improvised setups, lacking dedicated operational teams. The evolution required building dedicated Broadcast Operations Centers (BOCs) to manage the live broadcast process. To ensure signal reliability, Netflix enforces strict specifications on video and audio feed contributions from venues. Netflix's operational model evolved through several phases, from engineering-led operations to specialized engineering teams. The Transmission Operations Center (TOC) model was developed to improve efficiency and manage a high volume of events concurrently. Within the TOC model distinct roles like Transmission, Streaming, and Broadcast Control Operators are crucial. For high-profile events, a specialized "Big Bet" model provides dedicated resources. The Live Command Center (LCC) monitors the entire live streaming pipeline, offering real-time data to operators. The LCC uses a custom observability stack to manage massive data volumes and react to issues promptly. LCC Operations Leads and Technical Launch Managers are responsible for incident response.