Behind the Scenes: Building a ... Note

Behind the Scenes: Building a Robust Ads Event Processing Pipeline

At Netflix, a robust event processing platform was built to monitor, measure, and optimize ad campaigns. The ad serving system relies on a steady stream of ad events to adjust decisions, frequency capping, pacing, and personalization. The initial ad event handling system consisted of three main components: the Microsoft Ad Server, Netflix Ads Manager, and Ad Event Handler. The system was designed to ensure the feedback loop functioned effectively, providing insights on impressions, frequency capping, and monetization processes. As the business expanded, a new persistence layer using Key-Value abstraction was introduced to address challenges such as growth in data volume and third-party tracking URLs. The event processing pipeline was further evolved to support in-house advertising technology, incorporating features such as frequency capping, pricing information, and robust reporting system. A centralized ad event collection system was planned, providing a single unified data contract to consumers and separating concerns between upstream systems and consumers. The new pipeline supported various functions such as measurement, finance/billing, reporting, frequency capping, and maintaining an essential feedback loop back to the ad server. The development of the ads event processing systems has been a carefully orchestrated journey, showcasing teamwork, planning, and coordination across various teams. The new system has significantly accelerated the ability to launch new capabilities for the business, supporting programmatic buying capabilities, sharing opt-out signals, and ensuring accurate reporting and measurement.
CdXz5zHNQW_x4VyedRjxa.png