Netflix TechBlog | Medium
Follow
Integrating Netflix’s Foundation Model into Personalization applications
Netflix developed a Foundation Model for personalization to centralize member preference learning. This powerful model, trained on extensive user interaction and content data, distributes its learnings to other models. Integrating this large model into production systems presented a challenge, leading Netflix to experiment with three distinct approaches: embeddings, subgraph, and fine-tuning. Each method caters to different application needs regarding latency, tech stack, and model utilization.The embeddings approach leverages pre-computed user profile and item representations from the Foundation Model, stored in an Embedding Store. While straightforward and cost-effective, it can lead to stale recommendations due to the time gap between computation and inference. Netflix enhances this with daily fine-tuning and an embedding stabilization technique. A near-real-time framework is being developed to address staleness and improve adaptiveness.The subgraph integration treats the Foundation Model decoder as a component within an application's larger model graph. This allows for deeper integration, uses up-to-date learnings, and unlocks application-specific value by accessing specific Foundation Model layers. However, it introduces complexity in feature generation and increases model size and inference time. This approach is reserved for high-impact use cases where metric improvements justify the costs.The fine-tuning approach adapts the Foundation Model for specific applications by further training it on product-specific data and objectives. This offers optimized performance and serves as a baseline for new applications. While requiring careful optimization for latency and SLAs, it can lead to a proliferation of models and pipelines. Netflix provides a fine-tuning framework to simplify this process.Ongoing innovations include near-real-time embedding inference and smaller, distilled Foundation Model versions to reduce latency. Standardized APIs and a robust fine-tuning framework aim to make leveraging the Foundation Model easier for application teams. These three integration patterns provide a comprehensive strategy for Netflix applications to benefit from the Foundation Model.