Netflix TechBlog | Medium
Follow
Foundation Model for Personalized Recommendation
Netflix is developing a foundation model for recommendations to centralize member preference learning and improve efficiency across various recommendation models. The current system has specialized models that are costly to maintain and struggle to share innovations. The foundation model aims to learn from extensive user interaction histories and content data, distributing these learnings to other models. Inspired by large language models, the approach shifts to a data-centric strategy using semi-supervised learning. Netflix tokenizes user interactions, balancing data granularity and sequence compression to handle long interaction histories. Sparse attention mechanisms and sliding window sampling are used to manage computational efficiency during training. Each token contains rich, heterogeneous information about the action and content, utilizing request-time and post-action features. The model employs an autoregressive next-token prediction objective, similar to GPT, but with modifications to account for varying interaction importance. The model predicts multiple tokens and uses auxiliary prediction objectives to capture long-term dependencies and improve accuracy. Addressing entity cold-starting, the model is built with incremental training and the ability to infer with unseen entities by using metadata information of entities and inputs.