MAPS: Netflix’s Multimodal Ass... Note

MAPS: Netflix’s Multimodal Asset Personalization at Scale

Netflix aims to connect members with content they'll love through personalized visual assets like artwork and video previews. A significant challenge arises with new titles, where insufficient interaction data prevents effective personalization, a problem known as the cold-start issue. Traditionally, models treated assets as opaque IDs, relying on popularity heuristics until enough data accumulated. This meant personalization for new content was delayed, impacting the discovery experience.Netflix's solution involves using multimodal embeddings to allow models to "see" and "hear" the assets. By encoding artwork with CLIP, a pre-trained image-text model, assets gain visual understanding. This CLIP embedding is concatenated with the asset's ID embedding, creating a richer representation that captures visual themes and talent. This enables personalization based on member preferences for visual styles, even before the asset has any interaction history, facilitating knowledge transfer across titles.This approach also enables model consolidation, merging separate models for different artwork canvases into a single unified model. Because CLIP embeddings are largely invariant to cropping and resizing, near-identical renderings of artwork map to similar vectors. This pooling of interaction signals across canvases, especially benefiting low-data canvases, allows a single model to personalize all artwork placements. To address data imbalance across canvases, reward-based weighting is used, prioritizing interactions based on their long-term value rather than impression volume.The effectiveness of this approach was validated through offline evaluations using inverse propensity scoring and large-scale A/B tests. Combining image embeddings with a unified model (V3) significantly outperformed models with only one of these improvements. This combined approach proved crucial during a major Netflix home-screen redesign, where a dominant canvas with minimal historical data was introduced. The unified model with CLIP embeddings successfully personalized content for this new layout, showing statistically significant gains in discovery metrics and streaming hours, highlighting the power of multimodal embeddings for overcoming the cold-start problem.