Netflix TechBlog | Medium
Follow
Recommending for Long-Term Member Satisfaction at Netflix
Netflix employs personalization algorithms to recommend content to members, aiming to enhance long-term satisfaction. Recommendations are viewed as a contextual bandit problem, where the system selects actions based on context and feedback from members. Traditional recommender systems optimize for short-term metrics like clicks, which may not fully capture long-term satisfaction. Optimizing for retention alone has drawbacks, so Netflix utilizes proxy reward functions aligned with long-term member satisfaction. Click-through rate (CTR) is a simple proxy reward, but Netflix expands beyond CTR to consider various user actions and their implications on satisfaction. Reward engineering is an iterative process of refining the proxy reward function to align with long-term member satisfaction, involving hypothesis formation, reward definition, bandit policy training, and A/B testing. Netflix addresses the challenge of delayed feedback by predicting missing feedback, enabling the use of all feedback in the proxy reward function. Despite offline model improvements, online-offline metric disparity can occur when the proxy reward is not fully aligned with long-term member satisfaction. Netflix resolves this by further refining the proxy reward definition. Open questions remain, such as automating proxy reward function learning, determining the optimal waiting time for delayed feedback, and leveraging Reinforcement Learning for alignment with long-term satisfaction.