Netflix TechBlog | Medium
Follow
Scaling LLM Post-Training at Netflix
Netflix developed a post-training framework to adapt large language models (LLMs) for their specific needs in recommendation, personalization, and search. This framework addresses the complexities of post-training at scale, including data pipelines, distributed computing, and workflow orchestration. The framework aims to help model developers focus on model innovation rather than infrastructure. Key challenges in post-training include data preparation, model sharding, and setting up training loops for production. The Netflix framework provides modular components for data, model, compute, and workflow management to simplify training. The framework supports SFT, DPO, RL, and knowledge distillation training recipes with reusable utilities. The framework evolved to support RL by integrating open-source tools and adopting a hybrid execution model. It prioritizes compatibility with the Hugging Face ecosystem to facilitate easy integration of models and tools. The post-training framework allows teams to experiment with advanced techniques and iterate more quickly.