Multi‑Ratio Video Resizing for... Note

Multi‑Ratio Video Resizing for Mobile Feed and Marketing

Netflix developed an automated multi-ratio video cropping system to efficiently create vertical video assets for its mobile app and marketing. This system replaces a manual reframing process that was not scalable as video content grew. The approach treats video resizing as a tracking and temporal optimization problem, processing assets in three stages. Initially, videos are segmented into shots, and then further split into sub-shots using active speaker detection to better follow conversations.The system then identifies a focus region within each frame by combining face detections with visual saliency maps. It prioritizes faces based on factors like active speaker status and relative size to determine the primary subject. Finally, temporal reframing is applied to ensure smooth camera movement. Small focus region displacements result in static crops, while significant motion triggers a smoothed, dynamic tracking window.This resizing algorithm is integrated into Netflix's internal Clips tool, allowing editors to generate and adjust multiple aspect ratios. For external use, resizing parameters can be exported as XML for integration into editors like Adobe Premiere. This automated system outperforms tools like Premiere's Auto Reframe in tests. The pipeline significantly reduces production time and manual effort while maintaining creative control. This advancement is part of Netflix's ongoing effort to scale content creation without sacrificing creative intent. The goal is to free up creatives' time and enhance the viewing experience.
CdXz5zHNQW_ODmB0vpPkr.jpeg