Meta's new AI model tags and tracks every object in your videos

Meta's new AI model, Segment Anything Model 2 (SAM 2), can label and track objects in a video as they move, enhancing video editing and analysis capabilities. Unlike its predecessor, which was limited to images, SAM 2's real-time segmentation represents a significant technical advancement. This ability allows AI to process moving images and distinguish between elements, even if they move out of and back into the frame. Segmentation is the process of determining which pixels belong to which objects, simplifying the editing of complex images. The original SAM facilitated various applications, such as analyzing coral reefs, aiding disaster relief with satellite images, and detecting skin cancer in cellular images. SAM 2 expands these capabilities to video, which was recently made possible due to advancements in AI. Meta has released a database of 50,000 videos for training SAM 2, on top of 100,000 previously used videos. While SAM 2 is currently open and free, its real-time video segmentation requires significant computing power, indicating it might not remain free indefinitely. SAM 2 can revolutionize video editing by allowing easier manipulation of objects within scenes and interactive videos. Meta envisions its use in training computer vision systems for autonomous vehicles, where accurate object tracking is crucial. Despite the hype around AI-generated videos, SAM 2's editing capabilities could be more influential in integrating AI into video creation. Other companies, like Google and Adobe, are also developing AI tools for video summarization and editing, showing a competitive landscape in AI video technology.