RSS Google AI Blog
Follow
Introducing interactive on-device segmentation in Snapseed
Selective image adjustments make photos better by allowing targeted enhancements. Previously, isolating objects for editing was difficult, especially on mobile devices with imprecise touch controls and limited processing power. Snapseed on iOS now introduces the Object Brush, making these adjustments quick and easy. The Object Brush allows users to simply draw a stroke on an object to select it for individual editing. This intuitive feature is powered by an on-device AI model called the Interactive Segmenter. This advanced model can detect and select entire objects or people in less than 20 milliseconds after a simple tap or tracing a line. The model generates an accurate mask for the selected object, adapting to its boundaries. Training the Interactive Segmenter involved a Big Transfer approach and knowledge distillation from a larger teacher model to a smaller, efficient edge model. This process ensures high-quality segmentation while maintaining real-time responsiveness. The system decouples image and prompt understanding into distinct sub-models to balance segmentation quality with low latency. Finally, image-size mask upsampling ensures high-resolution editing quality for detailed adjustments.