Ensuring Safety in the Generat... Note

Ensuring Safety in the Generative AI Ecosystem: Protecting Users from Non-Consensual Intimate Content

Google Play emphasizes that user safety and developer success are intertwined. The platform is seeing growth in apps incorporating AI-generated features, which offer creative possibilities. However, these AI advancements also introduce new safety challenges, particularly the creation and distribution of non-consensual intimate imagery (NCII). Google Play strictly prohibits such content and is committed to removing harmful applications. The company is investing heavily in platform protections, technical defenses, and developer resources to combat abuse. Safeguarding the ecosystem involves multi-layered defenses across the app lifecycle, continuous testing for robust controls, and cutting off monetization pathways for bad actors. Google Play also collaborates with third-party organizations and AI safety research groups to identify and address NCII abuse. Developers are encouraged to streamline app reviews by providing clear visibility into their AI features and safety guardrails. Building apps for safety includes stress-testing against adversarial prompts and integrating customized input/output moderation controls. Developers are reminded of their accountability for app advertisements and are urged to leverage user interactions as signals for continuous improvement and safety. Responsible AI innovation, coupled with safety and user trust, is a key focus for Google Play's ongoing support of developers.