From static classifiers to reasoning engines: OpenAI’s new model rethinks content moderation

Enterprises fine-tune LLMs for safety, but OpenAI introduces a more flexible approach with its open-weight models, gpt-oss-safeguard-120b and gpt-oss-safeguard-20b. These models, available under the Apache 2.0 license, interpret developer policies at inference time using chain-of-thought reasoning, providing explanations for decisions. This allows for iterative policy revisions, offering more flexibility compared to training classifiers. OpenAI emphasizes that these models enable adapting to evolving harm, handling nuanced domains, and working with limited training samples. The models take both policy and content as input, determining potential guideline violations. Based on OpenAI's internal Safety Reasoner, the gpt-oss-safeguard models outperformed previous models in benchmark tests. However, concerns arise about centralizing safety standards and potentially institutionalizing OpenAI's perspective. Despite not releasing the base model, OpenAI hopes the developer community will refine gpt-oss-safeguard, hosting a Hackathon to encourage further development.