adminRoblox AI safety tools are being made available to other organizations through an open-source safety initiative. The contribution includes three models already used on Roblox’s platform, along with evaluation resources intended to help organizations assess moderation systems and adapt them to their own needs.
Image credit: Roblox
Which Roblox AI safety tools are being shared?
Roblox is contributing its PII Classifier, Roblox Sentinel, and its latest voice safety classifier to the Robust Open Online Safety Tools Model Community (ROOST). ROOST is an independent nonprofit that develops and maintains open-source infrastructure for online safety. Roblox co-founded the community in 2025 with Google, OpenAI, and Discord.
PII Classifier
The PII Classifier is designed to identify attempts to share or request personally identifiable information. It can detect not only direct references, but also misspellings, coded language, and disguised mentions of other platforms. Roblox is also contributing an evaluation dataset with synthetic examples of English-language multiuser chats modeled on common ways people may try to evade personal-information filters.
For organizations considering a similar classifier, evaluation should include indirect wording and attempts to bypass filters—not just straightforward requests. Testing against realistic examples can help reveal gaps before a system is used in practice.
Roblox Sentinel and voice moderation
Roblox Sentinel is intended to identify early signs of potential child endangerment. The latest voice safety classifier moderates voice chat in real time and incorporates feedback from the ROOST community. Roblox says its newest evaluation dataset supports moderation across 30 languages and eight violation categories.
Language coverage is an important consideration when assessing moderation tools. Organizations should check whether a model performs consistently across the languages and types of violations relevant to their users; the stated coverage alone does not establish how accurately it handles every situation.
Why share moderation models and evaluation data?
Roblox says the shared models can give other platforms a starting point for training and tuning their own moderation tools. Vice president of trust and safety engineering Naren Koneru said the company also hopes to learn from feedback and findings contributed by other organizations. The company described online safety as a shared responsibility and said it plans to continue collaborating through ROOST.
Open-source models and datasets can support evaluation and adaptation, but organizations still need to assess whether a tool suits their users, policies, and operating environment. Reviewers should pay attention to language coverage, the types of harmful behavior included in testing, and how the system is monitored after deployment.
Roblox faces a separate child-safety investigation
The announcement comes amid scrutiny of Roblox’s approach to child safety. Earlier this week, the US Senate Judiciary Subcommittee on Crime and Counterterrorism announced an investigation, saying the platform appeared to prioritize revenue and engagement metrics over child safety. Roblox has been asked to provide records concerning child safety, sexual or physical abuse, bullying, and financial exploitation by August 31, 2026.
Roblox has introduced other safety measures in recent years, including direct-messaging restrictions for users under 13, global facial-recognition checks, a “sensitive issues” content tag for experiences, and enhanced moderation policies. These measures are separate from the models being shared through ROOST.
What the Roblox AI safety tools mean for other platforms
The release gives other organizations access to models and testing resources for several distinct moderation tasks, from identifying personal-information sharing to reviewing voice chat. Their practical value will depend on careful evaluation and adaptation. Roblox AI safety tools are a contribution to a broader community effort, rather than a substitute for platform-specific safety policies and oversight.
