Google, OpenAI, Roblox, and Discord have partnered to launch a nonprofit organization to help improve the safety of children online - The Robust Open Online Safety Tools ( ROOST). The new initiative will aim to make basic security technologies more accessible and provide access to AI tools for detecting child sexual abuse materials. This was reported by The Verge.
Former Google CEO and ROOST founding partner Eric Schmidt shared that this initiative was partly driven by the changes that generative AI has brought to the online environment. He added that the organization will start with a platform focused on child protection and later focus on creating a safer internet for everyone.
While there are few details about the tools ROOST will provide to companies, the official statement refers to a "free and easy-to-use open source tool for identifying, reviewing and reporting child sexual abuse material (CSAM)." It will use extensive language models, and ROOST's dedicated teams will work with organizations to help integrate the new security measures.
Among ROOST's founders are also representatives of Roblox and Discord, where CSAM issues are quite serious. As of 2020, more than half of the children in the United States were registered on Roblox, where they could be exposed to inappropriate content. In addition, in 2022, both companies faced lawsuits for failing to fulfill their duty to protect children from receiving messages from unsupervised adults.
So far, ROOST has raised more than $27 million to fund its work for the first four years. The initiative is supported by philanthropic organizations such as McGovern Foundation, Future of Online Trust and Safety Fund, Knight Foundation, and AI Collaborative. Other partners include Microsoft, Mozilla, Bluesky, and others. In addition, ROOST will receive support from experts in the field of child protection, as well as in the areas of artificial intelligence, open source technologies, and countering violent extremism.