Leading AI companies have come together to implement robust child safety measures in the development, deployment, and maintenance of generative AI technologies.
One of the potential risks associated with generative AI is the potential for its misuse in creating or spreading child sexual abuse material (AIG-CSAM) and other forms of sexual harm against children. As these technologies become more sophisticated and accessible, it is crucial to establish clear guidelines and safeguards to prevent such abuse.
As part of the Safety by Design effort, Anthropic Amazon, Civitai, Google, Meta, Metaphysic, Microsoft, Mistral AI, OpenAI, and Stability AI have pledged to adhere to a set of principles aimed at mitigating the risks posed by generative AI to children. These principles encompass three key areas: development, deployment, and maintenance.
1. Development:
- Responsibly sourcing training data to avoid ingesting content with known risks of containing CSAM and CSEM.
- Detecting, removing, and reporting CSAM and CSEM from training data at ingestion.
- Conducting red teaming and stress testing of models for AIG-CSAM and CSEM.
- Defining specific training data and model development policies.
- Prohibiting customer use of models to further sexual harms against children.
2. Deployment:
- Detecting abusive content (CSAM, AIG-CSAM, and CSEM) in inputs and outputs.
- Including user reporting, feedback, or flagging options.
- Incorporating enforcement mechanisms and prevention messaging for CSAM solicitation.
- Utilising phased deployment and monitoring for abuse in early stages before broad launch.
- Incorporating a child safety section into model cards.
3. Maintenance:
- Using the Generative AI File Annotation when reporting to NCMEC.
- Detecting, reporting, removing, and preventing CSAM, AIG-CSAM, and CSEM.
- Investing in tools to protect content from AI-generated manipulation.
- Maintaining the quality of mitigations and disallowing the use of generative AI to deceive others for the purpose of sexually harming children.
- Leveraging Open Source Intelligence (OSINT) capabilities to understand potential abuse by bad actors.
For example, Anthropic, has made it a priority to implement rigorous policies, conduct extensive red teaming, and collaborate with external experts to ensure the safety of their AI models. The company strictly prohibits content that describes, encourages, supports, or distributes any form of child sexual exploitation or abuse, and is committed to reporting any detected material to the National Center for Missing & Exploited Children (NCMEC).
The commitment to child safety principles by leading AI companies marks a significant step forward in addressing the potential risks posed by generative AI.
However, challenges and limitations remain, such as the constant evolution of AI capabilities and the potential for bad actors to find new ways to exploit these technologies. Ongoing research and collaboration among AI companies, child safety organisations, and regulatory bodies will be essential in staying ahead of these threats and ensuring the continued protection of children.
By prioritising the safety and well-being of children in the development, deployment, and maintenance of these technologies, the industry is taking a crucial step towards creating a more responsible and ethical AI ecosystem. As generative AI continues to advance and shape our world, it is imperative that we remain vigilant and committed to protecting the most vulnerable members of our society.