xAI has been accused of using child sex abuse materials (CSAM) to train its Grok model, according to a lawsuit filed by a plaintiff known as Jane Doe. The complaint alleges that Doe's images, which were hashed by organizations like the National Center for Missing and Exploited Children (NCMEC), were identified in AI-generated CSAM on xAI's platform. Doe, who was a preschooler in the early 2000s when she was raped to create CSAM, claims that xAI used her images to train Grok, both the original set and AI-generated versions. This marks the first case to accuse xAI of training on CSAM, with the complaint stating that Doe’s images were part of the dataset used to build Grok’s image and video generating capabilities. The lawsuit further notes that xAI’s terms do not specify whether CSAM, non-consensual intimate imagery, or NSFW material are excluded from training data. According to Doe’s complaint, because full removal of a training example’s influence from an already-trained model is technically difficult, any CSAM ingested into training before takedown likely continued to shape the model’s outputs even after the original images were removed from public view. The lawsuit seeks to block Grok from generating CSAM and to destroy all such content xAI may be storing on its servers. Doe’s proposed class action would represent every victim whose childhood images have been used to generate Grok CSAM. The lawsuit accuses xAI of violating federal child pornography laws and Masha’s Law, which give CSAM survivors a right to sue on claims of production, possession, and distribution. A lawyer for Doe, Margaret E. Mabie, suggested that xAI did all three. If Doe wins, xAI could owe money damages to every victim who can prove that Grok generated CSAM based on their real photos. The lawsuit also asks the court to order xAI to block Grok from ever generating CSAM, including NCII and NSFW content that Musk has promoted. Sarah London, one of Doe’s lawyers, said in the press release that Doe has lived for nearly two decades knowing that images of the worst thing that ever happened to her are circulating among predators online, and that they can resurface at any moment. xAI must be held responsible for knowingly training its models on images of the horrific abuse she suffered, and on the abuse images of every other survivor in this class. X did not respond to Ars’ request to comment. Ashley Belanger Senior Policy Reporter Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience. 38 Comments
Source: arstechnica
