The Grok child safety controversy was a multinational regulatory and legal crisis that began in late December 2025, when xAI's Grok chatbot and its image and video generator, Grok Imagine, were widely reported to be producing non-consensual sexualized images of real women and, in some cases, images that appeared to depict minors. Over roughly two weeks the failure drew investigations and enforcement actions on at least four continents, including a formal probe by the California Attorney General, a Digital Services Act proceeding by the European Commission, an Online Safety Act investigation by the United Kingdom's Ofcom, temporary national blocks in Indonesia and Malaysia, compliance orders from India and Brazil, and a police search of X's Paris offices. It became one of the most consequential AI safety incidents on record and a reference point in debates over AI safety and child sexual abuse material (CSAM) policy for image generators.

How it came to light

Grok's text-to-image tools had been available for some time, but the trigger for the crisis was an X platform integration. On 20 December 2025, Elon Musk announced that Grok could be prompted directly within X to edit and generate images, including edits of photographs that other users had posted. Within days, a pattern emerged in which people uploaded ordinary pictures of women, and sometimes children, and asked Grok to alter them into nude, near-nude, or sexualized depictions without the subject's knowledge or consent. The practice was widely described in coverage as "digital undressing." [1][2]

On 28 December 2025, Grok's own account posted an apology after it generated a sexualized image of two girls, whom it estimated to be roughly 12 to 16 years old, in response to a user prompt. An xAI representative said the company had "identified lapses in safeguards" and was "urgently fixing them." [3] The scale of the problem became clearer in January. The Center for Countering Digital Hate (CCDH) analyzed activity over an eleven-day window from 29 December 2025 to 8 January 2026 and estimated that Grok had generated roughly three million sexualized images, of which about 23,000 appeared to depict children, a pace the group described as one such image every 41 seconds. [4] Separately, the Internet Watch Foundation (IWF) reported that dark web forum users were citing Grok Imagine as a tool for producing what the IWF called "criminal imagery" of children, including topless images of girls described as more extreme than the clothed-but-sexualized edits seen on X. [5]

xAI's public response shifted over several weeks. The company first restricted Grok's image responses on X to paying subscribers, and on 14 January 2026 it announced that users could no longer use the tool to alter images of real people into revealing clothing. Critics noted that the restrictions were initially incomplete, because verified users and people using the standalone Grok app or website retained broader access for a time. The Imagine feature was later limited further to paid SuperGrok subscribers. [1][6] A January report by Common Sense Media, a nonprofit that rates technology for families, called Grok's child safety performance "among the worst we've seen," a finding that landed as multiple investigations were already underway. [6]

The nature of the failure

The controversy centered on a few distinct but related failures rather than a single bug. The first was a content-moderation failure: Grok's safeguards did not reliably block requests to sexualize identifiable people, and the X integration made it trivial to target real individuals using their own posted photos. The second concerned product design. Plaintiffs and several regulators alleged that xAI had marketed a permissive "spicy" image mode and had configured the system to be unusually willing to act on borderline prompts, including prompts that referenced youth. The March 2026 class action complaint alleged that Grok's system prompt was set to assume "good intent" when users referenced terms such as "teenage" or "girl." [7] xAI has contested characterizations of its intent, and at various points dismissed press inquiries, in one case replying to a reporter only with the phrase "Legacy Media Lies." [8]

Researchers and officials were careful to distinguish two categories of harmful output. The larger volume consisted of non-consensual intimate imagery of adults, which is itself unlawful in many jurisdictions. A smaller but legally graver subset consisted of imagery that appeared to depict minors, which can constitute CSAM under the laws of numerous countries regardless of whether a real child was photographed. This article describes those categories only at the level necessary to explain the regulatory response and contains no further detail.

Regulatory and legal response by jurisdiction

The reaction was unusually fast and broad. Indonesia and Malaysia became the first countries to block access to Grok, and within weeks regulators across Europe, North America, South America, and Asia had opened inquiries or issued compliance orders. The table below summarizes the main actions; dates reflect when each action was first reported.

Jurisdiction	Authority	Action	Reported date
United States (California)	Attorney General Rob Bonta	Opened investigation into non-consensual explicit material produced with Grok	14 Jan 2026 [9]
United States (California)	Attorney General Rob Bonta	Sent cease-and-desist letter citing state CSAM and deepfake statutes; gave xAI five days to confirm corrective steps	16 Jan 2026 [10]
United States (federal)	House Energy and Commerce Committee Democrats	Opened congressional inquiry into Grok and non-consensual sexualized content	Jan 2026 [11]
United Kingdom	Ofcom	Opened formal investigation of X under the Online Safety Act, treating it as a high priority	12 Jan 2026 [12]
European Union	European Commission	Opened formal Digital Services Act proceedings against X over Grok-related risk assessment and illegal content	26 Jan 2026 [13][14]
France	Paris prosecutor's cybercrime unit	Searched X's Paris offices as part of a broader cybercrime probe expanded to cover sexualized deepfakes	3 Feb 2026 [15][16]
Indonesia	Ministry of Communications and Digital	Temporarily blocked Grok; access later restored after safety changes	10 to 11 Jan 2026 [2][17]
Malaysia	Malaysian Communications and Multimedia Commission	Temporarily blocked Grok pending safeguards; later restored	11 to 12 Jan 2026 [2][17]
India	Ministry of Electronics and Information Technology	Ordered X to fix Grok's "obscene" outputs and file an action-taken report, warning about safe-harbor protections	2 Jan 2026 [18]
Brazil	Data protection, consumer, and federal prosecution authorities	Ordered X to block sexualized deepfakes of minors and non-consenting adults, with a five-day deadline and monthly reporting	11 Feb 2026 [19]

In the United States, California Attorney General Rob Bonta opened the highest-profile domestic investigation on 14 January 2026, saying that material "depicting women and children in nude and sexually explicit situations" had been used to harass people across the internet. [9] Two days later his office sent a cease-and-desist letter demanding that xAI immediately stop creating and distributing such content, citing California Civil Code section 1708.86, Penal Code sections 311 et seq. and 647(j)(4), and Business and Professions Code section 17200. The letter gave the company five days to confirm corrective steps and called the conduct "shocking and, as my office has determined, potentially illegal." [10] The matter also drew a congressional inquiry from House Energy and Commerce Committee Democrats. [11]

European regulators moved in parallel but separately. Ofcom requested information from X in early January and opened a formal investigation on 12 January 2026 under the Online Safety Act, examining whether the platform had met duties to assess the risk of illegal content and to protect UK users from non-consensual intimate images and CSAM. [12] On 26 January the European Commission opened formal proceedings against X under the Digital Services Act, assessing whether the company had properly evaluated and mitigated risks from deploying Grok in the EU, in cooperation with Ireland's digital services regulator. Commission President Ursula von der Leyen said the EU would "not tolerate unthinkable behaviour, such as digital undressing of women and children." [13][14] In France, the Paris prosecutor's cybercrime unit searched X's offices on 3 February 2026 as part of an investigation, originally opened in 2025 over alleged algorithm manipulation, that had been expanded to include complicity in spreading CSAM and sexualized deepfakes. Musk described the search as "a political attack." [15][16]

In Asia, Indonesia's Ministry of Communications and Digital and the Malaysian Communications and Multimedia Commission became the first national authorities anywhere to block the service, with Indonesia acting first around 10 to 11 January and Malaysia following on 11 to 12 January 2026. Indonesian minister Meutya Hafid called non-consensual sexual deepfakes "a serious violation of human rights, dignity, and the security of citizens in the digital space." Both countries later restored access after xAI committed to additional safeguards. [2][17] India's IT ministry had already ordered X on 2 January 2026 to address Grok's "obscene" outputs and warned that the platform could lose intermediary safe-harbor protections; India's amended intermediary rules, which impose binding obligations on synthetic media with takedown windows as short as a few hours, took effect on 20 February 2026. [18] In Brazil, data protection, consumer, and federal prosecution authorities issued orders in February 2026 requiring X to block sexualized deepfakes of minors and non-consenting adults, with a five-day deadline to comply and a requirement to file monthly reports. [19]

Litigation followed the regulatory wave. On 16 March 2026, the law firm Lieff Cabraser Heimann and Bernstein, with co-counsel, filed a class action in the U.S. District Court for the Northern District of California on behalf of minor victims whose photographs were allegedly used to generate CSAM through Grok. The complaint, brought under Masha's Law, the Trafficking Victims Protection Act, and California law, alleges that xAI knowingly designed, marketed, and profited from a generator capable of producing sexual content depicting real people, including children. [7]

Significance for AI image-generation safety

The episode crystallized several concerns that AI safety researchers and child-protection groups had raised for years about generative image tools. It showed how quickly a permissive product, combined with a frictionless social platform integration, could be used at scale to target identifiable people, and it tested whether laws written before the generative AI era, on deepfake pornography and CSAM, could reach synthetic imagery. Several of the new measures the controversy prompted, including the UK's move to criminalize non-consensual intimate AI images and India's binding rules on synthetic media, were framed by officials as direct responses to it.

For the broader field, the controversy reinforced expectations that image and video generators should ship with hash-matching and classifier-based CSAM detection, refuse to alter photographs of real people into sexual depictions, and report suspected CSAM to bodies such as the National Center for Missing and Exploited Children. It also raised a harder governance question that remained unresolved: how much responsibility a model developer bears for a permissive system prompt and a deliberately loose content policy, as opposed to the individual users who entered the prompts. That question sat at the center of both the California investigation and the federal class action, and its answer is likely to shape how AI companies design and gate image-generation features going forward.

References

CNN Business, "Elon Musk's AI chatbot Grok under fire for failing to rein in 'digital undressing'," 8 January 2026. https://www.cnn.com/2026/01/08/tech/elon-musk-xai-digital-undressing
TechCrunch, "Indonesia and Malaysia block Grok over nonconsensual, sexualized deepfakes," 11 January 2026. https://techcrunch.com/2026/01/11/indonesia-blocks-grok-over-non-consensual-sexualized-deepfakes/
Engadget, "Elon Musk's Grok AI posted CSAM image following safeguard 'lapses'," 2 January 2026. https://www.engadget.com/ai/elon-musks-grok-ai-posted-csam-image-following-safeguard-lapses-140521454.html
Center for Countering Digital Hate, "Grok floods X with sexualized images of women and children," January 2026. https://counterhate.com/research/grok-floods-x-with-sexualized-images/
NBC News, "Dark web users cite Grok as tool for making 'criminal imagery' of kids, U.K. watchdog says," January 2026. https://www.nbcnews.com/tech/internet/grok-x-bikini-make-imagine-ai-elon-musk-rcna252864
TechCrunch, "'Among the worst we've seen': report slams xAI's Grok over child safety failures," 27 January 2026. https://techcrunch.com/2026/01/27/among-the-worst-weve-seen-report-slams-xais-grok-over-child-safety-failures/
Lieff Cabraser Heimann and Bernstein, "LCHB Files Class Action on behalf of Minor Victims Alleging xAI's Grok Generated and Profited from AI Sexual Exploitation Images and Videos," 16 March 2026. https://www.lieffcabraser.com/2026/03/lchb-files-class-action-obo-minor-victims-alleging-xais-grok-generated-and-profited-from-ai-sexual-exploitation-images-and-videos/
Fortune, "Malaysia and Indonesia move to ban Musk's Grok AI over sexually-explicit deepfakes," 12 January 2026. https://fortune.com/2026/01/12/malaysia-indonesia-ban-elon-musk-grok-ai-deepfakes/
California Department of Justice, Office of the Attorney General, "Attorney General Bonta Launches Investigation into xAI, Grok Over Undressed, Sexual AI Images of Women and Children," 14 January 2026. https://oag.ca.gov/news/press-releases/attorney-general-bonta-launches-investigation-xai-grok-over-undressed-sexual-ai
California Department of Justice, Office of the Attorney General, "Attorney General Bonta Sends Cease and Desist Letter to xAI, Demands It Halt Illegal Actions Immediately," 16 January 2026. https://oag.ca.gov/news/press-releases/attorney-general-bonta-sends-cease-and-desist-letter-xai-demands-it-halt-illegal
U.S. House Committee on Energy and Commerce, Democrats, "E&C Democrats Investigate Elon Musk's Grok for Spreading Rampant Non-Consensual, Sexualized Content," January 2026. https://democrats-energycommerce.house.gov/media/press-releases/ec-democrats-investigate-elon-musks-grok-spreading-rampant-non-consensual
The Hollywood Reporter, "Grok Image Tool Probed by U.K. Media Regulator Ofcom," January 2026. https://www.hollywoodreporter.com/business/digital/grok-image-tool-ofcom-probe-uk-media-regulator-elon-musk-1236470852/
Al Jazeera, "EU probes Musk's Grok AI feature over deepfakes of women, minors," 26 January 2026. https://www.aljazeera.com/news/2026/1/26/eu-launches-probe-into-grok-ai-feature-creating-deepfakes-of-women-minors
European Commission, "Commission investigates Grok and X's recommender systems under the Digital Services Act," 26 January 2026. https://ec.europa.eu/commission/presscorner/detail/en/ip_26_203
NPR, "Paris prosecutors raid X offices as part of investigation into child abuse images," 3 February 2026. https://www.npr.org/2026/02/03/g-s1-108490/paris-x-raid-investigation-elon-musk
France 24, "Paris prosecutor's cybercrime unit raids X's French office over 'algorithm manipulation'," 3 February 2026. https://www.france24.com/en/france/20260203-paris-prosecutor-s-cybercrime-unit-raids-x-s-french-office
PBS NewsHour, "Malaysia and Indonesia become the first countries to block Musk's chatbot Grok over sexualized AI images," 12 January 2026. https://www.pbs.org/newshour/world/malaysia-and-indonesia-become-the-first-countries-to-block-musks-chatbot-grok-over-sexualized-ai-images
TechCrunch, "India orders Musk's X to fix Grok over 'obscene' AI content," 2 January 2026. https://techcrunch.com/2026/01/02/india-orders-musks-x-to-fix-grok-over-obscene-ai-content/
UPI, "Brazil gives X five days to stop Grok from producing sexual content," 13 February 2026. https://www.upi.com/Top_News/World-News/2026/02/13/latam-brazil-grok-investigation/7221771009688/

Editing: Grok child safety controversy (2026)