The UK’s media regulator has opened an official investigation into X under the Internet Safety Act. “There have been serious reports of the Grok AI chatbot account on X being used to create and share images of naked people – which may amount to sexually explicit or pornographic images – and child pornography which may amount to child sexual abuse material (CSAM),” Ofcom said.
The investigation will focus on whether X “has complied with its duties to protect UK citizens from illegal content in the UK.” That includes whether X takes reasonable steps to prevent UK users from seeing “sensitive” illegal content, such as CSAM and intimate images; if the platform removes the illegal content immediately after becoming aware of it; and that X conduct an updated risk assessment before making “any significant changes” to the platform. The inquiry will also consider whether X has assessed the risk the site could pose to children in the UK and whether it “has the most effective age guarantee to protect UK children from viewing pornography.”
The regulator said it contacted X on January 5 and received a response by a deadline of January 9. Ofcom is conducting a “quick assessment of the available evidence as a matter of urgency” and added that it has asked xAI for an “urgent explanation” of the measures the company is taking to protect UK users.
“Reports that Grok has been used to create and share illegal non-consensual pornography and child sexual abuse material on X are deeply concerning,” an Ofcom spokesman said. “Forums must protect UK people from content that is illegal in the UK, and we will not hesitate to investigate when we suspect that companies are failing to do their job, especially when there is a risk of harm to children. We will continue this investigation as a matter of priority, while ensuring that we follow the correct process. As the UK’s independent investigation ensures that it is legally important and ensures that the investigation is legally important, we ensure that the Internet investigation is legally important. he decided.”
If Ofcom finds that a company has broken the law, it “can require platforms to take certain steps to comply or to remedy the damage caused by the breach.” The regulator can also impose fines of up to £18 million ($24.3 million) or 10 percent of “qualified” global revenue, whichever is higher. It may also seek a court order to stop payment providers or advertisers from operating the platform, or require internet service providers to block the site in the UK. The UK government has said it will support any action Ofcom takes against X.
Reports over the weekend suggested the UK was holding talks with partners about a coordinated response to deepfakes produced by Grok. Regulators elsewhere, including India and the European Union, are also investigating X.
Last week, Grok’s account on X began telling users that its photo production and editing tools were limited to paying subscribers. But as of Monday it was still possible for non-paying users to generate images through the Grok tab on the X website and app.
Meanwhile, Malaysia and Indonesia became the first countries to ban Grok, saying that X’s chatbot did not have enough safeguards to prevent AI-generated deepfakes of women and children from being created and distributed on X. Indonesia temporarily blocked access to Grok on Saturday, as did Malaysia on Sunday, The Associated Press reports.
“The government sees serious non-consensual sexual acts as violations of human rights, dignity and safety of citizens in the digital environment,” said Indonesia’s Minister of Communications and Digital Affairs Meutya Hafid in a statement. State officials said initial findings showed Grok did not have effective controls in place to prevent users from creating and sharing sexually explicit deepfakes based on images of Indonesian citizens. The country’s director general of digital space supervision, Alexander Sabar, said that producing deepfakes can violate a person’s image and privacy rights when photos are shared or used without permission, adding that they can lead to reputational, social and psychological damage.
The Malaysian Communications and Multimedia Commission cited Grok’s “repeated misuse” to produce blatant and unauthorized deepfakes, some of which involved women and children. The regulator said Grok will remain banned from the country until X Corp and parent xAI establish strong enough safeguards.
