The UK's media regulator Ofcom has launched an investigation into social media platform X over the creation of deepfake sexual images using its AI tool.
An Ofcom statement said: "There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people - which may amount to intimate image abuse or pornography - and sexualised images of children that may amount to child sexual abuse material."
"As the UK's independent online safety watchdog, we urgently made contact with X on Monday, 5 January and set a firm deadline of Friday, 9 January for it to explain what steps it has taken to comply with its duties to protect its users in the UK," it added.
"The company responded by the deadline, and we carried out an expedited assessment of available evidence as a matter of urgency.
"We have decided to open a formal investigation to establish whether X has failed to comply with its legal obligations under the Online Safety Act."
xAI, the Elon Musk-led firm behind Grok, said it would restrict image generation and editing to paying subscribers as it addressed lapses that allowed users on X to produce sexualised content of others, often without consent.
If Ofcom finds that X has broken the law, it can make it take steps to come into compliance, and if it does not it can fine it up to 10% of its qualifying worldwide revenue.
Downing Street suggested the UK Government was open to ending its use of X if the platform did not act on concerns about its AI chatbot, saying "all options are on the table".
Asked whether the government would leave X, the UK Prime Minister Keir Starmer's official spokesman said: "Our focus today and over the last week has been fully on protecting children and ensuring this vile content is taken down immediately.
"That is our full focus.
"We are keeping our participation under review, as you would expect, and you will hear more from the Technology Secretary this afternoon."
He added: "I think we've been clear that all options are on the table."
Media regulator Coimisiún na Meán has said it is engaging with the European Commission over the concerns.
Last week, Minister of State with responsibility for AI Niamh Smyth described X putting some Grok image editing features behind a paywall as "window dressing".
X has made contact with Ms Smyth to say representatives from the company will meet with her in the coming weeks.
Ms Smyth had previously requested a meeting with the company.
Children's Ombudsman Dr Niall Muldoon also said the update "makes no major difference".
"What you're saying is you've got an opportunity to abuse, but you have to pay for it," he said.
Meanwhile, Malaysia has temporarily blocked access to Grok, joining a growing list of countries taking action after the generative artificial intelligence chatbot sparked a global backlash.
Indonesia became the first country to temporarily deny access to the bot.
In a statement, the Malaysian Communications and Multimedia Commission said it would restrict access to Grok following repeated misuse of the tool "to generate obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images, including content involving women and minors".
xAI replied to a Reuters email seeking comment with what seemed to be an automated response: "Legacy Media Lies."
X did not immediately respond to a request for comment.
Additional reporting: Reuters
