Online Safety Commissioner Niamh Hodnett has said she was "horrified" by the ability of the Grok AI tool to generate sexualised deepfake images of adults and children.
Ms Hodnett said X, which operates Grok, has taken some steps to make it safer.
"We're not saying it's there yet but we're assessing all of that," she said.
"We were horrified, like everyone else, to see AI-generated child sexual abuse material, and AI-generated, nonconsensual, intimate images being shared on those platforms," she said.
Ms Hodnett also defended her office's handling of the Grok situation amid criticisms that Ireland should have been more proactive because X has its European Headquarters here.
"We engaged extensively with the European Commission on it, with the Gardai, and with hotline.ie, who can help people get that type of content taken down," she said.
"We're supporting the European Commission's current investigation into this matter as a breach under the Digital Services Act, so that's an ongoing live matter," she added.
On banning social media access for under 16s, the Online Safety Commissioner said she is engaging with her Australian counterpart to see how the restrictions there are progressing.
"We're learning from that, and we're also engaged with our colleagues across Europe as well," Ms Hodnett said.
Last week, the European Commission accused TikTok of creating an "addictive design" in its app which could harm the physical and mental wellbeing of minors and vulnerable adults.
"We're supporting the European Commission in relation to that TikTok investigation, we're part of the team looking at it and we've shared complaints we've received with them," Ms Hodnett said.
Ms Hodnett's comments as Safer Internet Day is marked in Ireland and across the world today, with a focus this year on the risks and harms posed to young people by artificial intelligence.
The theme for the day is 'AI Aware: Safe, Smart and in Control' and aims to encourage parents, schools, educators, young people and all stakeholders to consider how AI shapes young people's lives.
Research from UCD showed that while young people do not trust AI, they reluctantly accept the privacy trade-offs, and while they know about its capacity for malicious use, they increasingly rely on it.
The key issues discussed during Safer Internet Day 2026 include the risks and harms emerging today, as well as ways of making generative AI technologies safe and age-appropriate for children.
An exploration of the potential opportunities of AI technologies to benefit children and young people is also being examined.
Experts from Ireland and Europe joined members of the Webwise Youth Advisory Panel at an event at the Department of Justice to discuss the impact of generative AI on young people.
"It is now more important than ever that we can empower our children and young people to develop the critical thinking and digital literacy skills needed to evaluate AI-generated content, protect their privacy and make responsible decisions," said Minister for Education and Youth Hildegarde Naughton.
The event was hosted by the Irish Safer Internet Centre - a partnership between Hotline, ISPCC, National Parents Council and Webwise.
Online safety charity CyberSafeKids said its latest research showed many children have no parental restrictions around online contact with strangers.
The study highlights concern around underage access to social media platforms and the sharing of images by children online.
It also showed that many parents feel ill-equipped and lack confidence when it comes to managing their children’s online activities.
"Our research shows many children are navigating complex online spaces largely alone, with little meaningful parental oversight," said Alex Cooney, CEO of CyberSafeKids.
"They are using platforms not designed for their age or safety, often sharing photos and videos without understanding who can see them or how far they can spread."
Without clear guidance, Ms Cooney said, "children may overshare and with the rise of nudification tools and AI girlfriend apps, their content can be altered and misused, putting them at serious risk".
Speaking on RTÉ's Morning Ireland, Ms Cooney said that there is not enough parental engagement with their children's online activity.
She said that it is particularly problematic overnight because many children have devices in their bedroom.
Research from the online safety charity found more than 90% of 3,000 eight- to 12-year-olds are active online.
Ms Cooney said that parents need to familiarise themselves with parental controls and the type of games and apps that their children are using.
Sinn Féin to bring legislation to amend Coco's Law
Sinn Féin will this evening bring forward proposed legislation that would amend Coco's Law to clarify that the creation of sexualised AI deepfake images is illegal.
Coco’s Law is a piece of legislation adopted by the Oireachtas in 2021, which criminalised the sharing and dissemination of intimate images without the subject’s permission.
It was named after Nicole 'Coco’ Fox, who died by suicide following years of abuse and harassment online.
Sinn Féin TD Matt Carthy said there was currently an anomaly in the legislation.
He said there was no "explicit criminalisation of the creation and generation" of intimate images, particularly through AI apps such as Grok, in the current law.
Speaking outside Leinster House, he said: "Our legislation will set out very clearly that the generation of such images will be a criminal act.
"There are also technical changes to the legislation in terms of extending the statute of limitations from two to five years and increasing the penalties applied to people in breach of the law."
Mr Carthy and his party colleague Máire Devine both said they were hopeful that the Government would support the Sinn Féin legislation.
Ms Devine added: "Many Government TDs and ministers have spoken out in the past few weeks about the need to ensure criminalisation of the generation of sexualised AI deepfakes.
"I therefore hope that this will translate into cross-party support tonight to pass Sinn Féin’s bill through to committee stages. Passing Sinn Féin's amendment to Coco's Law would demonstrate a true commitment to Zero Tolerance for violence against women and children in Ireland."
