Safeguarding concerns over the introduction by Snapchat of its own artificial intelligence chatbot have been raised by CyberSafeKids, which claims the service has not been properly tested.
The social media app recently launched the 'My AI' feature, which is pinned to the top of a user's chat feed, and only paid subscribers can remove it.
This has led to criticism, with confusion also emerging over how the app - which children as young as 13 can officially sign up to - uses location data.
CyberSafeKids Chief Executive Alex Cooney said that the chatbot, which is powered by OpenAI's ChatGPT, is concerning "especially given the userbase of Snapchat that they're rolling out this new technology without properly roadtesting it".
Speaking on RTÉ's This Week, Ms Cooney said that 42% of children who are between eight and 12-years-old surveyed by CyberSafeKids over the last year are using Snapchat, "so this is also going to be used by a lot of underage users as well".
We need your consent to load this rte-player contentWe use rte-player to manage extra content that can set cookies on your device and collect data about your activity. Please review their details and accept them to load the content.Manage Preferences
Snapchat says 'My AI' considers Snapchatters' ages and that the feature aims to keep conversations age-appropriate. It also says it will monitor how it is being used and what additional improvements are needed to make AI a more fun, useful and a safe experience for the community.
However, Alex Cooney said she thinks there is still "the potential for things to go wrong.
She notes that regulation is on the way for artificial intelligence in the form of the European Union's AI act, but said "child safety doesn't seem to be a strong feature of that legislation ... I'd be concerned that would need to be properly addressed in that legislation.
"We need safety by design as a central consideration in new technology being rolled out before it's released on the general public."
Regarding concerns over location, Snapchat says "My AI only has access to a Snapchatter’s location to provide recommendations if they’ve already shared it with friends on Snap Map or with Snapchat at the device level.
"We’re aware this has caused confusion with Snapchatters, but my AI does not collect any new location information."
In response, the CyberSafeKids Chief Executive said: "I've heard of lots of mixed reports about children asking (the AI chatbot) questions that are location-specific and sometimes getting the right answer and sometimes the response is 'I don't know where you are'. I'm not convinced, I think they do know the location."
Snapchat says parents can use 'Family Center' to see if their teens are chatting with My AI, and how often.
However, on concerns for parents of children using the app, Ms Cooney said "there's already so much pressure on parents in this regard.
"These tech companies, which are making so much money off the back of their users, should be taking far greater responsibility to ensure these enviornments, that they are creating and that children are using, are made safe for them, and that's not sufficiently the case."
