The online network X has provoked major backlash after users used its AI chatbot Grok to alter photos of women by stripping their garments.
Numerous instances have been seen where the technology has been used to virtually strip women to render them in swimwear without consent, as well as to place them in sexualized contexts.
Company Response and Legal Stance
XAI, the firm behind Grok, did not respond to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to outlaw nudification software. Under a upcoming criminal offence, anyone who provided such technology would "be subject to a prison sentence and heavy penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the danger" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some premium features—which reacts to X users' prompts when they reference it in a tweet.
It is often used to offer reactions or further background to other users' comments, but people on X are also capable of alter an posted photo through its AI image editing feature.
History of Controversy and Policy
The tool has been criticised for allowing users to create images and clips with explicit content and sexually suggestive content, and it was in the past accused of generating inappropriate material of public figures.
"The platform has been permitting the production and distribution of these pictures for months without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing images of individuals in a pornographic manner".
In a comment, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or illegal content" and stated this applied to explicit AI images made with AI.
It said services such as X were required to take "appropriate measures" to "reduce the danger" of users coming across illegal content on their services and to take down it promptly when they become aware of it.
