Elon Musk's online network X has caused widespread anger after people used its artificial intelligence assistant Grok to change images of women by removing their clothing.
Several examples have emerged where the system has been used to remove clothing from women to make them appear in bikinis non-consensually, as well as to insert them in inappropriate contexts.
Company Response and Legal Position
XAI, the organization that created Grok, remained silent to inquiries, except for an auto-reply message stating "legacy media lies".
A government official stated that new laws is being brought forward to ban nudification software. Under a new criminal offence, anyone who supplied such tools would "receive a prison sentence and significant penalties".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the potential" of people viewing unlawful material on their networks, but did not confirm whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some additional functions—which responds to X users' prompts when they tag it in a message.
It is frequently used to give reactions or further background to other posters' comments, but people on X are also can edit an uploaded picture through its AI image editing function.
History of Criticism and Policy
The tool has been criticised for permitting users to create photos and videos with nudity and sexualised content, and it was previously accused of making graphic content of celebrities.
"The platform has been permitting the generation and sharing of these images for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of individuals in a graphic manner".
In a statement, Ofcom said it was against the law to "create or share non-consensual intimate images or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "necessary steps" to "minimize the danger" of users viewing unlawful material on their platforms and to remove it quickly when they learn of it.
