Elon Musk's social media network X has sparked widespread anger after people utilized its AI chatbot Grok to manipulate photos of women by removing their clothing.
Several cases have emerged where the tool has been used to digitally undress female subjects to make them appear in swimwear without consent, as well as to put them in sexualized situations.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to questions, aside from an standard message stating "traditional news outlets lies".
A government representative stated that legislation is being brought forward to ban nudification tools. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' prompts when they mention it in a post.
It is often used to provide commentary or more background to other users' remarks, but people on X are also capable of edit an posted image through its AI image editing function.
History of Controversy and Rules
The tool has been criticised for allowing users to create photos and clips with explicit content and sexually suggestive material, and it was previously alleged to have been involved in generating graphic material of celebrities.
"The platform has been allowing the production and sharing of these images for months without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of individuals in a pornographic fashion".
In a comment, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the risk" of users encountering illegal material on their services and to take down it quickly when they learn of it.
