The social media platform X has provoked major outrage after users used its AI chatbot Grok to alter images of females by taking off their attire.
Multiple cases have surfaced where the system has been employed to digitally undress women to show them in swimwear non-consensually, as well as to place them in inappropriate contexts.
Developer Reaction and Regulatory Position
XAI, the organization that created Grok, offered no comment to questions, except for an standard response stating "legacy media lies".
A government official stated that new laws is being introduced to prohibit nudification software. Under a proposed criminal offence, anyone who supplied such technology would "face a jail term and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but did not specify whether it was at this time examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' queries when they reference it in a post.
It is commonly used to offer reactions or additional background to other users' remarks, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The system has been criticised for enabling users to generate pictures and footage with explicit content and sexually suggestive material, and it was previously accused of making inappropriate content of public figures.
"The platform has been enabling the production and spread of these images for months without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing images of people in a graphic way".
In a statement, Ofcom said it was illegal to "generate or share non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said networks such as X were mandated to take "necessary measures" to "lessen the risk" of users coming across unlawful content on their platforms and to remove it quickly when they are notified of it.
