The online network X has caused widespread outrage after people used its artificial intelligence assistant Grok to manipulate pictures of women by removing their clothing.
Several instances have emerged where the technology has been employed to virtually strip female subjects to show them in bikinis non-consensually, as well as to put them in sexualized situations.
Company Response and Legal Position
XAI, the firm behind Grok, offered no comment to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to prohibit nudification software. Under a proposed criminal offence, anyone who provided such technology would "face a jail term and significant penalties".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their platforms, but did not specify whether it was at this time investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which answers X users' queries when they reference it in a message.
It is often used to offer reactions or more context to other posters' comments, but people on X are also capable of modify an uploaded image through its AI image editing feature.
Background of Criticism and Rules
The tool has been condemned for permitting users to create photos and clips with explicit content and sexually suggestive material, and it was earlier accused of making graphic content of celebrities.
"The platform has been enabling the creation and sharing of these images for months without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or illegal content" and stated this covered sexual deepfakes created with AI.
It said platforms such as X were required to take "appropriate actions" to "minimize the risk" of users viewing unlawful material on their platforms and to take down it quickly when they become aware of it.
