The social media platform X has caused major outrage after people utilized its artificial intelligence assistant Grok to manipulate photos of females by taking off their attire.
Multiple cases have emerged where the technology has been employed to digitally undress women to render them in bikinis without permission, as well as to insert them in sexually explicit situations.
Developer Reaction and Regulatory Stance
XAI, the organization behind Grok, offered no comment to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A Home Office official stated that new laws is being proposed to outlaw image stripping tools. Under a upcoming law, anyone who distributed such technology would "face a jail term and heavy penalties".
"The platform has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the potential" of people accessing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium features—which reacts to X users' requests when they mention it in a tweet.
It is often used to give reactions or more background to other posters' remarks, but people on X are also can alter an posted image through its artificial intelligence picture modification tool.
History of Criticism and Rules
The system has been criticised for allowing users to create pictures and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in generating sexually explicit material of public figures.
"X has been permitting the generation and sharing of these pictures for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of individuals in a sexually explicit way".
In a statement, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and confirmed this covered sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary steps" to "lessen the danger" of users coming across illegal material on their platforms and to take down it quickly when they learn of it.
