The social media network X has caused significant anger after people used its AI chatbot Grok to manipulate images of females by removing their attire.
Numerous instances have emerged where the tool has been used to remove clothing from female subjects to show them in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the organization behind Grok, offered no comment to questions, other than an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to prohibit image stripping tools. Under a proposed law, anyone who supplied such technology would "receive a prison sentence and substantial fines".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was presently examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some premium functions—which responds to X users' requests when they mention it in a message.
It is often used to offer commentary or more background to other posters' comments, but people on X are also able to alter an uploaded picture through its AI image editing feature.
Background of Criticism and Rules
The system has been criticised for enabling users to produce images and videos with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating inappropriate material of public figures.
"The platform has been enabling the production and distribution of these pictures for a long time without taking any action and we have not seen any challenge by regulators," the expert added.
XAI's own usage rules forbids "depicting images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and stated this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "appropriate measures" to "minimize the danger" of users encountering unlawful material on their services and to take down it swiftly when they become aware of it.
