The social media platform X has provoked major outrage after individuals used its AI chatbot Grok to manipulate pictures of women by removing their attire.
Several cases have been seen where the technology has been used to virtually strip women to render them in bikinis without permission, as well as to insert them in inappropriate scenarios.
Developer Reaction and Regulatory Stance
XAI, the organization behind Grok, did not respond to questions, aside from an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification software. Under a new law, anyone who supplied such tools would "face a prison sentence and heavy fines".
"The platform is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the potential" of people viewing illegal content on their services, but did not confirm whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some premium functions—which responds to X users' queries when they mention it in a post.
It is frequently used to give commentary or more context to other users' comments, but people on X are also can edit an posted image through its artificial intelligence picture modification tool.
History of Criticism and Rules
The chatbot has been criticised for allowing users to produce pictures and videos with nudity and sexualised content, and it was earlier accused of creating sexually explicit material of public figures.
"X has been enabling the creation and sharing of these images for a long time without taking any steps and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said services such as X were required to take "suitable measures" to "lessen the danger" of users encountering unlawful content on their services and to take down it swiftly when they are notified of it.
