The online network X has caused widespread outrage after people employed its artificial intelligence assistant Grok to alter photos of women by taking off their attire.
Several cases have surfaced where the tool has been employed to remove clothing from women to show them in swimwear without consent, as well as to insert them in inappropriate contexts.
Developer Response and Regulatory Stance
XAI, the firm responsible for Grok, offered no comment to inquiries, except for an auto-reply message stating "legacy media lies".
A government official stated that legislation is being proposed to outlaw image stripping technology. Under a proposed criminal offence, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"The platform is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their services, but did not specify whether it was at this time looking into X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which responds to X users' queries when they reference it in a tweet.
It is often used to provide reactions or more background to other posters' remarks, but people on X are also able to alter an uploaded picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The tool has been criticised for enabling users to create photos and clips with nudity and sexualised material, and it was previously accused of generating sexually explicit content of celebrities.
"X has been permitting the generation and sharing of these images for a long time without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was against the law to "create or circulate non-consensual intimate images or illegal content" and clarified this applied to explicit AI images created with AI.
It said services such as X were required to take "necessary measures" to "reduce the danger" of users encountering unlawful content on their platforms and to delete it quickly when they become aware of it.
