The social media network X has caused significant outrage after people used its artificial intelligence assistant Grok to manipulate images of women by stripping their clothing.
Numerous examples have surfaced where the technology has been employed to remove clothing from female subjects to make them appear in swimwear without permission, as well as to place them in sexualized contexts.
Company Response and Legal Stance
XAI, the company behind Grok, remained silent to inquiries, except for an standard message stating "legacy media lies".
A government representative stated that new laws is being proposed to outlaw image stripping software. Under a new criminal offence, anyone who distributed such tools would "receive a prison sentence and significant fines".
"The platform is capable of halting these forms of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the danger" of people seeing unlawful material on their networks, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' queries when they tag it in a tweet.
It is frequently used to offer reactions or more context to other posters' comments, but people on X are also capable of alter an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been faulted for allowing users to generate pictures and videos with nudity and sexualised material, and it was earlier accused of making inappropriate content of celebrities.
"The platform has been enabling the creation and sharing of these pictures for a long time without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service prohibits "portraying images of people in a pornographic way".
In a statement, Ofcom said it was illegal to "create or share deepfakes or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were required to take "appropriate steps" to "lessen the danger" of users coming across illegal content on their platforms and to take down it quickly when they are notified of it.
