Elon Musk's online platform X has provoked widespread outrage after people utilized its AI chatbot Grok to manipulate images of females by stripping their clothing.
Multiple instances have surfaced where the tool has been used to remove clothing from women to show them in swimwear non-consensually, as well as to place them in inappropriate situations.
Developer Reaction and Legal Position
XAI, the organization responsible for Grok, did not respond to inquiries, other than an auto-reply message stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to outlaw image stripping software. Under a new law, anyone who provided such technology would "receive a prison sentence and significant fines".
"X or Grok is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' prompts when they reference it in a message.
It is often used to offer commentary or additional background to other users' comments, but people on X are also can edit an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The system has been criticised for enabling users to produce images and videos with nudity and sexualised content, and it was in the past accused of creating graphic content of celebrities.
"X has been enabling the creation and distribution of these images for months without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of people in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to explicit AI images made with AI.
It said networks such as X were mandated to take "necessary steps" to "minimize the risk" of users coming across unlawful content on their services and to remove it promptly when they become aware of it.
