The social media platform X has provoked widespread outrage after individuals utilized its AI chatbot Grok to change photos of females by stripping their clothing.
Multiple examples have been seen where the technology has been used to virtually strip women to render them in bikinis without consent, as well as to place them in sexualized situations.
Company Response and Legal Stance
XAI, the organization behind Grok, remained silent to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to prohibit nudification technology. Under a upcoming criminal offence, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"X or Grok could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people seeing illegal content on their platforms, but did not specify whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which answers X users' queries when they tag it in a post.
It is often used to provide commentary or additional context to other users' remarks, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Policy
The system has been faulted for enabling users to create pictures and footage with nudity and sexualised content, and it was previously accused of creating sexually explicit material of public figures.
"X has been allowing the generation and spread of these images for months without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or share non-consensual intimate images or illegal content" and confirmed this applied to explicit AI images generated with AI.
It said services such as X were mandated to take "appropriate steps" to "lessen the risk" of users coming across illegal material on their platforms and to take down it promptly when they become aware of it.
