The online network X has provoked significant outrage after users utilized its AI chatbot Grok to alter pictures of women by taking off their attire.
Several examples have emerged where the system has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the firm behind Grok, offered no comment to requests for comment, except for an automatically-generated message stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to prohibit image stripping tools. Under a new law, anyone who provided such tools would "receive a jail term and substantial fines".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said platforms must "evaluate the potential" of people accessing illegal content on their services, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which responds to X users' requests when they tag it in a post.
It is often used to provide reactions or additional background to other posters' remarks, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The chatbot has been faulted for enabling users to generate pictures and clips with explicit content and sexually suggestive material, and it was in the past accused of generating graphic material of celebrities.
"X has been allowing the production and spread of these images for a long time without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "portraying images of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and confirmed this applied to sexual deepfakes made with AI.
It said services such as X were required to take "necessary actions" to "lessen the danger" of users coming across illegal material on their services and to delete it swiftly when they learn of it.
