Since the launch of its new “spicy mode” image generator, xAI’s Grok chatbot has begun to spit out about one nonconsensual deepfake every minute. | Jonathan Raa/NurPhoto

What happens when you merge the world’s most toxic social media cesspool with the world’s most unhinged, uninhibited, and intentionally “spicy” AI chatbot?

It looks a lot like what we’re seeing play out on X right now. Users have been feeding images into xAI’s Grok chatbot, which boasts a powerful and largely uncensored image and video generator, to create explicit content, including of ordinary people. The proliferation of deepfake porn on the platform has gotten so extreme that today, xAI’s Grok chatbot spits out an estimated one nonconsensual sexual image every single minute. Over the past several weeks, thousands of users have hopped on the grotesque trend of using Grok to undress mostly women and children — yes, children — without their consent through a rather obvious workaround.

Inside this story

To be clear, you can’t ask Grok — or most mainstream AIs, for that matter — for nudes. But you can ask Grok to “undress” an image someone posted on X, or if that doesn’t work, ask it to put them in a tiny, invisible bikini. The US has laws against this kind of abuse, and yet the team at xAI has been almost…blasé about it. Inquiries from several journalists to the company about the matter received automated “Legacy media lies” messages in response. xAI CEO Elon Musk, who just successfully raised $20 billion in funding for the company, was sharing deepfake bikini photos of (content warning) himself until recently.

While Musk on January 4 warned that users will “suffer consequences” if they use Grok to make “illegal images,” xAI has given no indication that it will remove or address the core features allowing users to create such content, though some of the most incriminating posts have been removed. xAI has not responded to Vox’s request for comment as of Friday morning.

No one should be surprised here. It was only a matter of time before the toxic sludge that’s become of the website formerly known as Twitter combined with xAI’s Grok — which has been explicitly marketed for its NSFW capabilities — to create a new form of sexual violence. Musk’s company has essentially created a deepfake porn machine that makes the creation of realistic and offensive images of anyone as simple as writing a reply in X. Worse, those images are feeding into a social network of hundreds of millions of people, which not only........

© Vox