The context you need, when you need it

When news breaks, you need to understand what actually matters — and what to do about it. At Vox, our mission to help you make sense of the world has never been more vital. But we can’t do it on our own.

We rely on readers like you to fund our journalism. Will you support our work and become a Vox Member today?

Could you spot an AI-written book?

An author set up an experiment to find out.

However you feel about AI writing, it has a few giveaways. According to the writer Imogen West-Knights, “there’s things like negative parallelisms…or excessive use of metaphor and similes, especially ones that don’t quite make sense or that come very rapidly, one after another. Every noun having an adjective attached, certain kinds of repetitive syntactical blocks that appear.”

So naturally, when an author uses AI to write their book, the publishing industry can easily spot it, right? As it turns out, not necessarily. AI models are built using human writing, the good and the bad, which is why it can be hard to tell whether something was written by a chatbot or by a person who loves a bad metaphor. The problem is all the more acute with smaller fragments of text, where there’s less room for AI’s telltale patterns and flatness to emerge.

Anthropic owes authors $1.5B for pirating work — but the claims process is a Kafkaesque mess

To find out just how good AI has gotten at imitating human writing, the writer and journalist Vauhini Vara decided to run an experiment on the people who know her writing the best. She thinks there is a misconception among writers and readers that “there’s a certain kind of way that AI generates language and it’s super different from the way writers do.” So could her friends distinguish between her work and an AI-generated imitation of her work? She told Today Explained co-host Noel King about what happened next.

Below is an excerpt of their conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

Nothing we love more at Today, Explained than a person running an experiment on herself! Vauhini Vara, writer, journalist, author of Searches, in paperback now, tell me everything.

There’s a researcher named Tuhin Chakrabarty whose work I’ve covered before, and he had already conducted this experiment. He and colleagues basically trained AI models on the work of established, accomplished writers.

What that means is he basically got the AI model to generate language that looked a lot like language from those authors. And then he had readers who were graduate writing students read those passages generated by AI and also read imitations by fellow graduate writing students and say which one they liked better. And........

© Vox