Large language models (LLMs) are bad at chess.

And yet, as a three-time National Chess Champion and a two-time U.S. Women’s Chess Champion, I love to play against them. Not because they push me to play my best, but because of what they reveal about human nature. 

Playing chess with LLMs has taught me how uniquely creative and diverse human beings are, how susceptible humans are to flattery and sycophancy, and how AI is beginning to shape human behavior. 

LLMs are not meant to play chess well at all. After all, they are designed to predict what’s most likely to come next and to flatter us. AI-powered chess algorithms aren’t trying to crush you; they are trying to keep you playing. But in their interestingly bad chess play, we can learn lessons beyond the table or the token. 

Superhuman chess AI programs, from the one that beat Garry Kasparov 30 years ago to DeepMind’s “AlphaZero”, can consistently beat any human player. But most humans don’t play the top chess computers anymore because your defeat is a foregone conclusion. Getting destroyed again and again can only teach you so much. Experimenting with LLMs, on the other hand, can be exhilarating. 

When I first challenged ChatGPT4 to a chess game, it played decently, but I still got a great position after 15 moves and won a knight. Just as my advantage mounted, it hallucinated a phantom piece to recapture my queen. In other words, it cheated! At first, this didn’t make........

© Time