Why are AI chatbots so intelligent—capable of understanding complex ideas, crafting surprisingly good short stories, and intuitively grasping what users mean? The truth is, we don’t fully know. Large language models “think” in ways that don’t look very human. Their outputs are formed from billions of mathematical signals bouncing through layers of neural networks powered by computers of unprecedented power and speed, and most of that activity remains invisible or inscrutable to AI researchers.

This opacity presents obvious challenges, since the best way to control something is to understand how it works. Scientists had a firm grasp of nuclear physics before the first bomb or power plant was built. The same can’t be said for generative AI models. Researchers working in the AI safety subfield of “mechanistic interpretability” who spend their days studying the complex sequences of mathematical functions that lead to an LLM outputting its next word or pixel, are still playing catch-up. The good news is that they’re making real........

© Fast Company