Type a question into an AI chatbot. A few seconds later, voila! The response unspools in front of your eyes. But exactly why the bot came up with that specific answer is still something of a mystery— even to those who built it. Not understanding how AI really works has inadvertently led to models behaving in unexpected ways: obsessing over goblins and other magical creatures, showering people with fake praise and most concerningly, deceiving and blackmailing the very people who built them.

AI’s core enigma has given rise to a smattering of so-called "interpretability" startups that aim to investigate what’s happening under AI’s hood. That could help improve models’ capabilities, making them safer and training them to avoid spewing out wrong answers or acting in nefarious ways. That’s the premise behind Goodfire, a $1.25 billion-valued AI research lab that studies the inner workings of models. Its tools promise to help developers and researchers inspect models and control how they behave.

“It’s like these alien intelligences have crash landed on earth and they're........

© Forbes