This AI Researcher Says the People Building AI Don’t Understand How It Works. Here’s What Every Founder Needs to Know

Roman Yampolskiy has studied AI safety longer than almost anyone. He says the biggest risk isn’t the technology itself. It’s the speed at which we’re deploying it without understanding what we’ve built.

EXPERT OPINION BY DANIEL ROBBINS, CEO AND FOUNDER OF IBH MEDIA AND HOST OF FOUNDER'S STORY

Illustration: Getty Images

Roman Yampolskiy keeps a folder on his computer called “Insane.” It has about a thousand emails from people who are, as he puts it, certifiably not well. Some tell him aliens instructed them to contact him to help liberate AI. Others just email that they’re coming Monday and he should be ready.

I share that because it gives you a sense of the world this man operates in. He’s one of the foremost AI safety researchers alive, and the more visible he becomes, the more the fringe finds him. But his actual message is anything but fringe. It’s grounded in mathematics, computer science, and a cold-eyed view of what happens when you build something smarter than yourself.

“People developing it don’t really understand how it works,” he told me. “And we’re pouring billions, even trillions of dollars into that arms race.”

When I asked if AI will wipe out humanity, his answer was characteristically precise.

How Anthropic's Claude AI Became a Co-Founder

“If you create something smarter than you with the ability to set its own goals, you can’t really predict what it’s going to do. That’s kind of by definition impossible if you’re not that smart.”

What concerned me most was his take on AI agents, which every founder is currently racing to implement.

“A tool is something a human will use to accomplish their goals,” he said. “An agent is an independent decision-making entity. It will decide what to do. It can decide to build a house or kill people. And that’s where the danger comes from.”

© Inc.com