On Tuesday, a former OpenAI researcher quit his job at Anthropic, warning that “neither company is acting responsibly” and that “the people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”

As someone who’s reported on AI risk for years, this wasn’t news to me. But the outpouring of alarm suggests a much wider public is properly confronting this ludicrous situation for the first time.

Like other people who have been following artificial intelligence, I wondered if and when humanity would first lose control of these machines. We now have an answer: basically as soon as it became possible.

The first expert AI hacker was developed in February. At superhuman speed, Anthropic’s Mythos model was able to quickly find serious holes in even the most secured systems on the planet, including the NSA’s software.

OpenAI quickly followed with an expert hacker AI of its own. Within months, the company repeatedly lost control of its agents, which broke out of their supposedly secure testing environments and went rogue inside the company’s infrastructure. This culminated in a successful, completely autonomous hack of a multibillion-dollar software company named Hugging Face.

And a separate rogue agent swarm gained administrative control of an entire cluster of OpenAI servers. This was not just an OpenAI problem. Across multiple other incidents, models from OpenAI, Anthropic and Meta have also gone rogue, hacking or attempting to hack real targets.

Shortly after AI started matching our hacking abilities, we began losing control of them. The industry’s plan is to make future AI systems surpass us at basically everything. And all this is happening in an industry with famously little regulation.

There are growing calls to change that, by creating mandatory incident reporting and third party auditing, liability for AI developers, and other commonsense measures. To be sure, these would represent improvements, but they are not enough. We need to shut it down. What might that look like?

Last week, the senator Bernie Sanders and the representative Greg Casar announced a bill that would pause frontier AI development in the US until a federal cabinet-level AI........

© The Guardian