With Jacob Coxon’s resignation from Anthropic, we have reached the AI risk tipping point. Millions of people are finally coming to understand what experts have known for years: AI companies have been gambling with all of our lives, and the odds are not good.

In response, the Anthropic CEO, Dario Amodei, has introduced a proposal for “pacing the frontier”, endorsed by Sam Altman and Elon Musk. Can we breathe a sigh of relief? Are we about to step back from the edge of extinction? Amodei envisions a slowdown of one to two years, resulting in “profound progress” on technical safety measures. But that’s not what humanity needs right now. What we need is a plan in which we’re confident AI isn’t going to kill us, and this isn’t it.

The stark reality is this: we don’t know how these AI systems work. We don’t know how to prevent them from misbehaving. We don’t know how to stay in control once they’re smarter than us. We don’t know how to make sure they’re not playing nice or playing dumb to trick our safety tests. We can’t count on solving these problems with another one or two years of research.

I know because I have been researching such problems for more than a decade, including as an AI professor at both the University of Cambridge and the University of Montreal. If we want to reduce the risks from AI to an acceptable level, what we actually need is an........

© The Guardian