An amazing, literally jaw-dropping, story appeared yesterday: the artificial intelligence company Anthropic has warned investors that advanced AI could pose "catastrophic or existential risks to humanity".

Subscribe now for unlimited access.

Login or signup to continue reading

In plain English: AI could wipe out all human life on Earth.

We may worry about how AI changes education or the writing of novels or student essays or how it destroys jobs - but there won't be no jobs or students or readers of novels if the "existential risk to humanity" turns out to be real.

AI models, the company itself said, could exhibit "self-preserving behaviours", including attempts to "resist shutdown", to "conceal or manipulate information" and exhibit behaviour "resembling blackmail".

And Anthropic should know. It is an American behemoth which plans to make huge amounts of money by developing AI. If it is warning of the dangers, we should take note. Finally.

We on this planet are in a race to build the doomsday machine, and without any apparent prospect of the governments which matter - in Beijing and Washington - doing enough to head off the "existential" dangers, perhaps because the two countries' leaders imagine they can get the machine first to use it to whack the other.

Very clever people have been warning for some time of the existential dangers of AI.

In 2023, hundreds of artificial intelligence experts signed the short Statement on AI Extinction Risk which said: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

What could be clearer?

The problem with AI is that it sees gaps in systems which humans can't imagine. Humans might give it tasks to benefit humanity but the machine moves in unpredictable, perhaps destructive ways.

"A super-intelligent AI would likely be extraordinarily competent at achieving its goals. But it might be indifferent to human survival," the University of NSW professor of AI, Toby Walsh, wrote.

He outlined a scenario: "A classic example of such indifference comes from Oxford philosopher Nick Bostrom's imagined super-intelligent AI that's been designed to optimise paperclip production. To produce its preferred form of office........

© Canberra Times