Why the most dangerous AI may be the one we trust too much
Machines don’t have to become superintelligent to cause catastrophic harm. As artificial intelligence spreads through military decision-making, they just have to be wrong – and trusted by the humans in charge.
In 1991, Terminator 2: Judgement Day gave us one of popular culture’s most enduring images of AI gone wrong. Humanity builds Skynet, an artificial intelligence given control of strategic defence. It learns at a “geometric rate”, becomes self-aware and, when its human operators try to shut it down, it starts a nuclear war.
Versions of that story now feature in serious debate about AI. Increasingly capable large language models (LLMs), the argument goes, could develop into artificial general or super intelligence (AGI/ASI), acquire goals that diverge from ours and ultimately threaten humanity.
That debate deserves attention, but it depends on a chain of large and unproven steps. Today’s AI systems must somehow transform into genuinely general intelligences, develop independent goals and gain control over the physical world.
Meanwhile, a more immediate risk is taking shape that requires none of those steps.
In the film, Skynet builds the Terminators, machines that find, track and kill humans. Strip away the science fiction and the idea is simple: combine sensors, software and weapons and reduce the human role.
Human beings are doing exactly that, without anything resembling a malicious superintelligence like Skynet.........
