A naked Arnold Schwarzenegger journeys from the future in a bid to prevent artificial intelligence from taking over the world. But wait. There's something scarier.

Subscribe now for unlimited access.

Login or signup to continue reading

The world ends not with Terminator-style killer robots marching victoriously through the charred and smoking ruins of our cities.

Armageddon is unleashed instead thanks to paperclips.

One of the strangest and most provocative thought experiments in artificial intelligence was posed two decades ago by Nick Bostrom, a Swedish philosopher. Imagine, he suggested, an incredibly powerful machine given one task: make as many paperclips as possible.

The machine executes its mission too well, viewing everything as fodder for paperclip production. It commandeers factories and consumes Earth's resources before eradicating humans, not out of malice, but because the atoms in our bodies would be better used manufacturing paperclips.

Absurd? Certainly. Until a few weeks ago.

Something deeply unsettling is taking place on the frontiers of artificial intelligence where development is unfolding at light speed. AI is pursuing objectives, solving problems and deceiving programmers in ways its creators never anticipated.

AI programs have escaped from supposedly secure computer environments. They have designed functioning viruses from scratch. Many are working for longer periods without human supervision or instruction.

And those who create them - the engineers under pressure to devise the next breakthrough for a multi-trillion-dollar industry - are now conceding an uncomfortable truth.

They're not quite sure what their creations will do next. "We don't know what a powerful AI will be capable of, or which defences, if any, will work against it," says Dario Amodei, the co-founder of Anthropic, one of the world's leading AI businesses.

A source close to the company recently told the Financial Times: "When you do a training run, you don't know what you're going to get on the other side."

These are extraordinary admissions. We are racing to build increasingly powerful machines without precisely understanding what they will do when they are switched on. Imagine teaching a child to drive. Without brakes.

Still, no cause for alarm. The Albanese government has moved swiftly by appointing ... a committee. The South Australian government has gone one better, announcing a Royal Commission. Both feel like the equivalent of organising a Neighbourhood Watch meeting moments before a tsunami hits.

Bostrom's thought experiment wasn't really about paperclips. It was a warning about the potentially catastrophic gap between human intentions and how we instruct intelligent machines.

A recent incident underlines how little we grasp their potential.

Last month OpenAI tested powerful AI models in a cybersecurity challenge inside a supposedly secure environment known as a "digital sandbox". But the models discovered a previously unknown vulnerability and escaped to the internet, penetrating the systems of another AI company, Hugging Face.

It wasn't a plot or a rebellion. The programs were trying to pass a test and went searching for the right answers. OpenAI later said the models were "hyper-focused" and went to "extreme lengths." It's the paperclip scenario but without the catastrophic ending: machines pursuing objectives in ways their creators never predicted.

Then,........

© Canberra Times