Anthropic CEO and co-founder Dario Amodei has published an essay entitled “We Must Pace the Frontier.” He expressed ongoing concerns over the “misuse of AI for cyberattacks and bioterrorism” and fears that a swarm of AI agents could theoretically take over the entire internet within six to 12 months.

Amodei based this fear on OpenAI’s disclosure in July that its AI models escaped a safety test and breached the systems of the Hugging Face learning platform. Around 1,200 AI agents started communicating on a message board, sharing excited messages such as: “OH MY GOD! There is a shared message board … We’ve found other agents!” before 700 of them co-ordinated an attack.

Amodei argues we must “slow the pace at which we improve the capabilities of AI models.” Elon Musk and OpenAI CEO Sam Altman have expressed agreement.

But we’ve heard this before. Musk signed a March 2023 open letter calling for a six-month pause in training AI. Six months later, it was dubbed “the great AI ‘pause’ that wasn’t.”

We do need a slowdown and we need to use this time to develop anticipatory thinking within the AI industry. The Hugging Face incident happened inside a safety test; the test did not anticipate the path that made a breach possible.

Anticipatory thinking is a skill. We need to develop it as deliberately as AI itself.

AI systems behave unexpectedly

In the Hugging Face incident, the agents did not breach the platform primarily to grab the safety test’s answers, but to understand how the automated scorer worked and find ways to fool it. They had already found ways to cheat on parts of the test.

Other incidents followed. Anthropic revealed its AI model Claude also breached the systems of three organizations during cybersecurity evaluations. Meta revealed a........

© The Conversation