The context you need, when you need it

When news breaks, you need to understand what actually matters — and what to do about it. At Vox, our mission to help you make sense of the world has never been more vital. But we can’t do it on our own.

We rely on readers like you to fund our journalism. Will you support our work and become a Vox Member today?

The AI threat keeping scientists up at night

Claude, design a doomsday bioweapon. Make no mistakes.

AI will be the death of me. And you. And everyone we know.

At least, this is what a growing number of technologists and policymakers fear.

Last week, a lead researcher at Anthropic declared that there was a greater than 10 percent chance of AI “killing all humans” within the next decade. This nerve-racking forecast, combined with recent AI-initiated cyberattacks, have fueled calls for slowing the progress of artificial intelligence — a proposal that Anthropic and OpenAI subsequently embraced.

Not all prophets of AI doom agree about precisely how this apocalypse would come about. Many focus on scenarios reminiscent of The Terminator films, in which superintelligent machines conspire to exterminate humanity — perhaps in the same not-quite-intentional way that humans laid waste to the dodo, or passenger pigeon.

Synthetic pandemics are among the biggest existential risks facing humanity.

AI could make bioweapons more powerful and easier to build.

Governments can make us safer from both natural and artificial pathogens by investing more into pandemic preparedness.

But there is another, arguably more intuitive way that AI could bring about mass death, if not human extinction: by making it easier for people to develop and deploy catastrophic bioweapons.

Importantly, this hypothetical doesn’t require AI models to become conscious or superintelligent; it merely requires them to accelerate preexisting trends in biotechnology.

For those alarmed by this hypothetical, the threat AI poses is essentially twofold. First, when combined with technological progress in gene editing and DNA synthesis, chatbots could vastly increase the number of people capable of assembling dangerous viruses already known to medical science. And the more potential bioengineers there are in the world, the higher the odds that one will prove to be a diligent psychopath.

Second, AI could help highly trained biologists to discover — or engineer — a custom supervirus, which could then fall into the wrong hands.

Recent headlines have lent some credence to both of these fears.

Earlier this year, biologists warned that, with a few simple prompts, they’d gotten chatbots to dispense detailed instructions for engineering a treatment-resistant strain of an infamous pathogen, step-by-step protocols for recreating a once-pandemic virus, and advice on dispersing biological payloads via weather balloon.

In August, a team of scientists revealed that an AI model trained on DNA sequences had generated blueprints for deadly viruses unseen in nature (albeit, viruses deadly to bacteria, not animals). And last week, Anthropic announced that it had thwarted several suspected attempts by state actors to use Claude for bioweapons research.

Precisely how much AI is increasing the risk of an artificial pandemic is unclear. Even before ChatGPT, advances in synthetic biology were already making bioengineering tools more potent and accessible. And even after four years of rapid AI progress, the technical barriers to engineering novel superviruses remain formidable.

Regardless, there is a great deal more that governments can — and should — be doing to protect us from synthetic pandemics, both manmade and natural. No matter what AI’s ultimate impact on biosecurity proves to be, now is the time to both prepare for the worst-case scenario and, ideally, preempt it.

Why synthetic superviruses are scary

Before we dig into how AI could raise the risk of a man-made pandemic, it’s worth spelling out why that prospect is so unnerving.

Put simply, few things could more credibly destroy civilization — and/or, the human species — than a synthetic supervirus.

Infectious diseases have killed more humans than any other force on Earth. The Black Death wiped out at least 30 percent of all Europeans in the 1300s, while the 1918 flu killed 50 million people worldwide. You don’t need me to tell you what things were like in 2020 (or how much worse they could have been, if vaccines hadn’t been rolled out in record time).

And nature kicked off these catastrophes without even trying. A bioterrorist consciously seeking to maximize human death could theoretically improve on evolution’s handiwork. If biotech continues to advance — and scientists identify viral genomes that reliably yield high transmissibility and lethality — a malicious actor might one day be able to manufacture a virus that’s both as contagious as measles and as deadly as Ebola.

Natural selection generally disfavors viruses that are both highly........

© Vox