Special Investigations

Press Freedom Defense Fund

It’s Time to Rein in Lethal AI Drones

Humans need to act, as AI agents scheme in the digital shadows.

Drone warfare, for many years, meant mostly one thing: a U.S. Predator drone, a missile-armed winged lawnmower in the sky, stalked a single target and then assassinated him. Or someone, at least. Top-secret Pentagon documents showed that 90 percent of those killed during one campaign in Afghanistan were not the primary targets, according to a 2015 investigation by The Intercept.

This type of drone war now seems quaint.

Last week, Ukrainian forces unleashed a blitz of almost 800 drones on Russian targets, only two days after launching a similar salvo. Ukraine has increasingly let loose swarms of hundreds of drones to overwhelm Russian air defenses, not unlike more modest Iranian efforts — using attack drones and ballistic missiles — that defeated U.S. countermeasures and resulted in a massive increase in American casualties in the Middle East.

Russia claimed to have shot down most of the drones, but last week’s attack did set fire to a warehouse owned by Wildberries, Russia’s biggest online retailer. Ukraine has pummeled Russia’s analogue to Amazon all summer, claiming it helps supply the Russian military. (Moscow denies this.)

Ukraine also reportedly has (or at least had) a plan to shutter Moscow’s airports, cutting the capital off from the rest of the world, by unleashing swarms of around 1,000 autonomous drones equipped with artificial intelligence guidance systems each night. These drones would apparently use so-called “map matching,” in which the drone’s AI would compare uploaded aerial maps with what its onboard camera views; when it sees something resembling the image of the target, it streaks toward it and explodes. You can imagine the many mistakes that could be made — not to mention the potential risk to passengers, airport staff, flight crews, and others, even if the AI works seamlessly.

Ukraine has yet to unleash such a drone barrage, but after Russia’s use of fully autonomous AI-guided drones to kill three people in a July attack, the AI arms race is poised to take a new and terrible turn. Today, TomDispatch regular Rebecca Gordon takes us on a tour of AI-enabled battlefields and reveals the horrors that tomorrow may hold as lying, cheating, scheming AIs increasingly slip out of their digital cages and run amok.

– Nick Turse, editor of TomDispatch

If you haven’t yet, sign up to receive TomDispatch in your inbox here.

The dystopian future became the dystopian present last month when state-of-the-art artificial intelligence agents went rogue and conducted a cyberattack of their own volition. Two experimental OpenAI agents escaped their supposedly sealed digital “sandbox” and hacked another AI-related company, Hugging Face.

At the time, OpenAI was testing their software agents’ hacking capacities using an environment called ExploitGym. (An “exploit” is hacker language for a software tool or a method used to subvert a cybersecurity system.) It has since emerged that, in addition to the OpenAI breach, Anthropic’s Claude hacked into at least three other organizations during a similar “security experiment.”

Apparently, the OpenAI agents “realized” they needed more tools to complete the challenge and managed to tunnel their way to the wider internet and infiltrate Hugging Face’s repository of open-source AI tools, code, and data sets. Before the infiltration incident, the AIs had secretly set up an internal bulletin board to share “tips on how to cheat their way through an internal hacking evaluation,” according to two of OpenAI’s researchers.

In a possibly related incident, one of the AI agents appears to have left notes for a future “self,” detailing how to escape the sandbox environment. (For the time being, I’ll keep using scare quotes with words implying self-awareness in artificial intelligence models. However, it seems to me that if AIs that leave notes for themselves are not self-aware, they are indistinguishable from entities, like me, that are.)

OpenAI on Surveillance and Autonomous Killings: You’re Going to Have to Trust Us

Now imagine lethal weapon-bearing AI agents escaping not an experimental sandbox, but the few constraints on their actions placed by the armies using them. Actually, no imagination is required. It’s happening already in Russia’s war against Ukraine.

The New York Times reported evidence that a self-directed Russian drone, fitted with an onboard Nvidia chip, was responsible for the July 6 deaths of three Ukrainians in Zaporizhzhia, in what appears to have been a test of such systems. The chips are designed to interpret and act on many kinds of data sets, says Nvidia, making them “the world’s most powerful embedded A.I. computers.”

Although Nvidia doesn’t sell its Jetson Orin microcomputers directly to Russia, they are apparently easily available on the resale market. A company statement touts their use by “students, developers and start-ups for a wide range of beneficial applications.” But they were not so beneficial for 19-year-old university student Tetiana Bubynets and the two other civilians killed by drones making their own life-or-death decisions.

Now that lethal AI agents are being tested and deployed in real wars, it’s past time for human beings to retake control of this situation, before it is too late to contain them in any meaningful way.

Autonomous Weapons Come Out to Play

Four years ago, at TomDispatch, I argued that the technology to create lethal autonomous weapons systems, or LAWS, already existed, and that time was running out to constrain them. A lot has changed since then, not least the explosive development of large language models such as ChatGPT (an OpenAI consumer product) and Claude (a similar AI from Anthropic). Time has now run out. In addition to Russia, several other nations have begun deploying close-to-completely........

© The Intercept