The context you need, when you need it

When news breaks, you need to understand what actually matters — and what to do about it. At Vox, our mission to help you make sense of the world has never been more vital. But we can’t do it on our own.

We rely on readers like you to fund our journalism. Will you support our work and become a Vox Member today?

Can machines be war criminals?

A law professor explains how AI-assisted weapons subvert our justice system.

On the first day of its war against Iran, the United States killed 123 children when it targeted and struck an elementary school.

The strikes, ultimately approved by humans, were facilitated by the US’s Maven Smart System, an AI-enabled software platform developed by Palantir Technologies that allows the military to aggregate, organize, and process battlefield data. Operators fed Maven intelligence classifying the elementary school as an Iranian military facility, and the system, in turn, marked it as one of the priority targets for day one of the war. Though the site was formerly part of a naval military base, it had been converted into an elementary school in 2017 and had remained one since. The intel handed to Maven did not reflect this.

This mistake resulted in what some reporters have called “one of the single worst massacres committed by the US government since the Vietnam War.”

Explore the big, complicated problems the world faces and the most efficient ways to solve them. Sent twice a week.

Such scenarios may become more common as militaries continue to deploy a once-theoretical technology — AI-assisted or autonomous weapons, capable of making both targeting and launch decisions on their own — on real battlefields around the globe. And they raise a troubling, unresolved question: When human beings and machines collaborate in violating international law, who is to blame?

In this case, the government and private sector are pointing the finger at each other. Some US Central Command personnel, according to Pentagon officials who spoke to Bloomberg, expected Maven to flag stale information or inconsistencies in the underlying intelligence assembled for potential targets. Meanwhile, a Palantir spokesperson told Bloomberg that the company “is not responsible for the underlying data nor identifying intelligence deficiencies,” while also denying that the evidence indicated that its software was at fault in the strike.

This debate underscores the fact that AI and autonomous weapons are raising a new host of unresolved issues within the law, even as they’re actively assuming positions on the battlefield. In institutions where the line between human and machine decision-making continues to blur, how do we assign responsibility for war crimes? And, are the laws in place suited for the job?

In institutions where the line between human and machine decision-making continues to blur, how do we assign responsibility for war crimes?

To unpack some of these issues, I spoke with international law expert Allen Weiner, a senior lecturer at Stanford Law School, where he teaches courses on the law of armed conflict. Before teaching at Stanford, Weiner practiced international law in the US Department of State for more than a decade. We discussed the difficulties in attributing criminal responsibility, achieving justice for those harmed by lethal autonomous weapons, and much more. The conversation below has been lightly edited for length and clarity.

How do autonomous weapons challenge traditional understandings of responsibility when it comes to the law of armed conflict?

For most of the 20 years I’ve been teaching at Stanford, there’s been a real fascination and fixation on the problem of autonomous weapons. But the truth is that up until pretty recently, nobody really had any [of them]. It was more of an idea than a problem. It’s only in recent years that we have begun to see the deployment of tools that seem to have the capability [to make decisions themselves], so this does raise the question about compliance with illegal orders and where in the chain responsibility lies.

There are different theories about this. Some people say that if we’re engaged in an illegal attack, that has to be because of the programming or the code, so the manufacturers are somehow responsible for committing war crimes.

Other people would say no, that’s not the case, especially if the military has said the tool is not allowed to make very substantial autonomous judgments, or........

© Vox