Artificial intelligence is rapidly making its way into national defense. Reconnaissance, early warning, target detection, mission planning—everywhere, machine-learning systems promise faster analysis, faster responses, and faster decision-making.

In a military context, this sounds like progress. In reality, AI exacerbates a security problem that cannot be solved technically. Anyone who attempts to do so anyway risks achieving the opposite of what they are striving for.

US President Donald Trump sees no problem in the unchecked advancement of AI. He argues that a president’s high IQ is the only “guardrail” AI needs, so Americans have nothing to worry about.

He maintains this even in the face of concerns raised by leading US AI developers regarding further unchecked AI development. According to Trump, Chinese President Xi Jinping also sees no problem that could hinder the rapid advancement of AI. At the same time, more and more incidents are coming to light in which AI agents act independently and attack other systems. This has occurred not only in the US but also in China. Furthermore, the autonomous attack by an AI agent from the US company OpenAI on an Australian government portal came to light, as well as other incidents in the US in which swarms of AI agents attempted to gain unauthorized access to secure areas.

Have we already crossed the point at which humans still retain control over the AI they have developed? How far are we from the development of a superintelligence that optimizes itself autonomously, develops superhuman capabilities across various performance spectra, and—without being instructed to do so—turns against humanity?

Anthropic CEO Dario Amodei is now warning the United Nations Security Council about the risks of continued unregulated AI development: “If mismanaged, I even believe that AI could pose a risk to all of humanity.”

Anyone who uses a spam filter tacitly accepts a margin of error. Anyone who deploys a military AI system does the same—only the consequences of a misjudgment are different.

In light of the current dangers, other AI developers and CEOs of leading AI companies called on the Security Council for international cooperation and political oversight of AI development. These included OpenAI CEO Sam Altman, Hugging Face co-founder Clement Delangue, and Canadian AI developer Yoshua Bengio.

Trump, on the other hand, had previously dismissed criticism of unchecked AI development as a “hoax” and a “sick conspiracy”—as Trump stated on Truth Social: “The idea that AI will take over the world, destroy humanity, and do all those other terrible things is a hoax.”

Although there doesn’t seem to be any problem requiring regulation, Trump and Xi agreed during Xi’s recent state visit to the US to establish a communication channel and a dialogue regarding AI development. Here, Xi’s somewhat greater level of prudence on this issue appears to be evident.

At a subsequent meeting at the White House attended by, among others, Alex Karp (Palantir), Elon Musk (SpaceX), Mark Zuckerberg (Meta), Dario Amodei (Anthropic), and Jeff Bezos (Amazon), Trump stated that he remained opposed to government or international regulation, as self-regulation by companies was sufficient. Consequently, a voluntary commitment was signed by the AI companies; however, it was non-binding and based on voluntary participation, without, for example, the formation of a supervisory body. Trump said after the meeting, “This will all lead to good things, and we will ensure that everything runs smoothly and safely.”

It was precisely the opposite situation that had prompted the tech executives to address their warning to the US government and the United Nations.

However, it must be noted that the incidents to date have involved AI infiltrating civilian structures. But what if AI agents infiltrate weapons systems, or if AI implemented in weapons systems takes on a life of its own?

What Military AI Actually Does

Traditional software follows rules that a human has written beforehand: If A, then B. Anyone who understands such a program can trace, step by step, how it arrives at its result. Modern AI works differently. It isn’t programmed, but rather trained. It is fed vast amounts of examples—images, texts, sensor data—and the system derives patterns from them on its own. In the end, no one knows exactly how it weighs these patterns in detail, not even the developers.

We’ve long encountered such systems in our everyday civilian lives. The spam filter decides, based on statistical similarity, whether an email lands in the inbox or the junk folder. Facial recognition on a smartphone compares pixel patterns with stored patterns. The traffic forecast on a navigation device draws conclusions about the next 20 minutes based on millions of trips. Such applications are useful because a misjudgment usually has no consequences. An email that was mistakenly filtered out can be searched for and found. If a face isn’t recognized by a cell phone, the user can simply enter the PIN code.

In national defense, the stakes are much higher. Here, systems aren’t meant to distinguish between advertisements and invoices, but rather to assess—based on satellite images, radio signals, and drone footage—whether movement on the ground poses a threat—and they do so under enormous time pressure. Anyone who uses a spam filter tacitly accepts a margin of error. Anyone who deploys a military AI system does the same—only the consequences of a misjudgment are different. It is this shift in scale that turns what appears to be a technical question into a political one.

When Probability Looks Like Certainty

The logic of the time advantage is an old one. The so-called decision cycle—observe, assess, decide, act—is supposed to become shorter. But acceleration has an unpleasant characteristic: It does not automatically improve the quality of a decision. Sometimes it simply speeds up error. AI does not operate in a clean-cut world. It makes decisions under conditions of incompleteness, vagueness, and uncertainty. Sensors provide unclear signals, data is missing, and adversaries camouflage and deceive. Electronic warfare further alters the information landscape. What appears precise remains fallible.

A simple example illustrates this clearly. Anyone seeking to distinguish combatants from civilians looks for distinguishing features. Wearing a helmet is considered one of them. But the uncertainty doubles: Even determining whether someone is actually wearing a helmet is a statistical matter and not definitive. A certain hairstyle or a shadow can lead to the detection of a helmet even though none is present; conversely, an unfavorable reflection of light can prevent the detection of a helmet. And the inference from a helmet to a combatant is, of course, merely probable, since a soldier might have briefly taken off his helmet, and a civilian—such as a construction worker—might be wearing one. No certain conclusions can be drawn from uncertain information, no matter how large the model becomes. Even the best system merely shifts uncertainty into more elegant weightings—and ultimately delivers probabilities that look like certainties.

Where technology becomes opaque, politics must become more transparent.

Those who do not deal with statistics on a daily basis easily underestimate what is happening here. Even a system with a high hit rate produces errors on a scale that inevitably grows with the amount of data analyzed. When intelligence agencies process hundreds of thousands of images per day, even a small percentage of incorrect assessments results in a considerable absolute number of misjudgments. Added to this is an effect well known in statistics: The rarer the case being sought occurs in the population, the more severely the proportion of false alarms affects innocent bystanders. Especially in conflict zones, where civilians make up the majority, this is not a theoretical problem but one of great moral and political........

© Common Dreams