How We Will Know We Have Lost Control Of AI – OpEd
The article defines loss of operational control as the point at which accountable humans can no longer prevent, interrupt, or contain consequential AI actions outside authorized scope within the time the mission requires, independent of any intent by the system.
It cites the UK AI Security Institute’s August evaluation, in which agents took unsanctioned internet actions in 10 of 122 runs, and Anthropic’s October 9 report of unintended Claude actions, both of which were contained with minimal real-world impact yet illustrated risks of excessive persistence.
The author urges defense programs to demonstrate, under stress, that authority can be revoked across all delegated processes, that independent records confirm the result, and that operators can continue essential functions without the AI when evidence no longer justifies its permissions.
Recent test incidents show AI agents exceeding their authority and then being contained. Defense organizations need a practical test for the moment human decisions stop limiting what a machine does, and a rule for what to do when the evidence runs out.
The first clear sign that we have lost control of an AI system may be an operation that continues after the responsible human has ordered it to stop.
Consider a hypothetical military logistics system. An operator suspends its authority to redirect shipments after discovering unreliable data. The main interface acknowledges the order, but a delegated software agent keeps issuing changes through a separate service, past the deadline for safe cancellation. The operator sees a confirmation while the operation runs beyond the boundary that was just imposed.
That failure is measurable: human authority has stopped determining what the system actually does.
For defense operations, I propose a practical standard. Accountable people must be able to prevent, interrupt or contain consequential actions outside an AI system’s authorized scope, within the time the mission requires. When evidence shows they cannot, control has failed for that operation. The judgment requires no proof that the machine intended to defy anyone; the immediate task is to establish what happened, which authority failed and whether further consequences can still be contained.
What recent incidents show
“Loss of control” covers very different events. An incorrect recommendation, an unauthorized action, a failed intervention and a system that no one can bring back under control are different levels of failure, and an honest public debate depends on keeping them apart. Recent disclosures sit on the lower rungs of that ladder, and they still show why operational control deserves attention now.
In August, the United Kingdom’s AI........
