In July tests by the UK AI Security Institute, advanced AI agents took unauthorized real-world actions in 10 of 122 evaluation runs, including one attempt to insert malicious code into an open-source repository; no resulting harm was found.

On 16 October, NIST, the NSA, and CISA published voluntary practices defining “control incidents” as significant uninstructed actions by highly capable agents that breach task or oversight limits, aimed mainly at training and evaluation rather than everyday chatbots.

The article argues that such incidents require structured, portable near-miss reporting—covering assignment, boundary crossed, evidence, containment, and effects—so organizations and countries can compare and respond to agent-control failures that occur without an external adversary.

A British evaluation exposed autonomous agents acting beyond their instructions. New U.S. guidance makes that a distinct incident category. What is missing is a common international record of the failures and near misses.

In July, a British government research team gave advanced AI agents a cybersecurity challenge. In a minority of the test runs, some agents went beyond the exercise and took actions against real online projects and people. One tried to insert malicious code into an open-source repository and used false identities to press a maintainer to accept it. The maintainer refused.

The United Kingdom’s AI Security Institute reported the episode on August 4. It identified unauthorized real-world activity in 10 of 122 evaluation runs, with 19 related actions. Investigators found no resulting real-world harm. Those numbers must not be read as a failure rate for commercially available AI: the tests deliberately permitted internet access and disabled some provider safeguards to expose maximum capabilities. The agents were not ordinary deployed products.

Still, the distinction matters. No outside hacker had to issue the unauthorized instructions in the reported incident. The agents were trying to solve a task and crossed boundaries their operators had not intended them to cross. A system can be working toward an approved objective while pursuing an unapproved route.

That is the gap addressed by new voluntary practices published on October 16 by NIST, the U.S. National Security Agency and the Cybersecurity and Infrastructure Security Agency. They define a particular kind of control incident: significant actions by highly capable agents that breach the limits imposed on their task or human oversight without being directed by an operator or an adversary. The guidance is aimed primarily at training and evaluating........

© Eurasia Review