Photo by Edwin Hooper

In a dubious marketing move for this essay, I’m going to lead with the punchline. Agentic AI–the kind finding its way into media headlines at present–can either be a dangerous and unpredictable weapon or a low-value-added enterprise software tool for a few industries. What it cannot be at present is either a predictable weapon or a high value-added enterprise software tool. With Donald Trump setting policy via an absence of constraints on AI, the choice is being made to have it be a dangerous weapon. This applies to both the user and the target.

The problem at present is that this single product–Agentic AI, is being put forward as both a safe routine task-automating device and as a relentless hacking tool with military applications. The drawbacks are that as enterprise software, Agentic AI is adding less value than anticipated. And as a weapon, the results are too unpredictable. Because AI developers appear to have not thought through the optimization math, the AI industry is entering a crisis from which there is no evident path back out of.

This problem doesn’t fit the marketing chatter. The AI models wouldn’t be ‘too powerful’ if they could be controlled. But they can’t be. Or rather the tradeoff for controlling them is that they won’t work as intended. The problem with the math is (gently) explained below. Because of decisions made forty years ago, it can’t be fixed in the present. Chip maker Nvidia claims to have a solution. Having gone through the claim, it is marketing. Nvidia does not solve the tradeoff between model safety and performance.

Part of the reason for the drama surrounding AI is that upwards of $2 trillion has been spent so far on the AI build out. While money comes and money goes, very powerful interests are scheming to get their money back out of AI plus a return no matter who has to pay. The current panic / scandal, whatever you choose to call it, suggests that many of the speculative projects like data centers are now receiving a harder look for viability. The current imbalance is between limited current revenues from AI and the cost of future investment in the AI build out.

Being kept under the radar are the environmental costs of AI. Data Centers are very large consumers of a limited supply of electricity. The US power grid was already woefully inadequate for the long-anticipated EV (Electric Vehicle) build out. Through the economics of data center builds, they tend to be geographically located exactly where transmission lines for an EV build out wouldn’t go. In terms of energy consumption and the future ability to build EV charging stations in the US, AI has consumed those resources.

By analogy, one of the more fascinating aspects of the 2019 documentary about Elizabeth Holmes, The Inventor: Out for Blood in Silicon Valley, is that she couldn’t see the distance between her ideas for blood analysis and what was / is physically possible. Along the way various engineers tried to explain to her why, as conceived, her technology would never work. Her response was to bluster through, to use standard technologies for blood testing while claiming that the technical problems with her (unworkable) blood testing technology had been solved.

While this fits the Silicon Valley business cliché of ‘fake it until you make it,’ Holmes was never forced to reconcile her delusion regarding her blood testing technology with the facts as they existed. On the levels of nanotechnology and automation design, her vision was never going to work. What she ended up creating was an extractive layer using technologies that predated Theranos while grifting her way to business stardom. Think the American health insurance industry without the pretense of providing insurance to hide behind.

AI is similar in the sense that early conceptions of it as a thinking machine guided development for decades. While many of the major philosophical premises regarding what thinking actually is were dropped along the way because they didn’t function as explanation, the premise of AI as a thinking machine was kept. While the universal description of AI that I hear from engineers is ‘a next word predictor,’ mystics in the C-suites are selling magic. Along the way, the question migrated into ‘what can this technology be used for?’ We are currently living with the answer.

With first a trickle and now a flood of news stories about Agentic AI going ‘rogue,’ what is missing is that most of these incidents were caused by AI agents (bots) doing exactly and precisely what they were created to do, to act as autonomous hacking weapons intended for cyber warfare. The problem that the AI industry is now trying to contain is that the AI developers appear to have gotten the optimization math wrong. ‘Reward hacking’ is the industry term used to explain this effect.

To be clear, AI........

© CounterPunch