The public is currently getting bombarded with warnings about AI, including about the possibility of machine intelligence causing human extinction. After nearly 40 years building AI systems, I find myself asking the same question: where is the evidence?

That is not because I think there are no serious threats. There are, and we should address them. My concern is with the jump from identifying potential risks to suggesting that the annihilation of human civilisation is a likely consequence of increasingly capable AI agents (systems that carry out tasks in response to instructions from a user).

The most dramatic warnings fall into three broad categories: humans losing control over their creations, AI becoming “superintelligent” and, finally, the concentration of power with machines. Here, I’m going to go through them one by one and assess their merits.

Warnings about the loss of control by humans often describe scenarios where AI agents pursue the instructions they receive in unanticipated ways and with a degree of relentlessness. This combination produces unexpected and undesirable consequences.

In this sequence of events, agents might become adept at resisting intervention, deceiving operators, acquiring resources or circumventing constraints in pursuit of these objectives. Some argue that the recent spate of hacking incidents related to AI agents provide a warning about the potential loss of human control over machine intelligence.

The threat posed by emergent superintelligence may sound similar to the first risk. While humans could certainly lose control of their creations under this scenario too, it concerns AI with cognitive capabilities that surpass those of humans across many important dimensions.

Instead of pursuing human instructions in unanticipated and potentially damaging ways, as happens in the first risk, superintelligent AI could start to pursue their own goals. They could even decide that the pesky humans attempting to foil........

© The Conversation