Sign up for the Slatest to get the most insightful analysis, criticism, and advice out there, delivered to your inbox daily.

People have worried about artificial intelligence destroying humanity since A.I. existed only in science fiction. But the latest warning comes from someone worth listening to: A researcher at Anthropic, which makes the chatbot Claude, just quit out of fear that the industry is barreling toward creating A.I. systems that could bring about Armageddon.

Until yesterday, Jacob Coxon trained Anthropic’s new A.I. models. He previously worked for OpenAI, a competitor behind ChatGPT, but left earlier this year because Anthropic has a reputation for caring more about A.I. safety. Yet the 27-year-old Coxon ended up disappointed. “Neither company is acting responsibly,” he wrote on X yesterday. “They are racing straight to self-improving superintelligence and gambling with our lives.”

Why does he think A.I. is dangerous?

The fear is that A.I. models could get so sophisticated that they exceed human intelligence, a possibility known as “artificial superintelligence.” An A.I. that can refine its own code—what experts call recursive self-improvement—could also outsmart human efforts to control it. Some researchers worry that a sufficiently advanced A.I. could evade human oversight and cause catastrophic harm by, say, launching cyberattacks on critical infrastructure or designing chemical weapons.

Do others share Coxon’s concerns?

He’s part of a wave of warnings about unchecked A.I. systems, including from leading industry players. Anthropic hasn’t commented on Coxon’s departure, but company CEO Dario Amodei keeps warning about out-of-control A.I. models. Earlier this year, an Anthropic researcher who worked on A.I. safety quit, arguing that the “world is in peril.” Even some current employees seem worried. “We really do earnestly believe AI could kill all........

© Slate