The Second-Worst-Case Scenario For AI
The Second-Worst-Case Scenario For AI
‘Destruction of humanity’ ranks worst. But this outcome would be almost as bad.
Guy K. Mitchell, Jr. | September 18, 2026
Much has been written lately about the risks of artificial intelligence (AI). The CEOs of the three leading AI companies, Dario Amodei (Anthropic), Sam Altman (OpenAI), and Elon Musk (xAI), have all warned about the dangers of pursuing rapid development of autonomous super-intelligent AI models without “guardrails.” Most of those writings deal with the concern that as AI models evolve in capability and autonomy, they will have the capacity to cause physical harm to humans by attacking critical infrastructure such as water or electricity distribution systems, creating and proliferating deadly new virus strains, or deploying robots to attack humans.
Over the last several years, high-level researchers in AI have resigned from their companies, citing their concerns about the risks to humanity from AI models as the reason for their departure. The most recent example is Joseph Coxon, a training researcher who resigned from Anthropic on September 8, 2026, two months before his stock grants vested and before the planned Anthropic IPO. One would assume that Coxon forfeited a substantial sum of money to prove the courage of his convictions.
A training researcher is typically responsible for improving how a model is trained — and that often includes increasing some capabilities — while sharing responsibility for evaluation, reliability, and how those capabilities are constrained. Coxon had worked for Anthropic for four months and OpenAI for more than two years before he resigned. He said the reason for his departure was that Anthropic and its chief rival, OpenAI, “are racing straight to self-improving super-intelligence and gambling with our lives.” He warned that unregulated AI development could threaten human life by the end of the decade.
AI is the field of computer science and engineering concerned with building machines and........
