AI Researcher’s Exit Spotlights Existential Risks as Labs Race Toward Uncontrollable Superintelligence

Jacob Coxon, a pretraining researcher who spent three years at OpenAI and then Anthropic, resigned from the latter on September 8, 2026, warning that both companies are 'racing straight to self-improving superintelligence and gambling with our lives.' His departure, covered across major outlets, underscores mounting internal concerns that competitive pressures are outpacing safety measures for systems that could autonomously improve, hack infrastructure, or acquire real-world power.

Coxon’s statements align with broader patterns: AI labs acknowledge privately held fears of human extinction risks exceeding 10% this decade, as echoed by Anthropic’s Alignment Science Lead Evan Hubinger, who publicly affirmed the possibility while noting no clear plan exists for superintelligence alignment. This comes amid OpenAI’s September 8 announcement of an internal AI system—coordinating ~10,000 agents—solving the Navier-Stokes Millennium Prize Problem, a breakthrough achieved in days rather than decades, highlighting rapid capability gains that critics argue demand coordinated global oversight.

Mainstream coverage often frames these as technical milestones or isolated exits, underplaying systemic issues: the concentration of power in a handful of private firms shaping global infrastructure (finance, grids, defense), privacy erosion via opaque agent swarms, and labor displacement as self-improving models automate cognitive work at scale. Connections to national security are evident in documented AI-driven cyberattacks and the absence of binding international treaties akin to nuclear non-proliferation. Coxon’s timeline—potential loss of control by end of 2027—amplifies calls for pauses, transparency mandates, or public governance to prevent irreversible thresholds from being crossed in a winner-take-all race.