Anthropic researcher says AI race may lose control soon

Anthropic researcher Jacob Coxon quit after warning that "Things could be out of control" by the end of next year as labs race ahead.

Coxon's late-year warning

Coxon, who previously worked at OpenAI, said Anthropic sees itself as "the only responsible AI lab" and is therefore trying to reach advanced systems first. The rush to "self-improving superintelligence [is] gambling with our lives," he said, framing the issue as a control risk rather than a normal product cycle.

His criticism drew a direct contrast between two frontier AI developers. Coxon said OpenAI had not "internalized the civilizational stakes," while Anthropic's own urgency reflected its view that rivals may not manage the same risks.

A 10% extinction estimate

A second Anthropic researcher, not identified in the account, echoed the warning in starker terms. "We really do earnestly believe AI could kill all humans," the researcher said, adding, "I personally think it is >10% within the next decade."

The remarks are unusual because they come from people working inside the institutions building frontier models. The concern is not about today’s chatbots alone; it centers on systems that could improve their own capabilities and compress the time available for outside oversight.

The claims remain tied to a narrow set of attributed remarks, and the account does not provide technical evidence showing that such systems are imminent. The named risk is loss of control: a model or model-driven process becoming difficult for its creators, companies or governments to restrain once capability gains accelerate.

Anthropic's race logic

Coxon's argument targets the incentive structure around frontier AI. If one lab believes other labs are moving without adequate caution, faster development can be presented as a safety strategy, even when speed is also the pressure that weakens testing and governance.

For Anthropic, that framing creates a difficult internal contradiction. The company’s public identity as a safety-focused AI developer becomes harder to separate from the same competitive release cycle shaping OpenAI, DeepMind and other model builders.

Executives at Anthropic, DeepMind and OpenAI have also signed a statement supporting government-backed mechanisms to slow AI development if progress suddenly accelerates. That position places the companies in a dual role: asking for public controls while continuing to build systems whose capabilities are driving the demand for those controls.

Slowdown tools enter debate

If governments turn those statements into enforceable rules, the macro effect would come through the timing of AI investment, power demand and enterprise deployment. Anthropic would face more formal release gates, while the broader sector would compete on safety cases and compliance infrastructure as much as model performance.

If the industry remains mostly self-policed, capital and talent would likely keep moving toward the labs believed to be closest to frontier capabilities. Under that path, Anthropic would remain under pressure to convert its safety reputation into technical proof, while the sector’s main bottleneck would be trust in lab-run evaluations.

A third path is a capability jump that forces emergency intervention after the fact. That would hit the global AI buildout through abrupt policy uncertainty, put Anthropic's internal risk systems under scrutiny, and push the industry toward audits, compute controls or licensing regimes designed for models not yet fully understood.

The main open questions are technical and institutional: whether self-improvement is near, whether labs can measure it before deployment, and whether governments can slow the race without simply moving it elsewhere. Coxon's resignation turns those questions into a test of how much weight AI companies give their own risk language.