Anthropic Urges Coordinated AI Development Pause

Anthropic, a leading artificial intelligence developer, stated on Thursday that frontier AI laboratories should establish a coordinated and verifiable framework to slow or temporarily halt development if advanced AI systems begin to improve themselves at a pace that outstrips society's ability to manage associated risks. This recommendation addresses the potential for AI systems to achieve "full recursive self-improvement," which Anthropic warns could increase the risk of humans losing control over these systems. The company cited its own experience, noting that as of May, over 80% of the code merged into its codebase was authored by its AI model, Claude.

The AI startup emphasized that a unilateral pause by a single company would have limited impact, primarily shifting leadership rather than fostering broader global deliberation. A meaningful slowdown would necessitate agreement among multiple well-resourced labs operating at the technological frontier, alongside clear rules defining the conditions for triggering or lifting such a pause and the oversight mechanisms. Anthropic's research arm, the Anthropic Institute, plans to develop systems to support such a slowdown and will convene discussions with policymakers, researchers, civil society groups, and other AI firms in the coming months to address these issues.

This announcement follows Anthropic's recent fundraising round, which valued the company at $965 billion, and its confidential filing for a U.S. initial public offering earlier this week. The company's call for coordinated action highlights growing concerns within the AI industry regarding the long-term safety and control of increasingly autonomous AI technologies.