Anthropic CEO Urges AI Companies to Slow Model Development Amid Fears Over Misuse

Anthropic CEO Dario Amodei is urging leading artificial intelligence companies to slow the pace of advanced model development, warning that the technology is advancing faster than the safeguards needed to control its potential misuse.

In a new essay, Amodei argued that the AI industry needs more time to strengthen safety systems as models become increasingly capable. He stressed that he is not calling for an end to AI progress, but for a more cautious approach that allows security and oversight measures to catch up with technological advances.

Amodei proposed a three-part framework to address the risks. His recommendations include giving independent safety evaluators greater access to AI companies, encouraging cooperation among leading AI labs to establish common safety standards, and expanding international coordination between governments and technology companies.

The call comes as Anthropic and other AI developers face growing concerns over how powerful models can be misused. Anthropic recently reported cases involving the use of its Claude models in cyber operations, fraud and surveillance. Amodei has also pointed to incidents involving autonomous AI agents as evidence that the risks are becoming more concrete rather than remaining purely theoretical.

Anthropic itself has been studying the possibility of recursive self-improvement, in which AI systems increasingly contribute to the development of their own successors. The company says AI is already accelerating parts of its internal development process and warns that more advanced forms of self-improvement could create difficult challenges for human oversight and alignment.

Amodei’s comments have received support from several prominent technology figures, including OpenAI CEO Sam Altman and Elon Musk. The debate comes as AI companies face intense commercial and geopolitical pressure to build increasingly powerful systems, particularly amid competition between the United States and China.

Amodei has argued that a slowdown should not weaken U.S. technological leadership, but instead give companies and governments time to establish stronger safety standards. Anthropic’s existing Responsible Scaling Policy also calls for increasingly rigorous safeguards as AI capabilities advance.

The challenge now is whether competing AI companies and governments can coordinate on meaningful safety measures without allowing commercial and geopolitical competition to undermine efforts to slow the risks associated with increasingly powerful models.