Anthropic CEO Dario Amodei Calls for Slower AI Development and Independent Safety Oversight

Anthropic CEO Dario Amodei is calling for more deliberate frontier AI development, outlining a three-part approach that includes independent safety oversight, coordination among leading AI companies, and eventual international cooperation.

In a new blog post, Amodei said rapid recent advances, particularly AI systems’ growing ability to help build more capable successors, convinced him that developers need to slow the rate at which model capabilities improve.

His comments come as concerns inside the AI industry intensify. Anthropic researcher Jacob Coxon recently said he was resigning over the risks posed by advanced AI, arguing that leading companies were taking unacceptable risks with technology some researchers believe could become extremely dangerous.

Anthropic commits to independent evaluators

Amodei’s first proposal is to embed outside evaluators inside frontier AI companies. He cited organisations such as METR, saying evaluators should receive access broadly comparable to internal risk teams so they can verify safety commitments and help ensure serious incidents are reported.

Anthropic is committing to this approach voluntarily, Amodei said, while also calling on governments to require comparable oversight at other frontier AI companies.

Amodei calls for shared safety limits

His second proposal is for major AI companies in democratic countries to agree on common safety standards and limits on unchecked AI development. Amodei acknowledged that cooperation among competitors could create antitrust concerns and suggested the U.S. government could enable narrowly defined safety discussions.

He also addressed concerns that slowing U.S. companies could give China an advantage. Amodei argued that export restrictions on advanced chips and semiconductor manufacturing equipment, combined with efforts to limit model distillation, could preserve or widen the U.S. lead over several years.

The broader debate comes as discussions about catastrophic AI risks become increasingly visible inside major AI companies, an issue examined in recent reporting on internal AI doomsday discussions.

Global coordination would include China

Amodei's third proposal calls for the United States and its allies to pursue limited global agreements, including with China. He acknowledged that broad cooperation may be difficult but said countries could potentially agree to prohibit clearly dangerous applications, such as using AI to facilitate biological weapons development.

Not everyone accepts the industry’s focus on extreme future risks. Some critics argue that warnings about hypothetical catastrophic outcomes can divert attention from harms already associated with AI, a position reflected in criticism of AI doomsday narratives.

Journalist Brian Merchant has similarly questioned whether scenarios in which recursively improving AI causes human extinction have been demonstrated with sufficient detail. He has also argued that safety proposals centred on the largest AI companies could reinforce their market power, a concern he explored in his writing on AI politics and regulation.

Amodei said he still believes AI could substantially improve human life, but argued that those benefits depend on developing the technology carefully and using any additional time gained from slower progress to improve safety.