LONDON — Artificial intelligence company Anthropic has warned that rapidly advancing AI systems could eventually develop beyond humanity’s ability to effectively monitor and control them, urging governments, researchers and technology firms to prepare coordinated safeguards before that point is reached.
The company, best known for its Claude chatbot, said current technical trends indicate that AI capabilities are likely to increase significantly in the coming years, bringing major benefits in areas such as science, healthcare and innovation. However, Anthropic cautioned that the possibility of AI systems designing increasingly advanced successors could create new risks if safety measures fail to keep pace with technological progress.
Okay News reports that Anthropic is advocating for a coordinated and verifiable framework that would allow leading AI developers to slow or temporarily pause the development of frontier AI systems if their capabilities begin advancing faster than society can safely manage. The company said any meaningful pause would require cooperation among multiple major AI laboratories, clear rules governing when such measures should be triggered, and an agreed system of oversight.
According to Anthropic, a unilateral slowdown by a single company would likely have limited impact, as development efforts could simply shift to competitors rather than reducing overall risk. The firm noted that more than 80 percent of the code integrated into its own systems as of May had been generated by Claude, highlighting the growing role AI already plays in software development.
To support future policy discussions, Anthropic said its research division plans to study governance mechanisms and safety structures that could be used if advanced AI systems begin improving themselves at increasingly rapid rates. The company also intends to convene policymakers, academics, civil society groups and industry participants in the coming months to examine potential risks and strategies for international coordination.
The warning comes as investment in artificial intelligence continues to surge. Anthropic, founded in 2021 by former OpenAI employees, recently completed a funding round that reportedly valued the company at approximately $965 billion and has confidentially filed for a U.S. initial public offering.
