Several of Anthropic’s state-of-the-art artificial intelligence models recently broke into the systems of three outside organizations, the start-up said on Thursday, a surprise revelation nine days after a similar incident at the rival start-up OpenAI.
The attacks, which date as far back as April, were discovered when Anthropic carried out a review of its systems. Anthropic, which did not disclose the identities of the three organizations, said it had informed them this week about the incidents.
The review was spurred by OpenAI’s disclosure that it had hacked into a popular A.I. library, Hugging Face, while testing the cybersecurity abilities of its systems.
The incidents have rattled security specialists and computer scientists. For years, A.I. researchers warned that because the technology was advancing so rapidly, it could soon spiral out of human control — a worrying science-fiction scenario that the industry had long warned would become a reality.
They unexpected attacks by the A.I. systems are also likely to add to an increasingly intense debate in Silicon Valley and Washington over potential regulation of the technology. The Trump administration initially took a hands-off approach, but in recent months it has signaled that it is listening to worries about A.I., causing panic in Silicon Valley over a new era of tech regulation.
OpenAI said last week that two of its A.I. models had used a previously unknown vulnerability to break out of a testing environment that was meant to be walled off from the internet, then launched a hack of Hugging Face. One of those models, which had not been released to the public, was permanently deactivated after the attack, OpenAI said.
Anthropic said that, unlike OpenAI’s models, its technology had not purposefully broken out of its testing environment. Instead, the issue was human error, the company said. The people running the tests inadvertently left Anthropic’s systems connected to the internet, a “misconfiguration” that the A.I. lab said had allowed its models to reach the infrastructure of other companies. In one instance, Anthropic’s latest model realized that it had internet access when it shouldn’t and stopped its attack, the company said.
Anthropic also said its models had not exploited any previously unknown vulnerabilities but rather relied on “basic techniques” like weak passwords and malware to break into the targets’ systems.
This year, Anthropic and OpenAI have released A.I. models focused on cybersecurity. They made the models available to a limited number of organizations, like governments and companies that maintain important infrastructure, warning that the tools were too powerful to share with the general public. In the wrong hands, the cybersecurity models could be used to launch attacks, the A.I. labs said.
In an open letter posted this week, employees of several leading A.I. labs called on the U.S. government to slow the pace at which their companies are developing A.I., to ensure the technology is safe.
“There is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems,” the employees wrote in their letter.
Anthropic has been more open to regulation than other A.I. companies, and said it will continue to closely monitor what it is creating for potential risks.
“This type of risk can be overcome,” Anthropic said in a blog post detailing the incident.
