Skip to content

Anthropic AI Biological Weapons Risks Prompt Account Blocks

Anthropic AI biological weapons concerns grow as the company blocks accounts linked to potentially dangerous biological research.

SAN FRANCISCO: Anthropic AI biological weapons concerns have prompted the artificial intelligence company to block multiple accounts that allegedly used its models in ways that could support the development of biological weapons.

In a report published Thursday, Anthropic, the company behind the Claude AI model, described biological misuse as one of the “most serious risks” associated with increasingly capable artificial intelligence systems.

The company outlined five real-world case studies involving users who allegedly circumvented safeguards and attempted to conceal the purpose of their research. The cases included potential gain-of-function research involving infectious diseases, as well as research involving novel venoms and toxins.

Anthropic Identifies Concerning Research Activity

After reviewing 30 days of activity, Anthropic said it identified around 35 distinct research efforts that involved potentially concerning activity. However, the company acknowledged that it could not determine with certainty whether the individuals involved intended to cause harm or were conducting legitimate scientific research.

Anthropic said the people featured in its case studies were working scientists, but it did not disclose their institutions or countries.

One case involved a request for help preparing a grant application for gain-of-function research examining the transmissibility and immune evasion of chikungunya virus.

Another case involved a researcher outside the United States using Claude for bird flu research focused on how viruses adapt to mammals and cause severe disease.

The report also highlighted research involving orthopoxviruses, a group that includes smallpox and mpox, as well as research involving venom toxins.

AI Benefits Also Create New Security Risks

AI is increasingly being used in drug discovery and biological research, with technology companies highlighting its potential to accelerate treatments for cancer and other diseases. However, the same capabilities that can help scientists discover new medicines and materials could potentially be misused by individuals seeking to develop biological weapons.

Anthropic said it wants its AI models to support legitimate scientific research, but acknowledged that newer models can perform increasingly complex tasks. As their capabilities improve, the company said, the potential for both beneficial and harmful applications also increases.

The company warned that bad actors could potentially exploit legitimate scientific applications of AI to create “plausible deniability” around harmful research.

Anthropic Strengthens AI Safety Measures

Anthropic said its latest models, including Claude Fable 5, feature stronger safeguards designed to restrict access to a broad range of dual-use biological research queries.

The company said there is evidence that its existing safeguards are working, but acknowledged that increasingly sophisticated AI systems will require more robust safety measures.

Anthropic also reported disrupting attempts to use its models for surveillance operations, scams and the development of conventional weapons, including drones and missiles, allegedly involving actors in China, Russia and Yemen.

The findings highlight the growing challenge facing AI companies as increasingly powerful models become accessible to researchers and users worldwide, raising concerns about how advanced artificial intelligence could be exploited for harmful purposes.

Leave a comment

Leave a Reply