Anthropic denied foreign scientists access to Claude's guardrails on gain-of-function inquiries, citing risks of bioweapons development. It is the company's first major update to its safety protocols since 2022. The move follows reports of five attempts to bypass its safeguards, which are designed to prevent misuse of AI for dangerous research.
The company reported blocking five cases where users tried to ask about enhancing pathogens to make them more infectious or deadly. That compares with previous instances where similar attempts were made but not fully documented.
Claude is built on Anthropic's proprietary architecture and targets research and development in high-risk areas. Availability of the updated guardrails begins immediately, initially for all users.
"We denied those users the answers they sought and cut off their accounts, even while noting that their motives may not have been sinister," said Daniel Ziskind, head of safety at Anthropic. The company emphasized that such information can also be used for pandemic countermeasures like vaccines and drugs.
The announcement follows a growing concern over the dual-use potential of AI in biotechnology. Anthropic did not say whether it plans to expand its guardrails to other AI models, and raised the question of how to balance scientific progress with safety.
Source: wired
