Anthropic has proposed a new AI safety bill that could block the release of a model if it poses a risk of killing over 50 people or causing over a billion dollars in damage. The legislation mirrors California’s Senate Bill 53, which defines similar scenarios as catastrophic risks.
The bill outlines scenarios in which the release of an AI model could be blocked, including cases where the model could evade its developer’s controls, be used to create a biological weapon, or result in a cyberattack. These metrics are similar to those in California’s SB 53, which was recently passed into law.
The proposed legislation suggests that independent evaluators, registered with the U.S. Commerce Department, would assess a model’s potential for disaster. These evaluators could be paid by AI labs, provided the payment is not contingent on the evaluation results, according to Politico.
"This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June," said a spokesperson for Anthropic. The framework aims to address the risks posed by advanced AI models through a structured regulatory approach.
The theoretical risks outlined in the draft policy are somewhat removed from the current concerns of the tech industry, which has focused more on agentic hacking incidents. However, Anthropic’s proposal represents a step further than other industry suggestions, such as those from Nvidia, which has argued against new AI laws.
Anthropic did not comment on specific details of the proposal, and it remains unclear how the proposed system would ensure that payments do not influence evaluations. The legislation would require the Commerce Department to request a D.C. judge to halt a model’s release if evaluators determine an imminent catastrophic risk.
Source: techcrunch
