The central disagreement between Anthropic and OpenAI is who should have the authority to stop a frontier AI system when it presents a serious risk. Anthropic would let the state seek a court order to suspend a model. OpenAI wants developers to maintain effective shutdown controls and be required to use them when other interventions would not address the danger.
Both companies made their proposals in letters to California Governor Gavin Newsom after his September 18 executive order asked state officials to consider additional safeguards. They support independent safety assessments and broader reporting when AI systems act outside their intended permissions. The order itself does not impose the proposed requirements; it directs state agencies to deliver recommendations by November 16.
Anthropic’s October 15 submission says the attorney general should be able to seek a court order if an independent evaluation indicates a model poses a near-term risk of catastrophic harm. That order could halt deployment, internal use, or further development.
OpenAI’s October 20 letter takes a different route: amend California’s Senate Bill 53 to require frontier developers to keep shutdown controls in place and use them when unsafe conditions arise and no lesser step would adequately reduce the risk.
Oversight needs independent scrutiny
The two proposals also differ in how evaluators should review companies’ safety work. Anthropic wants independent evaluators to examine developers’ claims and publish their own findings. It proposes risk reports at least every six months, covering serious hazards from models used internally as well as those released to customers. Evaluators would receive unredacted reports, access to the developers’ most capable models, and relevant internal information.
To protect evaluator independence, Anthropic proposes that developers contribute to a state-administered pooled fund instead of paying evaluators directly. The company said the independent evaluation organization METR had received employee-like access in a pilot, with work involving SecureBio and Accenture also under development.
OpenAI supports independent technical assessments but draws a distinction between those reviews and compliance audits. Technical assessments would test the evidence behind safety claims and whether safeguards work. Compliance audits would check that companies follow their published safety frameworks and legal obligations. OpenAI recommends annual compliance audits, along with focused technical reviews of serious risks during training, evaluation, and internal deployment. It also wants California to work with the National Institute of Standards and Technology on shared assessment standards.
Sanchit Vir Gogia, chief analyst at Greyhound Research, described the proposals as addressing complementary needs: operators must contain unsafe activity quickly, while independent reviewers examine both intervention and inaction. Credible oversight, he said, has to withstand an adverse finding.
Shutdown controls must match how systems run
OpenAI says “shutdown controls” better captures the range of actions needed than a single kill switch. Responses could include stopping a training run, isolating an agent, removing its tool or network access, or suspending a service. Responsibility should follow operational control across developers, inference providers, and downstream deployers.
That division matters because developers can’t reliably disable independent copies of open-weight models running on outside systems. Anthropic makes a similar case for targeted safeguards, describing a “circuit breaker” that can contain a problem at the agent, customer, deployment environment, or model level.
For companies adopting AI, Gogia says contracts should spell out which party can intervene, how incident reports will be shared, and who can authorize a restart. Each assigned responsibility needs a control that the responsible party can actually execute. CIOs should also test recovery plans instead of assuming a switch to another model will keep operations safe.
California’s review will consider independent verification within AI labs, outside checks of safety assessments, shutdown mechanisms, and expanded incident reporting. The practical test for any resulting framework will be whether it connects clear authority to controls that can act quickly across the different environments where AI systems run.
The article originally appeared on ComputerWorld.