Sep 2024, policy, California Legislature (bill by State Senator Scott Wiener); vetoed by Governor Gavin Newsom

The veto of California's SB 1047

Governor Newsom's veto of a California bill that would have imposed safety duties on developers of the largest AI models, after a summer in which Hinton and Bengio backed it and LeCun, Ng and Li opposed it.

What it was

SB 1047, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, was written by State Senator Scott Wiener. It would have applied only to models that cost at least $100 million and used more than 10^26 floating-point operations to train, and it would have made their developers liable for putting in place safety protocols to prevent "critical harms," including the ability to shut down models under their control. The Legislature passed it at the end of August 2024.

On 29 September 2024 Governor Gavin Newsom returned it unsigned. His veto message said that "By focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that could give the public a false sense of security," that "Smaller, specialized models may emerge as equally or even more dangerous," and that the bill "applies stringent standards to even the most basic functions - so long as a large system deploys it." He added, "I agree with the author - we cannot afford to wait for a major catastrophe to occur before taking action." The same day he asked Fei-Fei Li, Mariano-Florentino Cuéllar and Jennifer Tour Chayes to help the state develop what he called "workable guardrails."

Read the original

What it changed

The fight split prominent AI researchers in public. On 7 August 2024 Yoshua Bengio, Geoffrey Hinton, Lawrence Lessig and Stuart Russell wrote in a letter published by TIME that the bill was the "bare minimum for effective regulation of this technology" and "a remarkably light-touch piece of legislation." Fei-Fei Li had written in Fortune the day before that it would "harm our budding AI ecosystem." Andrew Ng wrote in TIME on 29 August that it made "the fundamental mistake of regulating a general purpose technology rather than applications of that technology." Yann LeCun also opposed it.

The companies split too. OpenAI opposed the bill. Anthropic, after its amendments were partly adopted, wrote to Newsom on 21 August 2024 that "we believe its benefits likely outweigh its costs," while adding that "we are not certain of this." Former House Speaker Nancy Pelosi called the bill "well-intentioned but ill-informed." After the veto, Wiener called it "a setback for everyone who believes in oversight of massive corporations that are making critical decisions that affect the safety and welfare of the public and the future of the planet."

The arguments it moved

Who should set the rules for AI?

SB 1047 put the argument over whether to regulate AI models or AI applications to a yes-or-no decision. Ng's TIME essay made the applications case with an electric motor: "It makes more sense to regulate a blender, rather than its motor." The Bengio, Hinton, Lessig and Russell letter replied in advance that the bill was already the minimum. Newsom's veto message named the question directly: "Key to the debate is whether the threshold for regulation should be based on the cost and number of computations needed to develop an AI model, or whether we should evaluate the system's actual risks regardless of these factors." He chose the second, and asked Li, one of the bill's best-known opponents, to help write the alternative.

Compare every leader's position on this

Who should have access to powerful models?

Much of the opposition concerned open-weight models. Ng wrote that "By raising compliance costs for open source efforts, this will discourage the release of open models," and Li wrote that "Open-source development is important in the private sector, but vital to academia, which cannot advance without collaboration and access to model data." The bill's thresholds meant it would have applied only to models costing at least $100 million to train.

Compare every leader's position on this

Positions it bears on

Geoffrey Hinton, Regulation of frontier AI

Argues that market incentives will not produce safe AI and that only government regulation can force companies to spend more on safety; supported California's SB 1047 and has pressed the US Congress to act.

Hinton co-signed the August 2024 letter calling SB 1047 the "bare minimum for effective regulation."

Critics and counterpoints
Yoshua Bengio, Regulation of frontier AI

Wants binding rules that scale scrutiny with risk, registration of frontier models, large public investment in safety research, and independent evaluation before deployment; supported California's SB 1047 and argued companies cannot be trusted to assess themselves.

Bengio co-signed the August 2024 letter and wrote separately in Fortune in support of the bill.

Critics and counterpoints
Fei-Fei Li, Regulating frontier models (SB 1047)

Opposed California's SB 1047 on the grounds that holding original developers liable for downstream misuse would chill open-source development and academic research; prefers rules aimed at applications and evidence of harm.

Li opposed the bill in Fortune in August 2024 and was asked by Newsom on the day of the veto to help develop an alternative.

Critics and counterpoints
Andrew Ng, Regulation

Regulate harmful applications, not general-purpose models. He opposed California's SB 1047, supports rules against specific harms such as non-consensual deepfakes, and argues that licensing or liability at the model level entrenches incumbents and suppresses open source.

Ng's August 2024 TIME essay argued that SB 1047 regulated a general-purpose technology instead of its applications.

Critics and counterpoints
Yann LeCun, Regulation

Supports policy that keeps research and model release open, and opposes rules such as California's SB 1047 that would hold developers liable for downstream harms from large models, which he says rest on inflated estimates of near-term capability.

LeCun opposed the bill as one that would hold developers liable for downstream harms.

Critics and counterpoints
Dario Amodei, Regulation

He wants rules aimed at a few catastrophic risks, opposed a federal moratorium on state laws, and since June 2026 has argued that voluntary commitments and transparency are no longer enough.

Amodei's Anthropic gave the amended bill qualified support, writing that its benefits "likely outweigh its costs."

Critics and counterpoints

Sources

Last verified 2026-09-22.