The Statement on AI Risk
A single sentence placing extinction risk from AI alongside pandemics and nuclear war, signed by Hinton and Bengio and by the heads of OpenAI, Anthropic and Google DeepMind, but not by LeCun or Ng.
What it was
On 30 May 2023 the Center for AI Safety, a nonprofit directed by Dan Hendrycks, published one sentence: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." Its preamble said that "it can be difficult to voice concerns about some of advanced AI's most severe risks," and that the statement was "meant to create common knowledge of the growing number of experts and public figures who also take some of advanced AI's most severe risks seriously."
The signatories included Geoffrey Hinton and Yoshua Bengio, and the chief executives of three frontier labs: Sam Altman of OpenAI, Dario Amodei of Anthropic and Demis Hassabis of Google DeepMind. Bill Gates also signed. Yann LeCun and Andrew Ng did not. Hendrycks said in the accompanying press release, "We need to be having the conversations that nuclear scientists were having before the creation of the atomic bomb."
What it changed
The statement differed from the March pause letter in two ways: it asked for nothing specific, and it was signed by the people running the labs. Altman had not signed the pause letter; he signed this one, two weeks after telling the Senate he would support a licensing agency for the most capable models. Hinton, who had declined to sign the pause letter, signed this a month after leaving Google.
The response from those who did not sign became its own argument. On 5 June 2023 Ng wrote on X, "Honestly, I don't get how AI poses this risk." By October he had gone further, telling the Australian Financial Review, "There are definitely large tech companies that would rather not have to try to compete with open source [AI], so they're creating fear of AI leading to human extinction." That month LeCun wrote that Altman, Hassabis and Amodei were "doing massive corporate lobbying" and that "If your fearmongering campaigns succeed, they will *inevitably* result in what you and I would identify as a catastrophe: a small number of companies will control AI." Hassabis replied to CNBC on 31 October that Google DeepMind was not seeking "regulatory capture."
The arguments it moved
What are the biggest risks from AI?
The statement turned a diffuse worry into a signed list, which made it possible to say exactly who stood where. On one side were Hinton, Bengio, Altman, Amodei and Hassabis. On the other, Ng said on 5 June 2023 he would "like to have a real conversation about whether AI is a risk for human extinction," and LeCun has called the idea "preposterous." A third position came from researchers focused on present harms: Timnit Gebru and her co-authors had argued in March 2023 that hypothetical risks were the focus of "a dangerous ideology called longtermism."
Who should set the rules for AI?
The regulatory-capture argument that followed was aimed at the lab leaders who had signed. In October 2023, days before the UK's AI Safety Summit at Bletchley Park, Ng and LeCun both argued that extinction warnings from companies at the frontier served to shut out open-source competitors through regulation. Hassabis rejected the charge on 31 October. Hendrycks's press release framed the goal as global coordination: "Mitigating the risk of extinction from AI will require global action."
Positions it bears on
-
Geoffrey Hinton, Existential risk from AI
Believes there is a real chance that AI more intelligent than humans takes control, and puts the probability that AI causes human extinction within three decades at 10 to 20 percent.
Hinton signed the statement weeks after leaving Google to speak about the risks.
-
Yoshua Bengio, Catastrophic risk from AI
Holds that loss of human control to a misaligned AI is a real possibility, that no current system is demonstrably safe against it, and that policy cannot wait for a disaster to prove the point.
Bengio signed, two months after signing the pause letter.
-
Sam Altman, AI risk
Altman has said since 2015 that superhuman machine intelligence could threaten humanity's existence and signed the May 2023 Center for AI Safety statement, while maintaining that the benefits justify building it and that OpenAI's way of building it is the safe one.
Altman signed, having declined to sign the pause letter in March.
-
Dario Amodei, Catastrophic risk
Amodei holds that catastrophic outcomes from advanced AI are plausible but not predetermined, and that the serious risks fall into a small number of categories that can be named and planned for.
Amodei signed as chief executive of Anthropic.
-
Yann LeCun, AI existential risk
Rejects the idea that AI poses an existential threat to humanity. He argues that intelligence does not imply a drive to dominate, that safety can be designed into systems through objectives and guardrails, and that catastrophic narratives rest on pessimism about people.
LeCun did not sign and in October 2023 accused the lab leaders who did of "massive corporate lobbying."
-
Andrew Ng, AI existential risk
Ng regards extinction scenarios as science fiction and argues that the fear is being manufactured, in part by companies seeking regulatory capture, and that it harms the field by scaring off students and distorting policy.
Ng did not sign and wrote on 5 June 2023 that he did not see how AI posed an extinction risk.
Sources
- Statement on AI Risk, Center for AI Safety
- AI Extinction Statement Press Release, Center for AI Safety, 30 May 2023
- Transcript, Senate Judiciary Subcommittee hearing on Oversight of AI, 16 May 2023 (Tech Policy Press)
- Andrew Ng, post on X, 5 June 2023
- Google Brain founder Andrew Ng says threat of AI causing human extinction is overblown, SiliconANGLE, 31 October 2023
- Google DeepMind boss hits back at Meta AI chief over 'fearmongering' claim, CNBC, 31 October 2023
- Meta's AI chief Yann LeCun on AGI, open-source, and AI risk, TIME, February 2024 (Internet Archive copy)
- Statement from the listed authors of Stochastic Parrots on the "AI pause" letter, DAIR Institute, 31 March 2023