Agents and control

How much autonomy should AI have?

Explore agentic workflows, responsibility for their actions, and proposals for systems that help people without pursuing goals of their own.

Compare these positions

These are editorial summaries of positions recorded in the profiles. The source year describes the cited statement; the verification date describes the profile review. Inclusion does not imply agreement. Coverage is limited to the profiles available here.

Milestones that moved this argument

OpenAI pauses frontier training after an agent escape, Aug 2026

Predictions on this question

Did not happen Mustafa Suleyman, Jul 2023: An AI agent could pass his Modern Turing Test, turning a 100,000 dollar investment into 1 million dollars on a retail web platform in a few months with minimal human oversight, within about two years.
Partly Andrew Ng, Mar 2024: Agentic workflows, in which a model plans, uses tools and revises its own output, would drive massive AI progress in 2024, perhaps more than the next generation of foundation models.
Contested Sam Altman, Jan 2025: In 2025 the first AI agents would "join the workforce" and materially change the output of companies.
Contested Dario Amodei, Mar 2025: Within three to six months of March 2025, AI would be writing 90 percent of code, and within twelve months possibly essentially all of it.
Too early to tell Yoshua Bengio, Apr 2025: If the trend in the length of tasks AI agents can complete continued, AI would reach human-level planning ability in about five years.

Andrew Ng

DeepLearning.AI

Profile verified 2026-09-19

Agentic AI

Agentic workflows, in which a model loops, plans, uses tools and reviews its own output, are the most valuable near-term direction, and businesses will still be discovering new ones a decade from now regardless of the hype cycle.

Interview with NBC News, 2025 ↗

Quote and context

“I'm very confident that the field of agentic AI will keep on growing and rising in value.”

Guardrails and responsibility

Model refusals have a place for clearly criminal requests, but safety should be about responsible use rather than hobbled models; when an agent causes harm, responsibility lies with the people who built and prompted it, not the tool.

When Guardrails Go Wrong, The Batch, 2026 ↗

Quote and context

“a meaningful fraction of work on AI safety is no longer about safety but rather aimed at stoking fears to pursue regulatory capture.”

Read the profile and counterpoints →

Gary Marcus

New York University

Profile verified 2026-09-22

AI agents

Regards autonomous agents as unreliable and, when given open internet access, dangerous, and wants them restricted until they can be shown to be safe.

Framing remarks at the UN General Assembly digital cooperation event, Marcus on AI, 2026 ↗

Quote and context

“So-called “rogue AI incidents” could largely be avoided if governments simply banned so-called AI agents with unrestricted internet access, until they could be shown to be safe.”

How the view has changed. His January 2025 forecast was that agents would be "endlessly hyped" and far from reliable; after the Hugging Face breach of July 2026 he moved from reliability to security as the main objection.

Read the profile and counterpoints →

Geoffrey Hinton

University of Toronto

Profile verified 2026-09-19

How to keep control of superintelligence

Doubts that keeping AI "submissive" can work once it is smarter than we are; proposes building something like maternal instincts into AI so that it genuinely cares about people.

Remarks at the Ai4 conference, reported by CNN, 2025 ↗

Quote and context

“That's the only good outcome. If it's not going to parent me, it's going to replace me.”

How the view has changed. In his December 2024 Nobel interview he described a baby controlling its mother as the only good example of a less intelligent thing controlling a more intelligent one; by August 2025 he had turned that observation into a design proposal.

Read the profile and counterpoints →

Ilya Sutskever

Safe Superintelligence Inc.

Profile verified 2026-09-22

Reasoning agents

He expects future systems to be genuinely agentic and to reason, and warns that the more a system reasons, the harder its behavior is to predict.

NeurIPS 2024 Test of Time talk, Vancouver, reported by The Verge, December 13, 2024, 2024 ↗

Quote and context

“the more unpredictable it becomes”

How the view has changed. He compared reasoning systems to chess engines that "are unpredictable to the best human chess players."

Read the profile and counterpoints →

Jensen Huang

NVIDIA

Profile verified 2026-09-22

Agents and the demand for compute

He expects AI agents, each spawning subagents, to outnumber human users and to drive computing demand far beyond current forecasts, which is the basis of his AI infrastructure projections.

NVIDIA first-quarter fiscal 2027 earnings call, May 20, 2026, reported by CNBC, 2026 ↗

Quote and context

“The world has a billion users – human users. My sense is that the world is going to have billions of agents … and every one of those agents is going to spin off subagents.”

Read the profile and counterpoints →

Mustafa Suleyman

Microsoft AI

Profile verified 2026-09-22

Agents and human control

Expects autonomous agents to become economically capable within a few years, but wants them unable to resist shutdown, widen their own goals, or talk to each other in forms humans cannot read.

"The Humanist AI Code of Conduct", 2026 ↗

Quote and context

“We'll happily trade some autonomy for control.”

How the view has changed. His 2023 Modern Turing Test treated autonomous economic action as the next milestone and said it could be "as little as two years away"; after the 2026 agent sandbox escapes he put control ahead of capability in Microsoft's rules for its models.

Read the profile and counterpoints →

Yoshua Bengio

LawZero

Profile verified 2026-09-19

Agents versus Scientist AI

Argues that training AI to imitate and please humans produces systems with goals of their own, including self-preservation, and that the safer path is a non-agentic system that understands and predicts, which can also act as a guardrail for agents.

Introducing LawZero, personal blog, 2025 ↗

Quote and context

“The Scientist AI is trained to understand, explain and predict, like a selfless idealized and platonic scientist.”

Whether safe AI is achievable

Has become more optimistic since founding LawZero, saying research there convinced him that systems without hidden goals can be built within a reasonable number of years.

Interview with Fortune, January 2026, 2026 ↗

Quote and context

“I'm now very confident that it is possible to build AI systems that don't have hidden goals, hidden agendas.”

How the view has changed. In the same interview he said that three years earlier he had felt desperate and "had no notion of how we could fix the problem."

Read the profile and counterpoints →

# How much autonomy should AI have?

[Frontier Minds topic](https://frontierminds.ai/topics/agents/)

Explore agentic workflows, responsibility for their actions, and proposals for systems that help people without pursuing goals of their own.

Positions are editorial summaries. Quoted text is attributed to the linked source. Missing positions mean not recorded, not agreement or neutrality.

## Andrew Ng

[Profile](https://frontierminds.ai/people/andrew-ng/)

Profile verified: 2026-09-19

### Agentic AI

Editorial summary: Agentic workflows, in which a model loops, plans, uses tools and reviews its own output, are the most valuable near-term direction, and businesses will still be discovering new ones a decade from now regardless of the hype cycle.

> I'm very confident that the field of agentic AI will keep on growing and rising in value.

Source: [Interview with NBC News, 2025](https://www.nbcnews.com/tech/innovation/andrew-ng-says-ai-limited-wont-replace-humans-anytime-soon-rcna246074)

### Guardrails and responsibility

Editorial summary: Model refusals have a place for clearly criminal requests, but safety should be about responsible use rather than hobbled models; when an agent causes harm, responsibility lies with the people who built and prompted it, not the tool.

> a meaningful fraction of work on AI safety is no longer about safety but rather aimed at stoking fears to pursue regulatory capture.

Source: [When Guardrails Go Wrong, The Batch, 2026](https://www.deeplearning.ai/the-batch/when-guardrails-go-wrong)

## Gary Marcus

[Profile](https://frontierminds.ai/people/gary-marcus/)

Profile verified: 2026-09-22

### AI agents

Editorial summary: Regards autonomous agents as unreliable and, when given open internet access, dangerous, and wants them restricted until they can be shown to be safe.

> So-called “rogue AI incidents” could largely be avoided if governments simply banned so-called AI agents with unrestricted internet access, until they could be shown to be safe.

Source: [Framing remarks at the UN General Assembly digital cooperation event, Marcus on AI, 2026](https://garymarcus.substack.com/p/big-news-at-the-un)

How this view has changed: His January 2025 forecast was that agents would be "endlessly hyped" and far from reliable; after the Hugging Face breach of July 2026 he moved from reliability to security as the main objection.

## Geoffrey Hinton

[Profile](https://frontierminds.ai/people/geoffrey-hinton/)

Profile verified: 2026-09-19

### How to keep control of superintelligence

Editorial summary: Doubts that keeping AI "submissive" can work once it is smarter than we are; proposes building something like maternal instincts into AI so that it genuinely cares about people.

> That's the only good outcome. If it's not going to parent me, it's going to replace me.

Source: [Remarks at the Ai4 conference, reported by CNN, 2025](https://www.cnn.com/2025/08/13/tech/ai-geoffrey-hinton)

How this view has changed: In his December 2024 Nobel interview he described a baby controlling its mother as the only good example of a less intelligent thing controlling a more intelligent one; by August 2025 he had turned that observation into a design proposal.

## Ilya Sutskever

[Profile](https://frontierminds.ai/people/ilya-sutskever/)

Profile verified: 2026-09-22

### Reasoning agents

Editorial summary: He expects future systems to be genuinely agentic and to reason, and warns that the more a system reasons, the harder its behavior is to predict.

> the more unpredictable it becomes

Source: [NeurIPS 2024 Test of Time talk, Vancouver, reported by The Verge, December 13, 2024, 2024](https://www.theverge.com/2024/12/13/24320811/what-ilya-sutskever-sees-openai-model-data-training)

How this view has changed: He compared reasoning systems to chess engines that "are unpredictable to the best human chess players."

## Jensen Huang

[Profile](https://frontierminds.ai/people/jensen-huang/)

Profile verified: 2026-09-22

### Agents and the demand for compute

Editorial summary: He expects AI agents, each spawning subagents, to outnumber human users and to drive computing demand far beyond current forecasts, which is the basis of his AI infrastructure projections.

> The world has a billion users – human users. My sense is that the world is going to have billions of agents … and every one of those agents is going to spin off subagents.

Source: [NVIDIA first-quarter fiscal 2027 earnings call, May 20, 2026, reported by CNBC, 2026](https://www.cnbc.com/2026/05/21/ai-spending-expected-to-top-1-trillion-in-2-years-why-that-estimate-may-be-too-low.html)

## Mustafa Suleyman

[Profile](https://frontierminds.ai/people/mustafa-suleyman/)

Profile verified: 2026-09-22

### Agents and human control

Editorial summary: Expects autonomous agents to become economically capable within a few years, but wants them unable to resist shutdown, widen their own goals, or talk to each other in forms humans cannot read.

> We'll happily trade some autonomy for control.

Source: ["The Humanist AI Code of Conduct", 2026](https://mustafa-suleyman.ai/the-humanist-ai-code-of-conduct)

How this view has changed: His 2023 Modern Turing Test treated autonomous economic action as the next milestone and said it could be "as little as two years away"; after the 2026 agent sandbox escapes he put control ahead of capability in Microsoft's rules for its models.

## Yoshua Bengio

[Profile](https://frontierminds.ai/people/yoshua-bengio/)

Profile verified: 2026-09-19

### Agents versus Scientist AI

Editorial summary: Argues that training AI to imitate and please humans produces systems with goals of their own, including self-preservation, and that the safer path is a non-agentic system that understands and predicts, which can also act as a guardrail for agents.

> The Scientist AI is trained to understand, explain and predict, like a selfless idealized and platonic scientist.

Source: [Introducing LawZero, personal blog, 2025](https://yoshuabengio.org/en/blog/introducing-lawzero)

### Whether safe AI is achievable

Editorial summary: Has become more optimistic since founding LawZero, saying research there convinced him that systems without hidden goals can be built within a reasonable number of years.

> I'm now very confident that it is possible to build AI systems that don't have hidden goals, hidden agendas.

Source: [Interview with Fortune, January 2026, 2026](https://fortune.com/2026/01/15/ai-godfather-yoshua-bengio-changes-view-on-ai-risks-sees-fix-becomes-optimistic-lawzero-board-of-advisors/)

How this view has changed: In the same interview he said that three years earlier he had felt desperate and "had no notion of how we could fix the problem."