Glossary
Plain definitions of the terms that come up in the arguments about AI. Terms in the profiles link here the first time they appear.
-
AI agents
-
An AI agent is a system that takes actions toward a goal over many steps, rather than giving a single answer. Today the term usually means a language model that can use tools, such as running code, searching the web or operating software, then check the results and decide what to do next. "Agentic" describes systems that work this way.
Why it comes up. Ng argues agentic workflows are where near-term value lies, while Bengio argues autonomous agents carry the greatest risk and founded LawZero to build a non-agentic "Scientist AI" instead.
-
Algorithmic bias
-
Algorithmic bias is when an automated system produces systematically worse or unfair results for some groups of people, often because its training data under-represents them or reflects past discrimination. A 2018 audit of commercial face-analysis software found error rates far higher for darker-skinned women than for lighter-skinned men.
Why it comes up. Gebru co-wrote that audit (Gender Shades) and argues that documented harms like these deserve more attention than speculative extinction scenarios.
-
Alignment
-
Alignment is the problem of making an AI system reliably pursue the goals its designers and users intend, rather than some proxy that merely scored well in training. A system rewarded for a measurable target can find shortcuts that hit the target while missing the point, a failure called reward hacking. Alignment research tries to specify, train and check intended behavior, especially for systems more capable than the people overseeing them.
Why it comes up. Amodei's 2016 "Concrete Problems in AI Safety" framed alignment as ordinary engineering, Hinton and Bengio warn it may not be solved before systems exceed human ability, and LeCun argues that safe objectives can be designed in.
-
Artificial general intelligence (AGI)
-
Artificial general intelligence is a hypothetical AI system that can do most or all of the intellectual work humans can, rather than one narrow task. There is no agreed definition. OpenAI's charter defines it as highly autonomous systems that outperform humans at most economically valuable work; Hassabis describes a system with all the cognitive capabilities humans have; LeCun prefers "human-level AI" because he argues human intelligence is itself specialized.
Why it comes up. OpenAI and Google DeepMind name AGI as their goal, and people here disagree on whether it is years away (Altman, Amodei, Hassabis), decades away (Ng), or, as Gebru argues, too ill-defined to test for safety.
-
Attention mechanism
-
An attention mechanism lets a neural network decide, for each piece of output, which parts of the input to focus on. When translating a sentence, for example, the network scores every input word for relevance to the word it is about to write and takes a weighted mix of them. It replaced the older approach of squeezing a whole sentence into one fixed summary.
Why it comes up. Bengio's lab introduced attention for machine translation in 2014 (Bahdanau, Cho and Bengio), and the 2017 Transformer built an entire architecture from it.
-
Backpropagation
-
Backpropagation is the method used to train neural networks. After the network makes a guess, the program measures the error and works backward through the layers, calculating how much each weight contributed to it, then nudges every weight slightly in the direction that reduces the error. Repeating this over many examples is how a network learns.
Why it comes up. A 1986 Nature paper by Rumelhart, Hinton and Williams made backpropagation the standard way to train multi-layer networks; earlier versions existed, and Hinton has since said he doubts the brain learns this way.
-
Boltzmann machine
-
A Boltzmann machine is a type of neural network, introduced in 1985 by David Ackley, Geoffrey Hinton and Terry Sejnowski, that borrows equations from statistical physics. Its units switch on and off at random with probabilities set by their connections, and it learns by adjusting those connections until the patterns it produces resemble the examples it was shown.
Why it comes up. The Boltzmann machine was the work the Nobel committee cited when it awarded Hinton the 2024 Nobel Prize in Physics, shared with John Hopfield.
-
Compute
-
In AI, compute means the processing power used to train and run models, usually supplied by thousands of specialized chips (GPUs) in data centers and measured in the total number of arithmetic operations. The compute used to train leading models has grown several times over each year since about 2010. Because it is scarce and expensive, compute is treated as a strategic resource and a point of control for regulators.
Why it comes up. Altman has spent heavily to secure it, Amodei backs export controls on AI chips to China, Fei-Fei Li campaigns for public compute for universities, and laws such as the EU AI Act and California's SB 1047 used training-compute thresholds to decide which models they cover.
-
Convolutional neural network (CNN)
-
A convolutional neural network is a neural network designed for images. Instead of connecting every pixel to every unit, it slides small filters across the picture, so the same pattern detector (for a vertical edge, say) is reused everywhere in the image. This makes the network far smaller and lets it recognize an object wherever it appears.
Why it comes up. LeCun developed convolutional networks at Bell Labs around 1989 to read handwritten digits on checks, and a much larger one, AlexNet, won the 2012 ImageNet competition that turned computer vision toward deep learning.
-
Deep learning
-
Deep learning is the use of neural networks with many layers. Each layer builds on the one below it, so a network looking at photos might detect edges in early layers, shapes in middle layers and whole objects near the top. The approach needs large datasets and fast hardware, which became available around 2010, and it now underlies speech recognition, image recognition and chatbots.
Why it comes up. Hinton, Bengio and LeCun shared the 2018 Turing Award for it, and the 2012 AlexNet result on Fei-Fei Li's ImageNet dataset is the usual starting point for the current wave of AI.
-
Existential risk
-
Existential risk, as used in these debates, is the risk that advanced AI causes human extinction or a permanent, catastrophic loss of human control over the future. The concern is that systems more capable than people could pursue goals at odds with human interests and be impossible to stop. How likely this is, and whether it deserves the attention it gets, is disputed among the people on this site.
Why it comes up. Hinton, Bengio, Altman, Amodei and Hassabis signed the May 2023 statement putting AI extinction risk alongside pandemics and nuclear war; LeCun and Ng call such scenarios overblown, and Gebru argues the focus distracts from harms AI causes now.
-
Frontier model
-
A frontier model is one of the most capable general-purpose AI models at a given time, usually the largest and most expensive to train. The term came into policy use in 2023, when the UK government and a group of labs used it to single out the systems whose risks were least understood. Laws that regulate frontier models often define them by the compute or money spent on training.
Why it comes up. Proposals such as California's SB 1047, which Hinton and Bengio supported and LeCun, Ng and Li opposed, targeted frontier models; opponents argue it is better to regulate uses of AI than the models themselves.
-
Generative adversarial network (GAN)
-
A generative adversarial network is a pair of neural networks trained against each other to create realistic data such as images. One network, the generator, produces fakes; the other, the discriminator, tries to tell fakes from real examples. As each improves, the generator's output becomes harder to distinguish from the real thing.
Why it comes up. GANs came out of Bengio's Montreal lab in 2014, in a paper led by his student Ian Goodfellow, and were the leading method for generating images until diffusion models took over around 2021.
-
Interpretability
-
Interpretability is the study of what happens inside a trained neural network, to explain why it produces a given output. Mechanistic interpretability tries to map the network's internal units and connections to human-understandable concepts and step-by-step procedures, much as a biologist traces a circuit in the brain. The work matters because a network's behavior is learned, not written by programmers, so no one knows in advance how it works.
Why it comes up. Amodei has made interpretability a central Anthropic program and in April 2025 described "a race between interpretability and model intelligence".
-
Joint Embedding Predictive Architecture (JEPA)
-
A Joint Embedding Predictive Architecture is a design proposed by Yann LeCun in 2022 for learning world models. Rather than predicting every pixel of a missing image patch or future video frame, it predicts an abstract summary of it, so the model can ignore details that cannot be predicted, like the exact motion of leaves. Meta released image and video versions (I-JEPA in 2023, V-JEPA later).
Why it comes up. JEPA is LeCun's proposed alternative to generative language models as the path to human-level AI.
-
Large language model (LLM)
-
A large language model is a neural network, usually a Transformer, trained on a very large body of text to predict the next word (more precisely, the next token, a word or word fragment). Trained at enough scale, such models can answer questions, write code and hold conversations. ChatGPT, Claude and Gemini are products built on large language models that have been further trained to follow instructions.
Why it comes up. Whether large language models understand anything is the central dispute here; Hinton says they do, LeCun says they will never reach human-level intelligence, and Gebru and her co-authors called them "stochastic parrots".
-
Neural network
-
A neural network is a computer program built from many simple units, loosely modeled on brain cells, arranged in layers. Each unit takes numbers in, multiplies them by adjustable values called weights, adds them up and passes the result on. Training means adjusting millions or billions of weights until the network's outputs match examples, such as labeling photos correctly.
Why it comes up. Hinton, Bengio and LeCun kept working on neural networks through the 1990s and early 2000s, when most of the field had written them off, and nearly every system discussed on this site is one.
-
Open weights
-
An open-weight model is one whose trained weights (the numbers that make up the model) are published for anyone to download, run and modify. It differs from fully open-source AI, which by the Open Source Initiative's 2024 definition also requires the code and detailed information about the training data. Once released, weights cannot be recalled, and any safety training can be removed.
Why it comes up. LeCun and Ng defend open-weight models as necessary for competition, research and security, while Hinton opposes releasing the weights of the largest models and Bengio argues the decision should not rest with companies alone.
-
Reinforcement learning (RL)
-
Reinforcement learning is training by trial and error. An agent (a program that acts) tries actions in an environment, receives a numerical reward when things go well, and gradually learns which actions lead to the most reward over time. Deep reinforcement learning uses a neural network to decide which action to take.
Why it comes up. DeepMind under Hassabis, with David Silver leading the work, used reinforcement learning to build AlphaGo and AlphaZero; Silver argues that reward maximization is enough to produce general intelligence, and labs now use it to train language models to reason.
-
Reinforcement learning from human feedback (RLHF)
-
Reinforcement learning from human feedback is a way to shape a model's behavior using people's judgments. People compare pairs of model outputs and pick the better one; a second model learns to predict those preferences; and the main model is then trained by reinforcement learning to produce outputs that score well. It is how raw language models were turned into helpful assistants.
Why it comes up. Amodei co-authored the 2017 paper that introduced the method, and OpenAI's use of it in InstructGPT and ChatGPT made it standard; critics note it teaches models to please raters, which is not the same as being truthful or safe.
-
Scaling laws
-
Scaling laws are measured relationships showing that a model's error falls smoothly and predictably as it gets more parameters, more training data and more computing power. A 2020 OpenAI paper found these follow power laws over many orders of magnitude, and a 2022 DeepMind paper (Chinchilla) revised how to split a budget between model size and data. The scaling hypothesis is the further bet that capability keeps rising along these curves.
Why it comes up. Amodei, senior author of the 2020 paper, built his case for rapid progress on them; LeCun, Ng and Hassabis argue that more scale alone will not produce human-level intelligence.
-
Self-play
-
Self-play is a way of training a game-playing program by having it play against copies of itself. Each game produces fresh training data, and as the program improves its opponent improves too, so it keeps facing a challenge at its own level. No human games are needed once the rules are known.
Why it comes up. AlphaGo Zero, led by David Silver, learned Go from self-play alone in 2017 and beat the version that had defeated Lee Sedol; Silver points to it as evidence that systems can go beyond what they learn from human data.
-
Self-supervised learning
-
Self-supervised learning is training a model on raw data with no human labels, by hiding part of the data and asking the model to predict it. A language model that predicts the next word, or a vision model that fills in a masked patch of an image, is learning this way. Because the data supplies its own answers, the method can use almost unlimited amounts of text, images or video.
Why it comes up. LeCun has argued since 2016 that self-supervised learning is the bulk of intelligence (the cake, with supervised and reinforcement learning as icing and cherry), while David Silver argues that agents must also learn from their own experience.
-
Superintelligence
-
Superintelligence refers to a hypothetical AI that far exceeds the best human minds in nearly every field, not just matching them. The philosopher Nick Bostrom popularized the term in his 2014 book of that name. Those who use it disagree about whether it would follow quickly after AGI, and critics question whether the idea of a single scale of intelligence makes sense at all.
Why it comes up. Altman has written it may arrive within "a few thousand days" and Silver founded a company to build it, while Hinton and Bengio signed an October 2025 statement calling for a prohibition on developing it until it can be shown to be safe.
-
Supervised learning
-
Supervised learning is training a model on examples that people have labeled with the right answer, such as photos tagged "cat" or "dog" or emails marked as spam. The model learns to predict the label for new examples it has not seen. Its limit is the cost of labeling: every example needs a human answer.
Why it comes up. Fei-Fei Li's ImageNet, with millions of hand-labeled images, made large-scale supervised learning possible in vision, and Andrew Ng built much of his applied work on it; LeCun argues that labels are too scarce for machines to learn most of what they need this way.
-
Transformer
-
The Transformer is a neural network design introduced by Google researchers in June 2017. It processes all the words in a passage at once, using stacked attention layers to relate every word to every other word, rather than reading one word at a time. That made it fast to train on large computer clusters, and it became the basis of nearly every large language model (the T in GPT).
Why it comes up. Arguments over whether scaling up Transformers leads to general intelligence run through this site: Amodei and Altman built their labs on scaling them, while LeCun, Hassabis and Ng have said scaling alone will not get there.
-
World model
-
A world model is an AI system's internal representation of how its environment works, which it can use to predict what will happen next, including what will follow from its own actions. With one, a system can plan by imagining outcomes before acting. Some world models are trained on video to generate or simulate whole scenes that a user or robot can move through.
Why it comes up. LeCun and Fei-Fei Li argue that world models, not language models alone, are the route to human-level intelligence; Hassabis's Google DeepMind builds them too (Genie), and Hinton argues large language models already have a working model of the world.
Andrew Ng
Yoshua Bengio
David Silver
Mustafa Suleyman
Timnit Gebru
Emily M. Bender
Dario Amodei
Geoffrey Hinton
Yann LeCun
Ilya Sutskever
Sam Altman
Demis Hassabis
Gary Marcus
Fei-Fei Li
Jensen Huang