Transformer
The Transformer is a neural network design introduced by Google researchers in June 2017. It processes all the words in a passage at once, using stacked attention layers to relate every word to every other word, rather than reading one word at a time. That made it fast to train on large computer clusters, and it became the basis of nearly every large language model (the T in GPT).
Why it comes up
Arguments over whether scaling up Transformers leads to general intelligence run through this site: Amodei and Altman built their labs on scaling them, while LeCun, Hassabis and Ng have said scaling alone will not get there.
Yoshua Bengio