Self-supervised learning
Self-supervised learning is training a model on raw data with no human labels, by hiding part of the data and asking the model to predict it. A language model that predicts the next word, or a vision model that fills in a masked patch of an image, is learning this way. Because the data supplies its own answers, the method can use almost unlimited amounts of text, images or video.
Why it comes up
LeCun has argued since 2016 that self-supervised learning is the bulk of intelligence (the cake, with supervised and reinforcement learning as icing and cherry), while David Silver argues that agents must also learn from their own experience.
Yann LeCun
David Silver