Reinforcement learning from human feedback (RLHF)

Reinforcement learning from human feedback is a way to shape a model's behavior using people's judgments. People compare pairs of model outputs and pick the better one; a second model learns to predict those preferences; and the main model is then trained by reinforcement learning to produce outputs that score well. It is how raw language models were turned into helpful assistants.