Concept· 3 min
What is RLHF?
Reinforcement Learning from Human Feedback aligns a model to human preferences after pre-training.
Key idea
Humans rank pairs of model outputs; a reward model learns those preferences; the LLM is then fine-tuned to maximize that reward.
Example
ChatGPT used RLHF to become helpful, harmless and honest instead of just a raw next-token predictor.
What to remember
- Turns a raw LM into a well-behaved assistant
- Modern variants: DPO, GRPO, RLAIF
- Sensitive to labeler quality and bias