Models· 3 min
How does LoRA work?
LoRA fine-tunes a big model cheaply by learning small low-rank update matrices instead of touching all weights.
Key idea
Freeze the base model; add tiny trainable rank-r matrices A·B to each layer. Train those; keep the base intact.
Example
Fine-tune a 70B model on your product docs by training ~0.1% of parameters on a single GPU.
What to remember
- Drastically fewer parameters to train
- Adapters swap in/out at inference time
- Basis for QLoRA (quantized) and LoRA merges