Models· 3 min
What is a Transformer?
The Transformer is the neural-network architecture behind almost every modern LLM — powered by self-attention.
Key idea
Instead of reading text left-to-right like an RNN, a Transformer looks at all tokens at once and learns which ones to 'attend' to.
Example
GPT, Claude, Gemini, Llama and Mistral are all Transformer variants — same core idea, different scale and training.
What to remember
- Self-attention scales in parallel — perfect for GPUs
- Introduced by Google in 'Attention Is All You Need' (2017)
- Decoder-only Transformers dominate text generation today