C11
Transformers and Attention from Scratch
5 lessons4 hoursPlannedFree
Outcome
Implement scaled dot-product attention and a small Transformer.
Prerequisites
- Required: C9 Neural Network Fundamentals
Lessons
- L01Attention mechanism · coming soonExercise: Implement attention on toy sequences.
- L02Multi-head attention and masking · coming soonExercise: Add causal masking.
- L03Positional encodings · coming soonExercise: Compare sinusoidal and learned encodings.
- L04Transformer block · coming soonExercise: Build and train a character-level language model.
- L05Inference: KV cache basics · coming soonExercise: Measure generation speed with and without caching.