C14
Policy Gradients and PPO
5 lessons4 hoursPlannedFree
Outcome
Implement REINFORCE and PPO and tune them on a benchmark.
Prerequisites
- Required: C13 MDPs, Dynamic Programming, and TD Learning
- Required: C9 Neural Network Fundamentals
Lessons
- L01Policy gradient theorem · coming soonExercise: Implement REINFORCE.
- L02Baselines and advantage estimation · coming soonExercise: Measure variance reduction.
- L03Actor-critic · coming soonExercise: Add a value head.
- L04PPO: clipping and the objective · coming soonExercise: Implement PPO on CartPole.
- L05Tuning and reporting results · coming soonExercise: Report mean and variance over seeds.