C19
Agent Evaluation and Safety
5 lessons3 hoursPlannedFree
Outcome
Test agents systematically and guard against common failures.
Prerequisites
- Required: C16 LLM Agents and Tool Calling
- Recommended: C27 LLM Evaluation and LLM-as-a-Judge
Lessons
- L01Test harnesses for agents · coming soonExercise: Write 20 task-level tests.
- L02Failure taxonomy: wrong tool, bad arguments, loops · coming soonExercise: Label failures from a run.
- L03Prompt injection and tool misuse · coming soonExercise: Test an agent with adversarial inputs.
- L04Guardrails and human approval · coming soonExercise: Gate a destructive action behind approval.
- L05Regression testing across model versions · coming soonExercise: Compare two model versions on your suite.