Distributed ML Systems

Purpose: Train and serve models that need more than one GPU or one machine.

Outcomes

• Estimate memory and pick a parallelism strategy.
• Train with DDP, FSDP, tensor and pipeline parallelism.
• Serve large models across GPUs with KV-cache-aware routing.
• Run jobs on Ray, Kubernetes, or Slurm and control cost.

Audience

Engineers who have shipped single-GPU models (T3, T6) and need to scale them.