Skip to content
Learn with AlakhSharma

C31

ML Clusters and Orchestration

6 lessons3 hoursPlannedFree

Outcome

Run distributed training and inference on real clusters with Ray, Kubernetes, or Slurm, and keep cost and utilization in check.

Prerequisites

Lessons

  1. L01GPU hardware for ML: memory, interconnects, NVLink, InfiniBand · coming soonExercise: Pick hardware for one training workload and one inference workload and justify it.
  2. L02Distributed Python with Ray: tasks, actors, Ray Train, Ray Serve · coming soonExercise: Parallelize a hyperparameter sweep with Ray.
  3. L03Kubernetes for ML: GPU scheduling, jobs, operators · coming soonExercise: Write a training Job spec with GPU requests and a checkpoint volume.
  4. L04Slurm and HPC-style training clusters · coming soonExercise: Write a multi-node sbatch script for a DDP job.
  5. L05Profiling distributed jobs: compute vs. communication · coming soonExercise: Find the communication bottleneck in a profiler trace.
  6. L06Cluster cost and utilization · coming soonExercise: Compute cost per training run and the cost of idle GPUs.