GPU Training Optimization Toolkit: Taming the Bottlenecks Behind Large-Scale AI

The immersive experiences the holodeck envisions—generative environments, real-time multimodal AI, and photorealistic avatars —rely on increasingly large neural networks, and training those networks at scale is fundamentally a problem of GPU efficiency. This talk presents a practical, vendor-neutral framework for understanding where GPU training time is spent and how to reclaim it. We begin with the underlying hardware, examining how CPUs, high-bandwidth memory, NVLink, and inter-node networks form a performance hierarchy that every training workload must respect. We then characterize performance problems into three bottlenecks. Compute-bound workloads are constrained by raw arithmetic throughput, address them by performing that computation more efficiently and in larger, fewer units of work. Memory-bound workloads are constrained by either capacity or bandwidth, and are resolved by reducing the data footprint or the volume of data in motion. Network-bound workloads, which often dominate at scale, leave GPUs idle during inter-device communication. Overlapping, tuning, or reducing that communication mitigates these types of issues. Throughout, we emphasize disciplined diagnosis over guesswork: identifying the true bottleneck before applying a remedy. Attendees will leave with a structured mental model for reasoning about training performance — one that applies equally to scaling today's foundation models and to enabling the next generation of immersive AI.