What changed
AMD has released a new profiling guide tailored for PyTorch applications running on AMD GPUs. This guide is part of a series aimed at optimizing GPU-accelerated applications, with a specific focus on AI workloads. The guide outlines a systematic workflow for profiling and optimizing PyTorch-based AI models, including the use of tools like rocprof-sys, rocprofv3, and rocprof-compute.
What this means in practice
The guide teaches users how to annotate PyTorch code with ROCTx markers to track performance and then use these markers to narrow down timelines for analysis. It also explains how to use hardware counters with rocprof-compute to evaluate different optimization techniques, such as PyTorch-level fusion, torch.compile, and custom Triton kernels. The guide emphasizes the importance of separating warm-up phases from measurement to ensure accurate profiling results.
Limitations and availability
The guide is available as part of AMD's ROCm software stack and is intended for users who are already familiar with running GPU-accelerated PyTorch code. It requires specific versions of ROCm, PyTorch, and Triton, including ROCm 7.13, PyTorch 2.11, and Triton 3.6. The guide is designed for AMD Instinct™ MI300X and MI355X hardware. The supplied announcement does not specify the exact performance improvements or benchmarks achieved with the optimizations described in the guide.
