AMD has published a detailed guide explaining how occupancy is calculated on its MI355X GPU, focusing on the four resource limiters that define how many wavefronts can run simultaneously. The guide emphasizes that maximizing occupancy is not always the optimal goal, as demonstrated by performance tests showing the matrix core remains near peak utilization even when occupancy drops significantly. The MI355X GPU, based on the CDNA4 architecture, features 256 Compute Units, each with four SIMDs and a shared LDS memory pool. The guide aims to help developers better understand and optimize their kernels by breaking down the hardware's resource allocation and how it affects performance.
The guide explains that occupancy is determined by the first of four resources to be exhausted: VGPRs and SGPRs, LDS, or workgroup/barrier slots. Each SIMD has a private VGPR file of 512 entries per lane, shared between regular and accumulator registers, while LDS is a per-CU shared memory pool. AMD highlights that the per-SIMD vs. per-CU split is critical for accurate occupancy calculations, as it affects how resources are divided across the compute units. The guide also includes worked examples using MXFP8 GEMM tiles, which are key for performance in many applications.
The guide is structured into three parts, starting with an overview of the MI355X GPU architecture, followed by a deep dive into occupancy math and concluding with insights on achieving better performance at lower occupancy. AMD notes that understanding occupancy is essential for identifying and fixing kernels that are genuinely occupancy-limited, while also recognizing those that are not. The guide also touches on how Little’s Law and ILP (instruction-level parallelism) relate to occupancy, offering a nuanced approach to optimizing GPU performance.
Source: amd
