What changed
AMD has introduced GPU-initiated SDMA (System DMA) copy engines in its RCCL and rocSHMEM software tools. This new capability allows GPUs to initiate data transfers using dedicated SDMA engines rather than compute units (CUs), improving bandwidth efficiency for intra-node data movement.
What this means in practice
The change enables GPU kernels to initiate data transfers while dedicated SDMA engines handle the actual data movement. This decouples data transfer from compute unit execution, freeing up GPU resources for computation while still saturating available bandwidth. Performance results from AMD Instinct™ MI350X hardware show that GPU-initiated SDMA achieves bandwidth comparable to or exceeding traditional implementations, using as few as a single compute unit instead of 64.
In RCCL, GPU-initiated SDMA (GIN-SDMA) is used for message sizes greater than 32MiB in AlltoAll operations and for messages 512KiB or larger in Allgather operations. For rocSHMEM, SDMA ON delivers significantly higher performance when using a single CU and one wavefront, compared to configurations without SDMA.
Limitations and availability
The supplied announcement does not specify exact hardware requirements for using GPU-initiated SDMA. The feature is available in RCCL and rocSHMEM, with performance improvements demonstrated on AMD Instinct™ MI350X hardware. The current rocSHMEM implementation does not use SDMA for messages smaller than 128KiB, and performance for these smaller messages may depend on the number of wavefronts used. The feature is part of AMD's ongoing efforts to optimize GPU-accelerated workloads, particularly for inference applications where communication lies on the critical path.
