What changed
AMD's Triton backend now includes warp-pipelining, a user-directed scheduling technique that allows two wave groups to execute one stage apart. This method helps overlap matrix operations with data preparation, improving efficiency in compute-bound GEMM tasks. The feature is designed to work on AMD Instinct accelerators, including MI300X, MI350, and MI450.
What this means in practice
The new warp-pipelining technique enables kernel authors to describe schedules that the compiler can preserve, potentially achieving performance comparable to hand-tuned kernels. On the MI355X FP16 GEMM, an 8-wave warp-pipelined kernel reached 1,437 TFLOP/s, compared to 1,425 TFLOP/s for a hand-tuned 4-wave kernel. The difference is interpreted as parity, with neither kernel spilling. The technique uses stage annotations and barriers to maintain the offset between wave groups, allowing one group to issue matrix work while the other handles data preparation.
Limitations and availability
The feature is available for compute-bound GEMM shapes with sufficient matrix work to cover data preparation. It targets AMD Instinct accelerators, including MI300X, MI350, and MI450. The examples provided in the blog include public kernel examples and commands for exploring the technique. The implementation uses stage annotations and barriers to maintain the offset between wave groups, with the compiler adding completion waits where necessary to ensure memory ordering. The reference launch uses num_warps=8 and waves_per_eu=2, with the latter being a compiler occupancy hint.
