What changed
AMD's AITER introduces a new method for optimizing glue kernels, achieving significant speedups over the default Inductor configuration for specific models. The AITER Pin is keyed by kernel text and offers faster execution times for certain models. The speedup against Inductor's single default config is 1.388x on Model A and 2.846x on Model B. Against the open-list autotune, Model B is 1.15x faster.
What this means in practice
The AITER approach replaces the default Inductor body with another Triton function that has the same arguments, while leaving the config at the Inductor default. This method allows for more efficient execution of glue kernels, which are pointwise, reduction, and persistent-reduction kernels. The AITER is a library of Triton kernels for AMD Instinct GPUs. The AITER pin in the title refers to the Tuned config file, which is part of the AOTInductor package plus the runtime graph. The build uses one compile thread, and the log records how many pins and bodies were applied.
Limitations and availability
The AITER approach is currently available for specific models, with Model A having 143 glue kernels and Model B having 25. The glue autotune cache is not shared with GEMM and attention template timings, so a glue change does not affect GEMM selection. The GEMM binary remains the same, though its duration can still vary. The AITER method is not available for all models and is part of an experimental build. The supplied announcement does not specify the exact models or versions that are affected.
