What changed

AMD's AITER introduces a new method for optimizing glue kernels, achieving significant speedups over the default Inductor configuration for specific models. The AITER Pin is keyed by kernel text and offers faster execution times for certain models. The speedup against Inductor's single default config is 1.388x on Model A and 2.846x on Model B. Against the open-list autotune, Model B is 1.15x faster.

What this means in practice

The AITER approach replaces the default Inductor body with another Triton function that has the same arguments, while leaving the config at the Inductor default. This method allows for more efficient execution of glue kernels, which are pointwise, reduction, and persistent-reduction kernels. The AITER is a library of Triton kernels for AMD Instinct GPUs. The AITER pin in the title refers to the Tuned config file, which is part of the AOTInductor package plus the runtime graph. The build uses one compile thread, and the log records how many pins and bodies were applied.

Limitations and availability

The AITER approach is currently available for specific models, with Model A having 143 glue kernels and Model B having 25. The glue autotune cache is not shared with GEMM and attention template timings, so a glue change does not affect GEMM selection. The GEMM binary remains the same, though its duration can still vary. The AITER method is not available for all models and is part of an experimental build. The supplied announcement does not specify the exact models or versions that are affected.

Source facts for AITER Pin Keyed by Kernel Text Offers Faster Glue Than Inductor Default: Software tools & optimizations AI/ML, Performance AI, Developers GEMM and attention kernels already have offline tuning paths. The kernels around them often do not. With autotune on, it times a candidate list and writes the winner into a cache keyed by a fusion number.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. amd: An AITER Pin Keyed by Kernel Text: Faster Glue Than the Inductor Default