What changed
AMD announced a series of optimizations for training Mixture-of-Experts (MoE) models using its Primus framework, specifically tailored for the Instinct MI355X GPUs. These optimizations are part of the Primus ecosystem, which includes Primus-Turbo for high-performance operators and Primus-SaFE for stability and platform infrastructure. The enhancements focus on improving the efficiency of training large MoE models, which are becoming increasingly common in frontier-scale language models.
What this means in practice
The optimizations include low-precision expert GEMMs in FP8 and MXFP8, a fused MoE megakernel (MegaMoE), and support for training at scale on DeepSeek-V3 using up to 1024 GPUs. These changes are designed to address the bottlenecks in MoE training, such as expert GEMM throughput, all-to-all dispatch and combine operations, memory limitations, and host and launch overhead. The improvements are demonstrated through benchmarking results and are available for both Megatron-LM and JAX backends.
Limitations and availability
The optimizations are available for use with the Primus framework and are supported on AMD Instinct MI355X GPUs. The specific model configurations and measured throughput improvements are detailed in the official Primus documentation and GitHub repository. The supplied announcement does not specify the exact release date or availability of these features beyond the current development and testing phases.
![Source facts for AMD Introduces MoE Training Optimizations for Large Models: Primus/Primus-LM is a flexible, high-performance framework for large-scale foundation model training and inference on AMD GPUs. Both the Megatron-LM[b] and JAX backends of Primus are covered. For configuration examples and further details, see the official Primus documentation and GitHub repository.](/images/articles/amd-introduces-moe-training-optimizations-for-large-models-editorial-facts.png)
