AMD released megakernel optimizations for large language model (LLM) inference on its Instinct™ MI355X GPUs on October 20, 2026, saying it delivers faster decode latency for interactive AI applications. It is the company's first software update in the AI/ML category since its previous release in 2025.
AMD reported 1.9x to 3.3x faster decode speed, measured on the GSM8K benchmark with single-request tests. That compares with the company's previous results using standard inference paths without the new optimizations.
The megakernel optimizations are built on AMD's MI355X GPU architecture and target interactive AI applications requiring real-time responsiveness. Availability begins with a preview release, initially for developers and researchers.
"Lowering time per output token is now critical for AI systems that feel conversational, responsive and useful," said Aditya Nandakumar, a software engineer at AMD. The optimizations combine megakernels, speculative decoding, low-precision data types and hardware-aware data placement.
The announcement follows AMD's previous release of the MI355X GPU in 2025. AMD said the optimizations are part of its ongoing efforts to improve AI inference performance across its GPU platforms.
AMD did not say how the optimizations will perform on other GPU models, and raised the open question of whether the techniques will be applicable to other architectures. The company said it plans to support additional model and engine combinations in the future.
Source: amd
