What changed
AMD has announced the quantization of Qwen3.8-Flash-Next using UINT4 for expert weights, paired with an FP8 n-gram table. This reduces the total checkpoint storage from ~360 GB to ~130 GB. The quantization process, based on activation-aware weight quantization (AWQ), is designed to optimize memory usage on Ryzen AI Halo devices with 128 GB of unified memory.
What this means in practice
To run Qwen3.8-Flash-Next on a Ryzen AI Halo, users must account for runtime buffers and the key-value (KV) cache. The quantized model, with a total checkpoint tensor storage of ~130 GB, leaves limited headroom for the operating system, KV cache, and runtime buffers. AMD suggests further reductions in model storage through additional quantization or storage offloading.
The evaluation includes comparisons of different n-gram lookup methods, such as zero lookup, which removes the n-gram table entirely, and a calibrated per-token table that replaces the FP8 n-gram table. These configurations are tested for their impact on checkpoint storage, accuracy, and response length.
Limitations and availability
The calibrated per-token table, which replaces the FP8 n-gram table, has been run on Ryzen AI Halo. However, the supplied announcement does not specify the exact availability of these configurations beyond the tested environment. Actual usable memory depends on system reservations and memory allocation settings. The results suggest that checkpoint storage can be reduced from ~130 GB to ~80 GB, with accuracy changes depending on the workload.
The experiments focus on the impact of removing or replacing the n-gram lookup in Qwen3.8-Flash-Next across different quantization and thinking configurations. The AWQ UINT4 g128 expert weights remain fixed across these configurations, and the accuracy under different thinking settings is compared using AMD Instinct™ systems with MI355X and tensor parallelism (TP) size 1.
![Source facts for AMD Quantizes Qwen3.8-Flash-Next for Ryzen AI Halo: For this evaluation, AMD Quark [5] quantizes expert weights to 4-bit unsigned integers (UINT4) using activation-aware weight quantization (AWQ). Actual usable memory depends on system reservations and memory allocation settings. Further quantization or storage offloading offers other ways to reduce memory pressure.](/images/articles/amd-quantizes-qwen38-flash-next-for-ryzen-ai-halo-editorial-facts.png)
