Intel and Hugging Face have demonstrated that the DFlash speculative decoding method significantly improves the performance of the Qwen3.6-35B-A3B model on AI PCs. Using Intel's Panther Lake laptop with OpenVINO, the method achieved an average speedup of 2.2x on HumanEval, 1.3x on MT-Bench, and 1.6x on GSM8K. The results show that DFlash can enhance both dense and MoE models, offering developers more flexibility in balancing performance and efficiency.
The Qwen3.6-35B-A3B model is an MoE model that uses a 35B network's capacity at a 3B decoding cost. It relies on a router to select only a few experts for each token, reducing memory and latency. However, accelerating such models with SD is challenging because verifying multiple tokens can require loading more experts than a single token decoding step. This makes MoE models a demanding test case, especially when combined with 4-bit weight quantization, as seen in the Qwen3.6-35B-A3B model.
The experiments used the Qwen/Qwen3.6-35B-A3B and z-lab/Qwen3.6-35B-A3B-DFlash models, both quantized to int4. The benchmarking included HumanEval, MT-Bench, and GSM8K datasets. The results showed that the DFlash pipeline improved throughput and acceptance length across all tests, with the highest gains on HumanEval and GSM8K. The performance improvements highlight the potential of DFlash for both dense and MoE models in the Qwen family.
Source: huggingface
