What changed
Alibaba Group Holding Ltd. today introduced Qwen4-Max, its newest and most capable large language model. The algorithm generates higher-quality output than its predecessor while costing less, thanks to a new architecture, Qwen4, and an upgraded attention mechanism.
What this means in practice
Qwen4-Max’s backbone comprises 2.4 trillion parameters, with 71 billion activated at any given time. It is supported by 328 billion N-gram parameters that act as a cache, reducing the need to recompute results. The model includes a feature that allows gradients to skip some layers during training, reducing errors and lowering costs. This feature, known as a residual, is implemented with four gradient transportation channels and adjustable gates to improve the training process.
The attention mechanism in Qwen4-Max features two customizations: Gated DeltaNet, which reduces hardware requirements for processing large prompts, and QSA, which skips unnecessary data to lower infrastructure costs.
Alibaba reported that Qwen4-Max outperformed its predecessor, Qwen3.8-Max, on an internal benchmark, achieving a higher score and using up to 35% fewer tokens for tasks. When tested on Terminal-Bench 4.0 coding challenges, the efficiency gains increased to 49%.
Limitations and availability
Qwen4-Max is available through Alibaba’s cloud platform. The company plans to open-source a collection of smaller models based on the same technologies in the future. It is also open-sourcing the Qwen-Live-Delivery-Plugin, which enables Q-4-Max to deliver initial drafts quickly and iteratively improve them based on user feedback. The supplied announcement does not specify the exact timeline for these open-source releases.
