What changed

Alibaba Group Holding Ltd. today introduced Qwen4-Max, its newest and most capable large language model. The algorithm generates higher-quality output than its predecessor while costing less, thanks to a new architecture, Qwen4, and an upgraded attention mechanism.

What this means in practice

Qwen4-Max’s backbone comprises 2.4 trillion parameters, with 71 billion activated at any given time. It is supported by 328 billion N-gram parameters that act as a cache, reducing the need to recompute results. The model includes a feature that allows gradients to skip some layers during training, reducing errors and lowering costs. This feature, known as a residual, is implemented with four gradient transportation channels and adjustable gates to improve the training process.

The attention mechanism in Qwen4-Max features two customizations: Gated DeltaNet, which reduces hardware requirements for processing large prompts, and QSA, which skips unnecessary data to lower infrastructure costs.

Alibaba reported that Qwen4-Max outperformed its predecessor, Qwen3.8-Max, on an internal benchmark, achieving a higher score and using up to 35% fewer tokens for tasks. When tested on Terminal-Bench 4.0 coding challenges, the efficiency gains increased to 49%.

Limitations and availability

Qwen4-Max is available through Alibaba’s cloud platform. The company plans to open-source a collection of smaller models based on the same technologies in the future. It is also open-sourcing the Qwen-Live-Delivery-Plugin, which enables Q-4-Max to deliver initial drafts quickly and iteratively improve them based on user feedback. The supplied announcement does not specify the exact timeline for these open-source releases.

Source facts for Alibaba Launches Qwen4-Max With Training Attention Optimizations: Alibaba Group Holding Ltd. today introduced Qwen4-Max, its newest and most capable large language model. The algorithm not only generates higher-quality output than its predecessor but also costs less. Those improvements are the result of a new architecture, Qwen4, that made the LLM’s training run more efficient.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. siliconangle: Alibaba launches Qwen4-Max with training, attention optimizations