Moonshot AI announced the release of its Kimi K3 model on July 16, 2026, with full open-source weights expected by July 27. At 2.8 trillion parameters, Kimi K3 is the first open-source model to reach the 3-trillion-parameter class. The model is designed for researchers and MLOps practitioners, offering advanced capabilities in text and vision processing. Kimi K3 introduces several architectural innovations, including Kimi Delta Attention (KDA), Attention Residuals (AttnRes), and Stable LatentMoE, which collectively improve scaling efficiency by approximately 2.5x compared to its predecessor, K2. The model also employs MXFP4 quantization for weights and MXFP8 for activations, enhancing deployment efficiency.
Kimi K3's architecture features a Mixture-of-Experts (MoE) transformer with native vision support, enabling efficient handling of large context windows up to 1 million tokens. The model uses a combination of 896 experts, with 16 active per token, and incorporates Quantile Balancing for load management and soft dropping for overflow tokens. Additionally, the model includes per-head Muon optimizer, Sigmoid Tanh Unit (SiTU) activation function, and Gated MLA for memory-efficient KV-cache management. These components collectively contribute to the model's performance and scalability, making it suitable for both API serving and self-hosting.
The release of Kimi K3 comes with a focus on open-source accessibility, with weights available by July 27, 2026. Moonshot AI emphasized the economic viability of the model, noting that MXFP4 quantization reduces storage requirements to approximately 1.4 TB, compared to 5.6 TB for FP16. This makes self-hosting feasible for organizations with multi-node GPU clusters. The model's performance is benchmarked against other models, with notable results in coding and general intelligence tasks. Moonshot AI also acknowledged known limitations, including thinking history sensitivity, excessive proactiveness, and a UX gap compared to other models.
Source: huggingface
