Kimi K3 opens its 2.8T-parameter weights: total size up 167% over K2, still behind Fable 5 on long-horizon coding
Synopsis
The Kimi team released Kimi K3, a mixture-of-experts model with 2.8 trillion total parameters and 104 billion activated per token, native vision and a one-million-token context window, together with the full model weights. The team reports that architectural, data and training changes together yield roughly 2.5× better overall scaling efficiency than Kimi K2. Across long-horizon coding, agentic, knowledge and vision evaluations, Kimi K3 still trails Claude Fable 5 and GPT-5.6 Sol overall but consistently beats the other open and proprietary models in the suite.
Interpretation
The open model ecosystem had largely stayed near the 1T-parameter class; Kimi K3 pushes the pre-trained foundation to the 3T class while scaling reinforcement learning, reasoning-effort levels and million-token long-horizon interaction alongside it. Earlier open models remained within or slightly above the 1T-class parameter regime, with reasoning and agentic RL methods applied to pre-trained foundations of similar scale. The architecture comparison lists Kimi K2 at 1.04T total and 32.6B activated parameters against Kimi K3 at 2.78T total and 104.2B activated, with layers rising from 61 to 93 and training context from 128K to 1M.
The model scales information flow along sequence length, depth and width at once, using hybrid attention, cross-layer attention residuals and a more sparsely routed expert layer to carry that scale. Kimi K2 used 61 MLA attention layers and SwiGLU activations; Kimi K3 switches to a hybrid of 69 KDA layers and 24 Gated MLA layers and adopts SiTU-GLU. Each block pairs three Kimi Delta Attention layers with one Gated MLA layer, and KDA extends the delta-rule recurrence with a channel-wise forget gate; Stable LatentMoE activates 16 of 896 routed experts per token, with RMSNorm and SiTU-GLU suppressing activation explosion.
Post-training consolidates nine expert models, three domains crossed with three reasoning-effort levels, into one unified model by distillation. Previously, domain- and effort-specialized policies were trained as separate experts. Multi-teacher on-policy distillation selects the matching teacher per domain and effort level, produces a clipped per-token reward signal, and plugs into a partial-rollout long-horizon RL framework.
Perspective
The report covers the model specifications, architecture, training and infrastructure described in the technical report, plus comparisons within the team's own evaluation suite; that suite spans long-horizon coding, agentic, knowledge, reasoning and vision tasks, and every comparison figure comes from it or from the external source it cites.
The team states that Kimi K3 still trails Claude Fable 5 and GPT-5.6 Sol overall and leads only the other open and proprietary models in the suite; Fable 5 results carry potential fallbacks and GPT-5.6 Sol results include potential cyberguards, conditions that affect comparability.
