Moonshot AI releases Kimi K3, the largest open-weight model to date

Source: VentureBeat — 2026-07-16

Summary

Beijing-based, Alibaba-backed Moonshot AI announced Kimi K3, a 2.8-trillion-parameter Mixture-of-Experts model that the company is calling the largest open-weight model released to date. K3 ships with native visual understanding, a 1-million-token context window, and an always-on "thinking mode," built on two new architecture pieces — Kimi Delta Attention and Attention Residuals. The model debuted at roughly #3 on the Artificial Analysis Intelligence Index when grouped by model family, putting it in the same tier as Claude Opus 4.8 and GPT-5.5-class systems. Full weights are scheduled to follow on July 27, timed just ahead of the World Artificial Intelligence Conference in Shanghai.

Key Takeaways

• K3 is a sparse MoE model using a "Stable LatentMoE" design that activates only 16 of 896 experts per token, keeping inference cheap relative to its 2.8T total parameter count.
• Kimi Delta Attention (KDA), a hybrid linear attention mechanism, is claimed to deliver up to 6.3x faster decoding at million-token context lengths.
• Attention Residuals (AttnRes) replaces standard residual connections with depth-wise selective retrieval of representations rather than uniform accumulation, which Moonshot says produces more consistent scaling gains.
• The model natively accepts text, image, and video input and supports a 1,048,576-token context window.
• On the Artificial Analysis Intelligence Index, K3 scored close behind Claude Fable 5 and GPT-5.6 Sol, landing around #3-4 depending on whether leaderboard configurations are grouped by model family.

Reel Script

Hook A Chinese startup just shipped the biggest open-weight AI model on Earth — 2.8 trillion parameters — and it's already trading blows with Anthropic and OpenAI's best. If you thought open-source was still a generation behind, this week says otherwise.

Core Concept Here's the trick: nobody runs all 2.8 trillion parameters at once. Kimi K3 is a Mixture-of-Experts model, which means it's really thousands of smaller specialist networks stitched together, and for any given word it only wakes up a tiny slice of them — 16 out of 896 experts, to be exact. Think of it like a massive hospital where a patient only sees the two or three specialists relevant to their case, not every doctor on staff. That's how you get a model with GPT-scale total knowledge without GPT-scale inference cost. Moonshot didn't stop at MoE, though — they built two new internal architecture pieces to make this thing actually usable at huge context lengths, which is where most models start to choke.

Hands-On The first piece is called Kimi Delta Attention, or KDA — a hybrid linear attention mechanism, and the headline number is up to 6.3 times faster decoding once you're operating in million-token context windows. That matters because K3 ships with a 1,048,576-token context window and native support for text, images, and video — so it's meant to read entire codebases or hours of video and still respond quickly. The second piece, Attention Residuals, tweaks something deep in how transformers pass information between layers: instead of every layer just piling its output onto the last one, the model selectively pulls representations from earlier in the stack. Moonshot says that produces steadier gains as you scale the model up, rather than diminishing returns. Put together, this got K3 to roughly third place on the Artificial Analysis Intelligence Index, grouped by model family — within range of Claude Opus 4.8 and GPT-5.5-class systems. Full weights land July 27, right before the World AI Conference in Shanghai, so this is a preview, not the final drop.

Takeaway This is the clearest signal yet that the gap between open-weight and closed frontier models is closing faster than most people expected, and it's happening on someone else's terms. Worth watching what K3 actually costs to self-host once those weights are public. Full weights and technical report drop July 27 — that's the number to watch.