Kimi K3, and What We Can Still Learn From the Pelican Benchmark

Source: Simon Willison's Weblog — 2026-07-16

Summary

Moonshot AI released Kimi K3, a 2.8-trillion-parameter mixture-of-experts model billed as the largest open-weight model released to date, with open weights promised by July 27, 2026. It activates just 16 of 896 experts per token and supports a 1-million-token context window. Simon Willison ran it through his now-familiar "draw a pelican riding a bicycle" SVG test on launch day, using the exercise as a jumping-off point to argue that a quick single-prompt art test says little about the agentic tool-calling ability that actually matters for a model like this.

Key Takeaways

• Kimi K3 has 2.8T total parameters but only about 1.8% of experts (16 of 896) active per token, keeping inference cost far below what the headline parameter count implies.
• Moonshot's Kimi Delta Attention, a hybrid linear-attention mechanism, reportedly delivers up to 6.3x faster decoding, a big part of why K3 stays usable across the full 1M-token context (scoring 90.4 on a 1M-token eval with no context management).
• Pricing lands close to Claude Sonnet tier: $3/M input tokens, $15/M output tokens, $0.30/M cache-hit input tokens — a notable jump over Kimi K2.6.
• The pelican-on-a-bicycle SVG prompt cost about 25 cents to run on K3, burning 13,241 reasoning tokens to produce 3,417 tokens of output; the model produced a solid image and wrote accurate alt text describing its own drawing.
• Self-reported benchmarks put K3 ahead of Claude Opus 4.8 and GPT-5.5 high on several measures, though behind Claude Fable 5 and GPT-5.6 Sol; Moonshot calls it the first "open 3T-class model," surpassing DeepSeek's 1.6T V4 Pro.
• Willison's core point: the pelican test is a fun, cheap sanity check for spatial reasoning and instruction-following, but it says nothing about how a model performs at agentic tool calling across long conversations — which is where models like K3 actually earn their keep.

Reel Script

Hook A 2.8-trillion-parameter model just drew a pelican riding a bicycle — and it cost 25 cents and 13,000 reasoning tokens to do it. Here's why that silly little test still matters, and why it's also becoming useless.

Core Concept Kimi K3 is Moonshot AI's new open-weight model: 2.8 trillion total parameters, but it only activates 16 of 896 "experts" per token — think of a huge hospital where only the specific specialists your case needs get called in, not the whole staff. That's why a model this big can still run at reasonable speed. It also holds a full 1-million-token context window, helped by a hybrid attention trick called Kimi Delta Attention that speeds up decoding roughly 6x. Simon Willison has run nearly every major model release through the same test since 2024: ask it to draw an SVG of a pelican riding a bicycle. It's a stress test for spatial reasoning, instruction following, and vector-graphics logic, all in one cheap, comparable prompt.

Hands-On The benchmark methodology is simple by design: one prompt, "generate an SVG of a pelican riding a bicycle," identical wording every time, run against every model Willison reviews. It's not about photorealism — it's a quick, apples-to-apples signal of whether a model can reason about proportion, position, and object relationships with zero tools and zero iteration. For K3, the run cost roughly 25 cents: the model burned 13,241 reasoning tokens before producing 3,417 tokens of final SVG output, then correctly described its own image in alt text, a decent vision-language sanity check. But Willison is explicit that this tells you almost nothing about the thing that actually matters for a model like K3: multi-step agentic tool calling across long conversations, which no single-shot art prompt can measure.

Takeaway The pelican test earns its keep as a five-second gut check, not a leaderboard. If you're evaluating K3 or any frontier model for real work, treat single-prompt benchmarks as a floor, not a verdict — go test it on the actual multi-step, tool-using workflow you plan to run.

Discussion

(No questions yet — ask follow-ups via a Claude Code chat session on this repo; answers get appended here.)