Kimi K2 Thinking — 1T MoE thinking model with native INT4 QAT and 200–300-step tool use

TL;DR

Kimi K2 Thinking is Moonshot’s open-weight “thinking” extension of the Kimi K2 base — the same 1T-total / 32B-active / 384-expert MoE backbone with MLA, now post-trained as an agent that interleaves chain-of-thought with up to 200–300 sequential tool calls. The headline release engineering is native INT4 quantization via QAT: all reported benchmark numbers are produced by the INT4 checkpoint, and the model is claimed to run at roughly 2× generation speed in low-latency mode with no accuracy loss versus the higher-precision variant. On agentic-search benchmarks under tool use the model claims state-of-the-art among non-frontier-closed models — HLE w/ tools 44.9, BrowseComp 60.2, SWE-Bench Multilingual 61.1 — released under a modified-MIT license on Hugging Face with vLLM / SGLang / KTransformers deployment paths.

Key claims

Architecture is identical to Kimi K2: 1.04T total / 32B activated MoE, 384 experts, 8 selected + 1 shared per token, 61 layers (1 dense), 64 attention heads, MLA, SwiGLU, 160K vocab, 256K context [§2 Model Summary].
The model is shipped as a native INT4 weight-only quantization on the MoE components, achieved via Quantization-Aware Training in post-training; all benchmark scores are reported under INT4 precision [§4 Native INT4 Quantization].
INT4 QAT delivers a claimed ~2× generation speed-up in low-latency mode without performance loss, addressing the failure mode where post-hoc quantization of thinking models with long decode lengths causes substantial regressions [§4].
Stable long-horizon agency: maintains coherent tool use across 200–300 consecutive tool invocations, vs. prior models that degrade after 30–50 steps; on agentic-search benchmarks evaluation cap is 300 steps × 24K reasoning tokens per step [§Key Features, §4.4].
Reasoning benchmarks (no tools): HLE text-only 23.9, AIME25 94.5, HMMT25 89.4, IMO-AnswerBench 78.6, GPQA 84.5 [§3 Evaluation].
Reasoning benchmarks (w/ tools): HLE 44.9 (heavy mode 51.0), AIME25 w/ Python 99.1 (heavy 100.0), HMMT25 w/ Python 95.1 [§3].
Agentic search (w/ tools): BrowseComp 60.2 (vs GPT-5 High 54.9, Claude Sonnet 4.5 Thinking 24.1, K2 0905 7.4), BrowseComp-ZH 62.3, Seal-0 56.3, FinSearchComp-T3 47.4, Frames 87.0 [§3].
Coding: SWE-bench Verified 71.3, SWE-bench Multilingual 61.1, Multi-SWE-bench 41.9, LiveCodeBenchV6 83.1, Terminal-Bench 47.1 (Terminus-2 / JSON parser) [§3].
Heavy Mode is an 8-trajectory parallel rollout + reflective aggregation strategy, producing the 51.0 HLE and 100.0 AIME25 numbers above; not a different model, just an inference-time aggregation scheme [§Footnotes].
Context-management strategy for long tool-use trajectories is simple: when accumulated input exceeds 256K, all previous tool outputs are hidden [§4.5].
HLE w/ tools sensitivity: blocking access to Hugging Face is required for a fair comparison — without that block, K2 Thinking reaches 51.3 on HLE, attributed to potential web data leakage [§4.6].
Deployment is supported on vLLM, SGLang, and KTransformers; checkpoints are stored in compressed-tensors format, convertible to FP8/BF16 via the vLLM-Project compressed-tensors repo [§5 Deployment].
License is Modified MIT on both code and weights [§7 License].

Method

K2 Thinking is not a from-scratch model — it is the K2 backbone (identical layer count, expert count, MLA, hidden dimensions) carried into a thinking-mode post-training pipeline. The release doesn’t publish a tech report at the time of filing, so the recipe is partially black-box. What the HF card does specify:

End-to-end training to interleave CoT with function calls as a single behavior, not as a CoT layer over a separately-finetuned tool-use model. Tool-calling schema is inherited from K2-Instruct.
Quantization-Aware Training applied in post-training — INT4 weight-only on the MoE components, leaving attention and dense layers presumably in higher precision (the card doesn’t say explicitly). The motivation is that thinking models emit long decode chains where post-hoc quantization errors compound; QAT keeps the long-decode loss flat.
Long-horizon stability is the main behavioral target. The card frames it as “200–300 consecutive tool invocations” with no quantitative breakdown of what fraction of calls are successful vs. how much of the gain over GPT-5/Claude Sonnet 4.5 comes from depth vs. per-call quality.

The evaluation methodology is unusually well-documented for a model card: temperature 1.0 (except SciCode at 0.0), 96K reasoning-token budget on no-tool reasoning benchmarks (128K on IMO/LiveCodeBench/OJ-Bench), avg@k over 32/16/8 runs on the volatility-prone benchmarks, o3-mini as HLE judge, max 120 steps × 48K reasoning tokens per step on HLE with tools and 300 steps × 24K tokens on agentic search. Coding numbers are averaged over 5 independent runs with an in-house SWE-agent-derived harness.

Results

The headline comparisons are vs GPT-5 (High), Claude Sonnet 4.5 (Thinking), K2 0905 (previous K2 instruct), DeepSeek-V3.2, and Grok-4. K2 Thinking wins on tool-use-heavy benchmarks: BrowseComp +5.3 vs GPT-5, +36.1 vs Claude Sonnet 4.5 Thinking; HLE w/ tools +3.2 vs GPT-5, +12.9 vs Claude Sonnet 4.5 Thinking; SWE-bench Multilingual +5.8 vs GPT-5, -6.9 vs Claude Sonnet 4.5. It loses on pure reasoning without tools (HLE text-only 23.9 vs GPT-5 26.3) and coding without tools is mixed (LiveCodeBenchV6 83.1 vs GPT-5 87.0).

The MMLU-Pro / MMLU-Redux / HealthBench numbers show GPT-5 and Claude Sonnet 4.5 still ahead on general-knowledge benchmarks (HealthBench: GPT-5 67.2 vs K2 Thinking 58.0). Longform Writing K2 Thinking 73.8 vs GPT-5 71.4 — a rare general-task lead.

Heavy Mode (parallel 8-trajectory + reflective aggregation) achieves 51.0 HLE, 100.0 AIME25, 100.0 HMMT25 — closing the AIME/HMMT gap with GPT-5 Pro but on a recipe that requires 8× the inference compute.

Why it’s interesting

K2 Thinking sits at the intersection of three concept threads this wiki tracks. For Open foundation-model releases it is the highest-stakes open agentic release since Kimi K2: Open Agentic Intelligence / Kimi K2: Open Agentic Intelligence — same 1T-MoE backbone, but now with thinking-mode behavior and an INT4-native deployment story that pushes the “open + competitive with frontier-closed” claim onto agentic-search benchmarks (BrowseComp 60.2 > GPT-5 High 54.9 is the most striking single datapoint). For LLM Inference Efficiency the INT4 QAT is a lossy weight-quantization lever applied to a frontier-scale thinking model — the first open release that combines QAT with a 1T MoE and a long-decode RL post-training stage, complementing 1-bit Bonsai 8B: End-to-End 1-bit Language Models from PrismML‘s 1-bit-Bonsai claim at a very different scale (8B end-to-end 1-bit vs 1T MoE-only INT4). For Tool-Use Agents the “stable 200–300-step tool use” claim raises MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling‘s interaction-depth axis to a frontier-scale datapoint, and the heavy-mode parallel-rollout aggregation echoes Kimi K2.5: Visual Agentic Intelligence‘s Agent Swarm (PARL) but as a simpler inference-time strategy without learned orchestration.

The QAT-as-default-deployment choice is the most consequential downstream-engineering signal: if INT4 QAT scales losslessly to 1T MoE thinking models, the open frontier’s effective serving cost drops by ~2× without requiring a new architecture.