Skip to content
Bud's Wiki

Kimi K3 — 2.8T-parameter Kimi Delta Attention + Attention Residuals model with native vision and 1M context (platform quickstart)

TL;DR

Section titled “TL;DR”

Moonshot’s platform-docs quickstart for Kimi K3, positioned as Kimi’s most capable model to date: 2.8 trillion parameters, built on Kimi Delta Attention (the hybrid linear-attention primitive introduced in Kimi Linear) and Attention Residuals (the softmax-over-depth residual mechanism from the Kimi Team’s May 2026 tech report), with native visual understanding and a 1M-token context window. Thinking mode is always on and currently only supports reasoning_effort=max; API-side inference bakes in fixed sampling (temperature=1.0, top_p=0.95), automatic context caching with no cache-ID plumbing, dynamic tool loading via system messages without content, and an official-tools integration through a “Formula” endpoint (/tools + /fibers). The doc is a product-surface confirmation that two of the wiki’s most-tracked Kimi architecture bets — 3:1 KDA:MLA and depth-wise softmax residuals — are now the backbone of Moonshot’s flagship frontier release, not just research artifacts.

Key claims

Section titled “Key claims”
  • Kimi K3 has 2.8T total parameters and is Kimi’s most capable model to date; it is built on Kimi Delta Attention (a hybrid linear attention mechanism) and Attention Residuals, offers native visual understanding, and supports a 1M-token context window [§Introducing Kimi K3].
  • Thinking mode is always enabled on K3 and is configured via the top-level reasoning_effort field; the K2.x thinking parameter is not accepted, and reasoning_effort currently only supports the max level [§Thinking effort, §Important limits].
  • max_completion_tokens defaults to 131,072 and can be raised up to 1,048,576 [§Important limits].
  • Several sampling parameters are fixed server-side and must be omitted: temperature=1.0, top_p=0.95, n=1, presence_penalty=0, frequency_penalty=0 [§Important limits].
  • Streaming responses emit separate reasoning_content and final-answer content deltas [§Streaming].
  • Vision input requires content as an array of objects (not a serialized string); public image URLs are not supported — only base64 or a ms://<file-id> reference is accepted [§Vision input, §Important limits].
  • Multi-turn conversations and tool calls must echo the complete assistant message returned by the API back into the next request; keeping only content breaks the loop [§Thinking effort, §Important limits].
  • Context caching is automatic for regular model requests — no cache ID, TTL, or extra parameter is required; keeping a long prefix stable across requests is the only precondition for a cache hit [§1M context and automatic caching].
  • tool_choice="required" on the first turn forces the model to make at least one tool call; the client must return the complete assistant message plus one tool result per tool_call_id after execution [§Custom tools and tool_choice].
  • Tools can be loaded dynamically by placing a full tool definition inside a system message with no content; the tool becomes available from that message onward and the server does not retain it (the client must keep the definition in later history) [§Dynamic tool loading].
  • Official tools are integrated through Formula: the client fetches tool definitions from /tools, injects them into Chat Completions tools, and posts each returned tool_calls entry (name + arguments) to /fibers, then loops until the model produces a final answer [§Official tools].
  • Partial Mode lets the client seed an assistant-side text prefix via partial=True and have K3 continue from it, with the prefix re-prepended for display [§Partial Mode].
  • Structured output uses json_schema with strict: true to constrain message.content; only that field should be parsed, not reasoning_content [§Structured output].
  • K3 uses flat pay-as-you-go pricing with no tiering by context length; input is billed at separate cache-hit and cache-miss rates, and web search via the official-tools path is being updated and not recommended for production near-term [§FAQ, §Important limits].

Method

Section titled “Method”

The doc is a product-surface quickstart rather than a research paper, but it pins down the shape of the release. Architecturally K3 stacks two Moonshot-native components the wiki has already filed: Kimi Delta Attention (KDA) — the finer-grained-gated Gated-DeltaNet variant deployed as a 3:1 KDA:MLA hybrid in Kimi Linear (Kimi Linear: An Expressive, Efficient Attention Architecture) — and Attention Residuals (Attention Residuals (AttnRes / Block AttnRes) — Technical Report), the depth-wise-softmax residual mechanism that replaces uniform-weight PreNorm accumulation with per-layer softmax over prior layer outputs. Kimi Linear’s public checkpoint was 48B-total / 3B-active; K3 scales the same architectural template to 2.8T total parameters, keeps the 1M context that Kimi Linear used as its long-context claim, and adds native vision as a first-class input (base64 or platform-file URI, not public URLs).

Beyond architecture, the API surface encodes several inference-side commitments. Sampling is fixed at temperature=1.0, top_p=0.95, n=1, zero presence/frequency penalties — the client has no sampling knobs, only length and structure. Thinking is not optional (there is no non-thinking mode) and reasoning effort has one setting (max) — a departure from Kimi K2 Thinking / K2.5, where thinking was a toggleable mode. Context caching is implicit: any request with a stable long prefix automatically attempts a cache hit and is billed at a separate cache-hit input rate; there is no cache ID, TTL, or manual invalidation. Tool use exposes three surfaces — standard tools with tool_choice, dynamic tool loading via content-less system messages, and Moonshot’s own Formula integration (a /tools and /fibers endpoint pair that acts as a server-side tool registry + executor). The multi-turn contract is strict: the whole assistant message (including reasoning content and tool-call metadata) must round-trip verbatim, which mirrors the same requirement Kimi K2.5’s Agent Swarm (Kimi K2.5: Visual Agentic Intelligence) already imposed on orchestrator loops.

Results

Section titled “Results”

The quickstart is a docs page, not a benchmark drop — no eval numbers, model card, or comparative table is included. The confirmed release-level facts are the ones the doc states outright: 2.8T total parameters, native visual understanding, 1M-token context window, thinking always on, max as the only current reasoning-effort level, max_completion_tokens up to 1,048,576, and flat pay-as-you-go pricing with cache-hit / cache-miss input tiers (linked out to a separate pricing page). Web search via the Formula integration is explicitly flagged as being updated and not recommended for near-term production use.

Why it’s interesting

Section titled “Why it’s interesting”

K3 is the first public evidence that Kimi Delta Attention and Attention Residuals — both filed on the wiki as smaller-scale research artifacts earlier this year — have graduated to the flagship production stack, at ~58× the parameter count of the Kimi Linear checkpoint they were validated on. That closes a loop the Hybrid Linear Attention concept page flagged as open: whether the 3:1 KDA:MLA recipe scales past the 48B-A3B regime. It also compounds two Moonshot bets in a single model: Kimi Linear’s cluster-wide claim that “KV-cache size, not parameter count, is the right efficiency target” (Kimi Linear: An Expressive, Efficient Attention Architecture) now has to hold at 2.8T total parameters with a 1M context; and Attention Residuals’ claim to be a drop-in stability primitive with <4% training and <2% inference overhead (Attention Residuals (AttnRes / Block AttnRes) — Technical Report) has its first frontier-scale validation. Relative to Kimi K2.5: Visual Agentic Intelligence, the notable product-side shifts are (a) thinking is no longer a mode, it’s the default and the only mode, (b) sampling is locked down server-side, and (c) the tool-use surface has grown a Moonshot-native “Formula” registry alongside the standard OpenAI-compatible tools API. The absence of an Agent Swarm / PARL surface in the docs is conspicuous — either it’s still exposed only through kimi.com/agent-swarm and not the platform API, or the K3 recipe is a step back from K2.5’s parallel-orchestrator direction.

See also

Section titled “See also”