Qwen3.6-35B-A3B Model Card

TL;DR

Qwen3.6-35B-A3B is the first open-weight checkpoint in the Qwen3.6 generation — a 35B-total / 3B-active MoE causal LM with an integrated vision encoder, released on Hugging Face under the same hybrid Gated DeltaNet + Gated Attention backbone the Qwen3.5 series introduced. The release is positioned as the open counterpart to the closed-API Qwen3.6-Plus and is foregrounded as an agentic-coding model: SWE-Bench Verified 73.4, Terminal-Bench 2.0 51.5, MCPMark 37.0, and the new preserve_thinking API parameter that retains historical thinking blocks across turns to reduce redundant reasoning in long agent loops. Native context is 262,144 tokens, extensible to ~1.01M with static YaRN; multi-token prediction (NEXTN) is supported for speculative decoding day-0 in vLLM / SGLang / KTransformers / Transformers. Of interest as the concrete open-weights datapoint behind the closed Qwen3.6-Plus launch and as a second open Qwen3.x VLM that bakes linear-attention layers into a frontier-quality post-trained model.

Key claims

35B total / 3B activated MoE causal LM with vision encoder, hidden dim 2048, 248,320 padded vocabulary, 40 layers [§Model Overview].
The 40 layers are organized as 10 × (3 × (Gated DeltaNet → MoE) + 1 × (Gated Attention → MoE)) — a 3:1 linear-attention to full-attention layer ratio identical in shape to the Qwen3.5 397B-A17B recipe [§Model Overview].
Gated DeltaNet layers use 32 linear-attention heads for V and 16 for QK at head dim 128; Gated Attention layers use 16 Q heads and 2 KV heads at head dim 256 with rotary embedding dim 64 (i.e. partial RoPE on a 256-dim head) [§Model Overview].
MoE width: 256 experts per layer with 8 routed + 1 shared per token (~3.5% of experts active), expert intermediate dim 512 [§Model Overview].
Native context length is 262,144 tokens; static YaRN with factor=4.0 extends to ~1,010,000 tokens, documented via both config.json rope_parameters overrides and per-framework CLI flags [§Processing Ultra-Long Texts].
Multi-token prediction is trained in via the NEXTN speculative algorithm and exposed day-0 in vLLM (--speculative-config) and SGLang (--speculative-algo NEXTN) [§SGLang, §vLLM].
The release adds a preserve_thinking API option that retains historical thinking blocks across turns, motivated as a way to “enhance decision consistency and, in many cases, reduce overall token consumption by minimizing redundant reasoning” in agentic scenarios [§Preserve Thinking].
Coding-agent benchmarks (Qwen-reported): SWE-bench Verified 73.4, SWE-bench Multilingual 67.2, SWE-bench Pro 49.5, Terminal-Bench 2.0 51.5, Claw-Eval Avg 68.7, NL2Repo 29.4, QwenWebBench 1397 [§Benchmark Results — Language, Coding Agent].
General-agent benchmarks (Qwen-reported): TAU3-Bench 67.2, DeepPlanning 25.9, MCPMark 37.0, MCP-Atlas 62.8, WideSearch 60.1 [§Benchmark Results — Language, General Agent].
Reasoning / STEM benchmarks (Qwen-reported): GPQA 86.0, HLE 21.4, LiveCodeBench v6 80.4, AIME26 92.7, IMOAnswerBench 78.9 [§Benchmark Results — Language, STEM & Reasoning].
Vision-language benchmarks (Qwen-reported): MMMU 81.7, MathVista-mini 86.4, RealWorldQA 85.3, OmniDocBench 1.5 89.9, RefCOCO-avg 92.0, VideoMMMU 83.7, MLVU 86.2 [§Benchmark Results — Vision Language].
SWE-bench reporting uses an internal agent scaffold (bash + file-edit tools) at temp=1.0 / top_p=0.95 / 200K context; SWE-bench Pro is run on a refined set where Qwen “corrects some problematic tasks” in the public benchmark [§Benchmark Results footnotes].
Qwen3.6 drops the /think and /nothink soft switch that Qwen3 exposed; thinking is on by default and disabled per-request via chat_template_kwargs.enable_thinking=False (or enable_thinking=False on Alibaba Cloud Model Studio) [§Instruct (or Non-Thinking) Mode].
Recommended sampling differs by mode: thinking-general temp=1.0, top_p=0.95, top_k=20, presence_penalty=1.5; thinking-precise-coding temp=0.6, presence_penalty=0.0; non-thinking temp=0.7, top_p=0.80, presence_penalty=1.5 [§Best Practices, §Quickstart].

Method

The released artifact is the post-trained Hugging Face Transformers checkpoint, paired with a model card that documents architecture, serving, and best practices but no training-recipe details. Architecturally it is the same hybrid-linear backbone introduced in the Qwen3.5 397B-A17B release applied at a much smaller scale: a 3:1 alternation of Gated DeltaNet (linear-attention, no KV cache) and Gated Attention (full-attention with partial RoPE) layers, with every layer’s FFN replaced by a 256-expert MoE routing 8 + 1 (shared) per token. At 35B total / 3B active and 40 layers, this is the agentic-coding-oriented sibling of the 397B-A17B native-multimodal flagship.

Concretely, head shape is asymmetric: linear-attention layers have 32 V heads / 16 QK heads at head dim 128; full-attention layers have 16 Q / 2 KV heads at head dim 256 with rotary applied only on the first 64 dims of each head. MoE expert intermediate dim is 512 — a “fine-grained” expert width consistent with the Qwen3 family’s scaling rule of more, narrower experts. The model is natively multimodal (Causal Language Model with Vision Encoder per the model card classification), with vision-language benchmarks reported alongside coding and STEM rather than in a separate VL variant.

Two serving mechanisms get foregrounded. Multi-token prediction (NEXTN, with num_speculative_tokens=2 recommended for vLLM and --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 for SGLang) is trained-in and day-0 supported. Static YaRN extends the natively-supported 262K context to ~1.01M via either config.json overrides on rope_parameters ({"rope_type":"yarn", "rope_theta":1e7, "partial_rotary_factor":0.25, "factor":4.0, "original_max_position_embeddings":262144, "mrope_interleaved":true, "mrope_section":[11,11,10]}) or per-framework CLI flags; the card explicitly warns that static YaRN with a constant factor “potentially impacting performance on shorter texts” and recommends only enabling when long-context use is required.

The new API behavior is preserve_thinking: by default only the latest user turn’s thinking blocks are retained (“interleaved thinking”), but with preserve_thinking=True historical thinking traces from prior turns are kept in context. Motivated specifically for agent scenarios where retaining the model’s prior reasoning improves decision consistency and KV-cache reuse.

Results

All numbers are vendor-self-reported on the model card; no independent reproduction.

SWE-Bench Pareto. SWE-Bench Verified 73.4, SWE-Bench Multilingual 67.2, SWE-Bench Pro 49.5 — for the 35B-total / 3B-active config, ahead of the prior Qwen3.5-35B-A3B (70.0 / 60.3 / 44.6) and below the 27B dense Qwen3.5-27B reference at 75.0 / 69.3 / 51.2. Beats Gemma4-26BA4B (17.4 / 17.3 / 13.8) by a wide margin at comparable active params.
Terminal-Bench 2.0. 51.5, ahead of Qwen3.5-27B (41.6), Gemma4-31B (42.9), and Qwen3.5-35B-A3B (40.5) on the card’s peer table.
Coding-agent suite. Claw-Eval Avg 68.7 (+3.3 vs Qwen3.5-35B-A3B), QwenClawBench 52.6, NL2Repo 29.4 (vs Qwen3.5-35B-A3B 20.5), QwenWebBench Elo 1397 (top of the table, ahead of Gemma4-31B at 1197 and Gemma4-26BA4B at 1178).
General agents. MCPMark 37.0 (highest in the peer table), DeepPlanning 25.9. TAU3-Bench (67.2) and VITA-Bench (35.6) — VITA-Bench below Qwen3.5-27B (41.8) and judged via claude-4-sonnet given the original claude-3.7-sonnet judger is unavailable.
STEM / reasoning. GPQA 86.0 (+0.5 vs Qwen3.5-27B), AIME26 92.7, HMMT Feb 26 83.6, IMOAnswerBench 78.9. HLE 21.4 (down from Qwen3.5-27B’s 24.3, expected for the smaller-active model).
Vision-language. MMMU 81.7, OmniDocBench 1.5 89.9, RealWorldQA 85.3, VideoMMMU 83.7. Mathvista-mini 86.4 (vs Claude-Sonnet-4.5 79.8). RefCOCO-avg 92.0 — a spatial-intelligence number not reported for Gemma4 or Claude-Sonnet-4.5 baselines on the table.
Spatial intelligence. RefCOCO 92.0, ODInW13 50.8 (up from Qwen3.5-35B-A3B 42.6), EmbSpatialBench 84.3, RefSpatialBench 64.3.
Sampling / output length. Recommended max output 32,768 tokens for typical use, 81,920 for math/programming competitions. Default API context 262,144.

Why it’s interesting

Two angles. First, this is the open-weights checkpoint behind the closed Qwen3.6-Plus: Towards Real World Agents launch — the Plus blog explicitly promised smaller open variants would follow, and 35B-A3B is the first of them. Comparing closed Plus (SWE-bench Verified ~78.8, Terminal-Bench 2.0 ~61.6) to open 35B-A3B (73.4 / 51.5) brackets the gap Alibaba is keeping between hosted and open at the Qwen3.6 generation. Second, it carries the Qwen3.5: Towards Native Multimodal Agents architectural bet down to a much smaller scale: at 3B active / 35B total with 3:1 Gated-DeltaNet:Gated-Attention layers, this is now a second size point for the hybrid-linear + ultra-sparse-MoE recipe, and the first open-weights instance where that recipe sits squarely on the Agentic Software Engineering Pareto. Two specific items deserve independent testing: preserve_thinking (the same flag exposed in Qwen3.6-Plus, now in open weights — concrete substrate to study how persisting CoT across turns interacts with multi-turn agent benchmarks), and the SWE-bench Pro footnote that Qwen “corrects some problematic tasks” in the public set — the open cluster on this concept page is converging on SWE-bench Pro as headline, and silently-refined evaluation sets are a reproducibility risk worth flagging.