Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

TL;DR

StepFun’s open-weights flagship: a 196B-total / 11B-activated fine-grained MoE (288 routed experts + 1 shared, Top-8 per token) positioned as frontier-parity on reasoning, coding, and agentic-search benchmarks while running fast enough for real-time agents (100–300 tok/s typical, peak 350 tok/s for single-stream coding). Headline numbers: 74.4% SWE-bench Verified, 51.0% Terminal-Bench 2.0, 97.3 AIME 2025, 84.5 GAIA (no file), 73.7 BrowseComp-ZH with a context-manager. Architectural choices aimed at agentic workloads: 3-way Multi-Token Prediction (MTP-3) for high throughput, 3:1 Sliding-Window-Attention ratio for a 256K context window at reduced compute, fp8/bf16/GGUF (Q4_K_S, ~111.5GB) variants with day-0 vLLM and SGLang recipes and llama.cpp support for Mac Studio M4 Max / DGX-Spark / Ryzen AI Max+ class hardware. Companion tech report is arXiv:2602.10604 (199-author release).

Key claims

196B-parameter sparse MoE that activates ~11B per token via Top-8 routing over 288 fine-grained experts + 1 always-on shared expert, with a 45-layer / 4,096-hidden backbone and a separate 0.81B MTP head [§4.1, §4.2].
3-way Multi-Token Prediction (MTP-3) head predicts 4 tokens per forward pass and pushes single-stream throughput to 100–300 tok/s typical, 350 tok/s peak for coding [§2, §4.3].
3:1 Sliding-Window-Attention ratio (3 SWA layers per full-attention layer) supports a 256K context window with reduced compute relative to standard long-context attention [§2].
Estimated decoding cost at 128K context on Hopper is 1.0× (baseline) vs 6.0× DeepSeek V3.2, 18.9× Kimi K2 / GLM-4.7, 3.9× MiniMax M2.1, 1.2× MiMo-V2 Flash, using a methodology related to arXiv:2507.19427 [§3, decoding-cost table].
74.4% SWE-bench Verified, 51.0% Terminal-Bench 2.0, 86.4 LiveCodeBench-V6 — claimed strongest in the 11B-activated open class and competitive with DeepSeek V3.2 (73.1/46.4) and Kimi K2.5 (76.8/50.8) at far higher activated-parameter counts [§3, Coding table].
97.3 AIME 2025 / 98.4 HMMT-Feb / 94.0 HMMT-Nov / 85.4 IMOAnswerBench — at the top of the open table alongside GLM-4.7 (95.7 / 97.1 / 93.5 / 82.0) [§3, Reasoning table].
Agentic-search results: 84.5 GAIA (no file), 83.7 xbench-DeepSearch (2025.05), 65.3 ResearchRubrics, 66.9 BrowseComp-ZH (73.7 with the context manager) — the context manager resets the agent loop when effective context exceeds a threshold, vs the “discard-all” strategy used by Kimi K2.5 and DeepSeek-V3.2 [§3, Agent table].
Local-deployment-ready at the 120GB-unified-memory class: Q4_K_S GGUF is 111.5GB plus ~7GB runtime, runnable on Mac Studio, NVIDIA DGX Spark, or AMD Ryzen AI Max+ 395 via llama.cpp; full-precision serving on 8×Hopper via vLLM or SGLang with EAGLE-style speculative decoding wired into the recipe [§6.1–§6.4].
Parallel Thinking (arXiv:2601.05593) provides a further inference-time boost over the headline numbers, shown as shadowed bars in the performance figure [§3].

Method

Step 3.5 Flash is a sparse MoE transformer (45 layers, 4,096 hidden, 128,896 vocab) with 196.81B total parameters; per-token activation is ~11B via Top-8 routing over 288 fine-grained experts plus one always-active shared expert per layer. The attention stack uses a 3:1 ratio of sliding-window attention to full attention, giving a 256K context window at sub-quadratic effective cost. A dedicated MTP head — a sliding-window-attention + dense-FFN module (0.81B params) trained as 3-way Multi-Token Prediction — drafts 4 tokens per forward pass and is the source of the 100–300 tok/s decoding throughput. The release is paired with vLLM and SGLang recipes that wire MTP into speculative decoding (step3p5_mtp for vLLM, EAGLE-style with --enable-multi-layer-eagle for SGLang), and a llama.cpp Q4_K_S quantization path targeting unified-memory consumer hardware.

The release framing is explicitly agent-first: optimizations are justified against the constraint that “agents must reason fast”, and the model card emphasizes SWE-bench / Terminal-Bench / GAIA / BrowseComp as the load-bearing benchmarks rather than chat quality. A scalable RL framework “driving consistent self-improvement” is referenced as the source of the coding and agentic numbers, though the specifics are deferred to the tech report.

Results

Decoding-cost table (Hopper, 128K context) puts Step 3.5 Flash at 1.0× normalized cost vs. 6.0× for DeepSeek V3.2 (37B activated, MTP-1, EP32), 18.9× for Kimi K2 Thinking / K2.5 (32B activated, no MTP, EP32) and GLM-4.7 (32B activated, MTP-3, EP8), 3.9× for MiniMax M2.1 (10B activated, MTP-3, EP8), and 1.2× for MiMo-V2 Flash (15B activated, MTP-3, EP8). Coding: 74.4 SWE-bench Verified, 51.0 Terminal-Bench 2.0 — vs DeepSeek V3.2 73.1/46.4, Kimi K2.5 76.8/50.8, GLM-4.7 73.8/41.0. Reasoning: 97.3 AIME 2025, 98.4 HMMT 2025 Feb, 94.0 HMMT 2025 Nov, 85.4 IMOAnswerBench. Agentic: τ²-Bench 88.2; BrowseComp 51.6 / 69.0 with context manager; BrowseComp-ZH 66.9 / 73.7 with context manager; GAIA (no file) 84.5; xbench-DeepSearch (2025.05) 83.7, (2025.10) 56.3; ResearchRubrics 65.3. Recommended inference temperature is 0.6 / top_p 0.95 for chat, 1.0 / 0.95 for reasoning and agent scenarios.

Known limitations stated in the card: longer generation trajectories than Gemini 3.0 Pro to reach comparable quality (token efficiency gap); reduced stability under distribution shift in highly specialized domains and long-horizon multi-turn dialogues, with failure modes including repetitive reasoning, mixed-language outputs, and inconsistencies in time/identity awareness.

Why it’s interesting

A second 2026 datapoint after Kimi K2.5: Visual Agentic Intelligence in which an open Chinese MoE claims parity-or-better with closed frontier models on a benchmark table they ship themselves — but with a much smaller activated-parameter footprint (11B vs K2.5’s 32B), and explicitly designed around decoding cost rather than just capability. The 1.0× vs 18.9× decoding-cost gap against Kimi K2 / GLM-4.7 is the most striking number on the page: if it holds up in independent benchmarks, this is the cheapest serving point at this capability tier and the natural baseline for self-hosted agentic workloads. The MTP-3 + SWA(3:1) + fine-grained-MoE recipe is the same direction as DeepSeek V3 / Kimi K2 but pushed harder on sparsity (11B / 196B = 5.6% activation, vs ~5.5% for DeepSeek V3) and decoupled into a dedicated MTP module rather than additional nextN layers — a design choice worth tracking for in-house DiT and LLM serving stacks.