DeepSeek-V4.1-Flash — Pushing the Limits of KV Cache Compression

TL;DR

DeepSeek-V4.1-Flash is a 552B-total multimodal MoE with a new Causal Encoder-Decoder (CED) architecture and a suite of KV-cache compression techniques that push global KV footprint to ~890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash and 1/437 of DeepSeek-V1. The CED backbone activates only 8B parameters per token during prefill and 16B during decode, sharpening the input-heavy asymmetry typical of agentic workloads. Native multimodal support ships in one release (vision + text, DeepSeek-ViT trained from scratch) at 1M-token context, with weights on Hugging Face under MIT and Compressed Sparse Attention 2 (CSA2), SWA Bounded Replay, FP4 KV caching, Single-Pass mHC, Engram conditional memory (196B parameters, sparsely accessed via token-based lookup), and DSpark speculative decoding all rolled into one drop. On agentic benchmarks it matches or beats much larger closed frontier models — Terminal-Bench 2.1 90.6, DeepSWE v1.1 74.2, CyberGym 88.1, Agent’s Last Exam 31.8 — at 8B/16B activated params.

Key claims

The CED architecture is a 40-layer Transformer organized as a 20-layer causal encoder + 20-layer decoder; the decoder’s global KV cache is projected from the final encoder hidden states rather than derived per decoder layer, making prefill and decode use different activated-parameter counts (8B vs 16B) [§Introduction].
SWA Bounded Replay reconstructs missing sliding-window KV states by replaying only the most recent n_win tokens, avoiding persisting SWA KV to SSD and cutting persistent KV footprint to ~1/8 of V4-Flash [§Introduction].
Compressed Sparse Attention 2 (CSA2) assigns each attention layer one of three static modes — Full, Reindex, or Reuse — to share main KV and indexer K across layers and reuse Top-K sparse-attention indices; a Hierarchical Sparse Indexer in the decoder bounds deeper indexer cost independently of context length [§Introduction].
Combined with FP4 main KV caching (E2M1 with one E4M3 scale per 16 channels), the global KV cache is 890 bytes per token — ~4× less than V4-Flash, ~437× less than V1 [§Introduction, Fig. 1(b)].
The MoE layer uses 1 shared expert + 384 routed experts, activating 6 routed experts per token [§Introduction].
Additional architectural components: Single-Pass mHC (residual-stream mixing with a Mega-mHC kernel), Engram conditional memory (196B parameters, sparsely accessed via token-based lookup), DSpark speculative decoding (semi-autoregressive draft + confidence-scheduled verification) [§Introduction].
Vision encoder is DeepSeek-ViT trained from scratch with 2D-RoPE and 3×3 pixel-unshuffle downsampling; visual embeddings are processed jointly with text embeddings from the start of language-model pre-training [§Multimodal architecture].
Pre-training corpus is 45T multimodal tokens; sparse attention trained at 64K sequence length; context extended to 1M tokens at 34T tokens [§Pre-training].
Post-training uses the standard SFT → RL → on-policy distillation (OPD) recipe with no algorithmic modifications; all substantive changes are on the data-pipeline side (large-scale automated synthesis of agent tasks and environments with progressive scaling of data, tasks, and rollouts) [§Post-training].
Continuously controllable reasoning effort (integer 1–100) trades inference cost for accuracy at serve time [§Post-training].
Base benchmarks vs DeepSeek-V4-Flash-Base (284B) and V4-Pro-Base (1.6T): V4.1-Flash-Base matches or beats V4-Flash-Base on most axes and lands close to V4-Pro-Base on MMLU-Pro (74.1 vs 73.5), HumanEval (79.4 vs 76.8), BigCodeBench (60.6 vs 59.2) at 8B/16B active vs 49B active [Base Model table].
Agentic benchmarks at max reasoning effort vs Opus-5.0 / GPT-5.6 Sol / K3 / GLM-5.3 / DS-V4-Pro: Terminal-Bench 2.1 90.6 (top), DeepSWE v1.1 74.2 (top), CyberGym 88.1 (top), Agent’s Last Exam 31.8 (top), AutomationBench 54.8 (top) — a 16B-active model taking the top slot on multiple agentic benchmarks vs models 50–100× larger active [Instruct Model table].
Terminal-Bench 3.0 / 4.0 regress hard vs Opus-5.0 (30.0 / 31.2 vs 43.3 / 51.8) — the frontier-scaling gap opens up as the terminal harness gets harder, so the top slot is workload-conditional [Instruct Model table].
On DeepSWE v1.1, DSH Minimal (74.2) outperforms all seven other scaffolds tested (Claude Code 69.8, Codex 65.6, OpenCode 65.5, Pi 66.2, mini-SWE 74.2, DSH Standard 70.5, DSH PTC 67.6); on Terminal-Bench 2.1, DSH Minimal (90.6) also wins over Claude Code (88.0) and mini-SWE (90.3) [Performance across agent scaffolds table].

Method

The core methodological novelty is the Causal Encoder-Decoder (CED) architecture, which asymmetrizes prefill and decode compute in a way tailored to agentic workloads (input-heavy, output-lighter). The encoder is a 20-layer causal Transformer that runs during prefill; its final hidden states are then projected to produce the global KV cache used by the 20-layer decoder. This is different from standard decoder-only Transformers where every layer maintains its own KV cache, and different from encoder-decoder MT models where the encoder is bidirectional. The claim is that a causal encoder that shares its final-layer projection with the decoder gives 2× more decode-time active params (16B) than prefill-time (8B) without paying full-model cost during long-context ingestion.

The KV-cache compression stack layers three independent mechanisms. CSA2 statically assigns each attention layer to Full / Reindex / Reuse mode so that main KV and indexer K get shared across layers; a Hierarchical Sparse Indexer in the decoder restricts later indexing layers to a candidate pool built by the first Full-Mode layer, so indexing cost stops scaling with context length past a fixed depth. SWA Bounded Replay reconstructs missing sliding-window KV states from the most recent n_win tokens rather than persisting them to SSD, cutting the persistent-cache write path. FP4 main KV (E2M1 mantissa with one E4M3 scale per 16 channels) is a Blackwell-friendly narrow-precision format applied to the compressed cache. Together these hit 890 bytes/token — the “less than 1KB/token” figure from the Slack note.

The MoE is a straightforward extension of the DeepSeek-V3 recipe (fine-grained experts + shared expert + auxiliary-loss-free bias balancing per DeepSeek convention): 1 shared + 384 routed, top-6 routed per token. What’s new architecturally beyond the CED backbone is Engram — a 196B-parameter conditional memory bank sparsely accessed via token-based lookup, sitting outside the main MoE budget. If Engram behaves like recent parametric-memory designs (Qwen3.8-Flash-Next’s N-gram Embedding table, PEER-style memories) it’s a parameter-scaling axis orthogonal to MoE; the model card is thin on details. Single-Pass mHC is a “revised residual-stream mixing” with a dedicated Mega-mHC kernel, positioning it alongside the mHC (manifold-constrained hyper-connections) work in the 2512 archive.

Post-training explicitly disclaims algorithmic novelty: standard SFT → RL → on-policy distillation. The recipe change is that “all substantive changes lie instead in the data pipeline” — large-scale automated synthesis of agent tasks and environments with progressive scaling of data, tasks, and rollouts. Consistent with the “recipe-not-scale” framing that has been dominant in the open agentic-SWE cluster since Q3 2026.

Results

Base-model comparisons on the internal framework (equivalence threshold 0.3): V4.1-Flash-Base at 8B/16B active matches or beats V4-Flash-Base at 13B active on MMLU-Pro (74.1 vs 68.3), HumanEval (79.4 vs 69.5), BigCodeBench (60.6 vs 56.8), and closes most of the gap to V4-Pro-Base at 49B active on world-knowledge tasks (AGIEval 83.4 vs 84.4, C-Eval 92.1 vs 93.1). The one place V4-Pro-Base is still clearly ahead is SimpleQA-Verified (55.2 vs 42.3), which is the noisiest knowledge-recall axis in the table.

Instruct model at max reasoning effort (temperature 1.0, top_p 0.95, DeepSeek Harness Minimal scaffold with 1M context):

Agentic-coding: Terminal-Bench 2.1 90.6 (vs Opus-5.0 89.1, GPT-5.6 Sol 88.8, K3 88.3, GLM-5.3 88.2), DeepSWE v1.1 74.2 (vs Opus-5.0 74.0, GPT-5.6 Sol 73.0, K3 67.5). V4.1-Flash is Pareto-competitive with Opus-5.0 at these two axes despite being open weights and ~1/10 the activated-param count assumed for Opus.
Cybersecurity agents: CyberGym 88.1 (vs GPT-5.6 Sol 84.5, GLM-5.3 84.5), HLE w/ tools 63.9 (vs GPT-5.6 Sol 62.5). V4.1-Flash takes top slot on both.
Terminal-Bench 3.0 / 4.0 (harder terminal environments): 30.0 / 31.2 vs Opus-5.0’s 43.3 / 51.8 — the harness scaling gap opens up sharply. Opus-5.0’s 20.6-point lead on Terminal-Bench 4.0 is the largest gap in the whole table and suggests the CED small-active-param bet loses on the hardest long-horizon terminal tasks.
Reasoning-heavy: GPQA Diamond 90.9 (vs 93.4 Opus-5.0), HLE 36.8 / 39.1 text-only (vs 56.3 Opus-5.0), MathArena Apex 65.6 (tied with K3). V4.1-Flash sits mid-pack on pure-reasoning benchmarks where Opus-5.0 opens a large gap on HLE.

Scaffold sensitivity: on DeepSWE v1.1, DSH Minimal (72.6→74.2 depending on setting) and mini-SWE (74.2) both top all other scaffolds — the ordering flips between DSH Standard, DSH PTC, Claude Code, Codex, OpenCode, Pi below them. On Terminal-Bench 2.1 the same DSH Minimal wins (90.6). This is unusually explicit evidence that scaffold choice moves the number by 5–10 points at fixed model.

Why it’s interesting

This is the most architecturally-dense open-weights release the LLM Inference Efficiency concept has ingested to date. Four distinct KV/decode compression levers ship in one model: (a) architectural asymmetry between prefill and decode (CED, 8B/16B activated), (b) cross-layer KV sharing with static Full/Reindex/Reuse modes (CSA2), (c) SWA state reconstruction from recent tokens (SWA Bounded Replay), and (d) FP4 KV quantization. The “less than 1KB/token” endpoint that Dario flagged is the composed result — 890 bytes/token, 4× lower than V4-Flash and 437× lower than V1. Kimi K3’s Kimi K3 — Open Frontier Intelligence (2.8T MoE with KDA + AttnRes) KDA + Block Attention Residuals hit an analogous “reduce state from O(Ld) to O(Nd)” bound from a different direction (linear-attention-side); V4.1-Flash’s CED gets there from the softmax-attention side while keeping FlashAttention-style kernels.

Complements DeepSeek’s earlier V4-Flash-0731 refresh (DeepSeek-V4-Flash-0731 — post-training upgrade of V4-Flash, benchmarks now beating GLM-5.2) which was post-training-only at fixed architecture; V4.1-Flash is the architecture-change follow-up in the same Flash line. Confirms the pattern from DeepSeek-V4 collection release (Flash + Pro, up to 1.6T) (V4 collection release) that DeepSeek continues to ship the Flash tier as the vehicle for architectural experiments while V4-Pro (1.6T) stays on the stable recipe. Also extends the modality frontier — V4-Flash-Base and V4-Pro-Base were text-only in the collection card; V4.1-Flash-Base ships MMMU-Pro 56.5, DocVQA 95.6, CVBench 77.9 numbers, so this is DeepSeek’s first natively multimodal frontier open-weights base model.

Against Agentic Software Engineering‘s ongoing question of whether small-active-param models can hold the Pareto frontier: the Terminal-Bench 2.1 (90.6) and DeepSWE v1.1 (74.2) top-slot results at 16B active are direct evidence for the “recipe + architecture beats scale” side of the ledger, at least on the easier terminal harnesses. The Terminal-Bench 3.0/4.0 regression is direct evidence for the other side — at the hardest long-horizon terminal tasks, Opus-5.0 opens up a 20-point gap that the CED small-active bet can’t close. Both data points are useful.

Engram’s 196B-parameter conditional memory bank is the largest filed instance of parametric memory accessed via token-based lookup — Qwen3.8-Flash-Next’s 51B N-gram Embedding table was the previous high-water mark (Qwen3.8-Flash-Next — Hybrid Attention with QSA, Gated Residual, N-gram Embedding). If Engram behaves like the recent parametric-memory literature it’s a scaling axis largely orthogonal to MoE, and V4.1-Flash’s 552B total = 552B MoE backbone + 196B Engram + shared bookkeeping, not a monolithic 748B.