DFlash 2: Keep Drafting Parallel

TL;DR

DFlash 2 is Inco AI’s follow-up to DFlash (arXiv 2602.06036) — a parallel diffusion-style drafter for speculative decoding whose original release (January 2026) reached >3.5M Hugging Face downloads and drives NVIDIA/Google/CoreWeave production stacks. DFlash 2 keeps the one-forward-pass parallel-drafting design but recovers two known gaps: (a) DFlash’s independent per-position picks are individually plausible but jointly incoherent, so blocks get cut short at verification; (b) later positions in the drafted block degrade (“suffix decay”). The fix is a 2M-param pairwise path selector over top-16 candidates and a 16.5M-param dynamic depthwise convolution with a two-tap kernel reaching one position back, inserted around attention/FFN sublayers. Together they add ~1.3% to draft–verify cycle latency and lift acceptance length by 21% over DFlash and 9% over DSpark, translating to 2.7–3.4× (Qwen3.8-27B) and 3.1–4.6× (Muse Glimmer) throughput over autoregressive decoding at batch 1.

Key claims

DFlash’s per-position top-1 is right 85.4% of the time at position 0 but the true token is in the top-16 candidates 99.5% of the time; an oracle path selector over top-16 lifts acceptance length from 4.27 → 6.79, exposing “pure selection headroom” that doesn’t need better prediction, only better routing [§The Right Tokens Are Already There, Table 1].
The path selector scores every adjacent (predecessor, current) candidate pair in parallel via a low-rank bilinear attention over 256-D embeddings gated by context, then does a single final greedy/sampled/rejection-sampled walk over precomputed scores — no additional backbone or LM-head pass [§A Lightweight Path Selector].
With only 2.0M added params and +0.6% latency, the path selector improves DFlash by +0.34 tokens (T=0) and +0.47 (T=1), beating a DSpark-style Markov correction that adds 77.8M params and +9.6% latency [Table 2].
Suffix decay is a local problem: 3/5/15-layer DFlash models are nearly identical at position 0 but fan apart down the block; 15-layer (3× params) reaches 78.7% at position 6 vs 5-layer’s 72.9%, but adds 15.2% cycle latency uniformly [Table under §Suffix Decay Is a Local Problem].
DFlash attention diverts progressively away from within-block work as layers deepen: within-block share falls from ~30% in Layer 1 to ~8% in Layer 5, concentrated in a shrinking handful of heads — motivating a dedicated within-block operator that doesn’t spend attention capacity [Figure 3, attention-head table].
A two-tap dynamic depthwise convolution (current position + one predecessor) inserted before/after each attention and FFN sublayer, with a learned base kernel plus a hidden-state-conditioned per-16-channel correction, recovers most of the 15-layer suffix accuracy with only 16.5M added params (~3%) and +0.7% latency; within-block attention in Layers 4–5 falls 9.4% → 0.5%, consistent with the conv absorbing local work [§A Lightweight Local Convolution].
Combined, DFlash 2 averages 5.97 tokens/pass vs DFlash 4.92 and DSpark 5.49 across GSM8K/MATH-500/HumanEval/MBPP/MT-Bench — a +21% gain over DFlash and +9% over DSpark at +1.3% total cycle-latency overhead [Table 3].
On MATH-500, DFlash 2 holds ~86% acceptance uniformly across positions 0–14, while MTP/DFlash/DSpark all end the 15-position block 6–9 points lower [Figure 5].
For Qwen3.8-27B, DFlash 2 averages 4.80 tokens/pass vs the model’s native MTP at 4.28 and a community DSpark drafter at 3.62; for Meta’s Muse Glimmer, DFlash 2 averages 5.70 vs the official DFlash drafter’s 4.44 [§Two Drafters, Out Today, Tables 4 & 5].
Reported end-to-end speedup over autoregressive at batch 1: 2.7–3.4× on Qwen3.8-27B and 3.1–4.6× on Muse Glimmer served under SGLang [§Two Drafters, Out Today].

Method

DFlash 2 keeps DFlash’s core design — a small transformer drafter predicting every position in the next block in parallel from the target model’s hidden states — and adds two independent, drop-in modules that don’t touch verification or the LM head.

Path selector. For each of the K drafted positions the drafter retains the top-16 candidate tokens instead of committing to the top-1. Every adjacent pair (candidate_i at position p−1, candidate_j at position p) is scored in one parallel shot: score(i,j) = logit_j + <E1(candidate_j), gate(context) ⊙ E2(candidate_i)>, where E1/E2 project each candidate token to a 256-D embedding and gate() is a per-context modulation vector. The full 16×16×K score tensor is precomputed once; the final decode is a sequential walk starting from the last verified token (greedy = argmax next-candidate, sampling = draw from the same scores, rejection sampling = correct back to the target distribution).

Dynamic depthwise convolution. Around each attention and FFN sublayer in the drafter, a two-tap depthwise conv slides across positions: y[p] = a * x[p] + b * x[p−1], where a and b are per-channel and per-position combinations of a learned base kernel and a small correction computed from the current hidden state (every 16 channels share one correction). Position 0 reads the last verified token’s representation; positions ≥1 read their in-block predecessor. Block-local and stateless — no changes to attention masks, KV cache, or LM head. Explicitly follows the Canon Layers (Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers) / Dynamic Short Convolutions / Convolution-for-LLMs line.

Both modules are trained end-to-end with the standard DFlash next-token loss, then compiled into the SGLang / vLLM / TensorRT-LLM / llama.cpp drafter interface.

Results

Average acceptance length across 5 benchmarks: MTP 4.54 / DFlash 4.92 / DSpark 5.49 / DFlash 2 5.97. DFlash 2 leads on every benchmark and is +1.05 tokens (21%) over DFlash, +0.48 over DSpark [Table 3].
Path selector alone (Table 2): +0.34 tokens at T=0 and +0.47 at T=1 over base DFlash, with 2.0M params (~+0.6% latency). DSpark’s Markov correction reaches +0.22 (T=0) / +0.30 (T=1) at 77.8M params and +9.6% latency — DFlash 2’s selector is roughly 40× fewer params and 16× lower overhead for a strictly larger gain.
Convolution alone: 5-layer DFlash + conv (3% more params, +0.7% latency) matches or exceeds 15-layer DFlash (3× params, +15.2% latency) at every position — most importantly the tail, where 5-layer + conv reaches 77.6% at position 6 vs 15-layer’s 78.7% and base 5-layer’s 72.9%.
MATH-500 position curve: DFlash 2 stays flat at ~85–86% across positions 0–14; base DFlash decays from 88.4% → 77.5%, DSpark from 87.2% → 79.9%, MTP from 84.6% → 77.9%.
Released drafters (Qwen3.8-27B, Muse Glimmer): DFlash 2 averages a full token ahead of DSpark on both models, beats each model’s official co-released drafter (MTP for Qwen, DFlash for Muse Glimmer), and delivers 2.7–3.4× / 3.1–4.6× throughput over autoregressive at batch 1 on SGLang.

Why it’s interesting

Directly extends the DFlash speculative-decoding line that the wiki has been tracking through several angles: Speculative Decoding: Performance or Illusion?‘s finding that verification dominates 42–95% of SD time at batch 1 makes each extra accepted-token-per-pass the most valuable lever in this regime, and Speculative Speculative Decoding attacks the same “draft-verify serialization” bottleneck by parallelizing across hardware — DFlash 2 attacks the complementary seam of raising per-pass acceptance instead. The convolution component is explicitly grounded in Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers‘s Canon Layers, giving Canon Layers their first filed application outside pretraining. It also refines the “drafter-in-the-box” packaging pattern already seen in Introducing Muse Glimmer: An Open Agentic Model That Runs on Your Device (DFlash 1) and 1-bit Bonsai 27B — GGUF: 27B-class Reasoning at 1.125 Bits Per Weight (DSpark) — Muse Glimmer now gets a strictly better community drafter than the one shipped with the model, opening the question of whether frontier releases should ship official DFlash 2-class drafters at launch. Finally, the “the tokens are already in top-16, we just need to pick the right path” framing is a distinctive re-diagnosis of what parallel drafters get wrong: the bottleneck isn’t candidate quality but candidate selection, a lesson that may transfer to parallel-decoding diffusion LMs tracked under Diffusion Language Models.