DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models

TL;DR

DiffThinker reframes multimodal reasoning as an image-to-image generation task: instead of an MLLM emitting a textual chain of thought, a fine-tuned diffusion model (Qwen-Image-Edit-2509/2511, 20B MMDiT) directly produces a solution image (a drawn maze path, a filled Sudoku, a TSP tour, a reassembled jigsaw) from the problem image + a text instruction. Across seven vision-centric tasks the diffusion reasoner crushes text-CoT baselines — GPT-5, Gemini-3-Flash, and a same-data-finetuned Qwen3-VL-32B — and the paper argues the paradigm has four intrinsic advantages over autoregressive CoT: fixed inference cost, native parallel exploration of candidates, training/inference efficiency, and clean composability with MLLM verifiers. The hard claim is that for long-horizon vision-centric problems, reasoning should live in pixel space, not token space.

Key claims

Generative multimodal reasoning is formalized as a multimodal-to-image map f(I, T) → I_sol (with a separate symbolic parser for evaluation), in contrast to standard MLLMs’ multimodal-to-text map and “Thinking with Image” MLLMs’ interleaved tool-call loop [§3.1, Eqs. 1–4].
DiffThinker outperforms a same-data fine-tuned Qwen3-VL-32B baseline by +39.0%, Gemini-3-Flash by +111.6%, and GPT-5 by +314.2% averaged across seven tasks in four domains (sequential planning, combinatorial optimization, constraint satisfaction, spatial configuration) [§4.1, Table 1].
Diffusion reasoning is naturally parallel: by decoding the predicted clean-latent at each timestep, the authors show the model explores multiple candidate paths simultaneously in early steps (e.g., several maze trajectories spread across the grid) and progressively prunes them to one, without an explicit reflection/backtracking mechanism [§4.2, Fig. 4].
Inference cost is fixed and competitive: 1.1 s per instance at 20 Euler steps (vs. 1.0 s for Qwen3-VL-8B-SFT and 1.4 s for Qwen3-VL-32B-SFT), and accuracy plateaus past 20 steps so the budget is genuinely deterministic [§4.2 Fig. 5b, §4.3 Fig. 7].
DiffThinker × MLLM is strictly better than either alone on Jigsaw-level-4: DiffThinker generates multiple candidate reconstructions, the MLLM picks one against the original constraints, and the combined pipeline beats both isolated models [§4.2 Fig. 6].
Image-based reasoning beats video-based reasoning at this scale: DiffThinker-Video (built on Wan2.2-TI2V-5B) does demonstrate inherent reasoning capability on Maze-level-8 (a yellow ball navigates toward the target), but yields lower accuracy with higher training overhead and ~2× the inference latency (2.0 s vs. 1.1 s) [§4.4, Fig. 11].
Training data scales meaningfully: on Maze-level-32, accuracy crosses 90% only past ~30k samples, with a qualitative transition from “learning the rendering syntax” (grid alignment, line continuity) to “structural reasoning” as data grows [§4.3, Fig. 8].
The whole approach is built on Qwen-Image-Edit with flow matching (logit-normal timestep, MSE on velocity), one model trained per task domain, with CFG scale 3.0 as the sweet spot [§3.2, §4.3 Fig. 9].

Method

DiffThinker takes the multimodal input (problem image + text instruction) and conditions a Qwen-Image-Edit-2509 MMDiT (20B) to generate a solution image: a drawn red trajectory for VSP/Maze, a connected loop over city dots for TSP, a fully-populated Sudoku grid, or a reassembled jigsaw canvas. Training is standard flow matching — sample timestep t from a logit-normal, interpolate the VAE-encoded ground-truth latent with noise, regress the MMDiT against the velocity field driving noise → data. The MLLM piece of Qwen-Image-Edit handles the conditioning latent from text+image; the diffusion piece does the reasoning. Separate models are trained per task family (one for VSP, one for Maze, one for TSP, etc.), each on ~30k samples spanning multiple difficulty levels. At inference, a 20-step Euler ODE solver with CFG=3.0 produces the solution image, which is then passed through a task-specific symbolic parser (e.g., trace the red line through grid cells) to recover an answer comparable against the ground truth.

The paradigm framing is the contribution at least as much as the architecture. The paper draws a sharp distinction between (1) text-CoT MLLMs that map (image, text) → text-trace → answer, (2) “Thinking with Image” MLLMs that interleave tool calls and visual observations across multiple turns, and (3) DiffThinker’s direct (image, text) → solution-image map. Property #3 — native parallel reasoning — is visualized by decoding the predicted clean latent at each diffusion step, showing the model holds multiple candidate trajectories simultaneously in early steps and consolidates only late.

Results

Headline averages across 7 tasks (Avg column in Table 1):

Model	Avg
GPT-5	21.1
Gemini-3-Flash	41.3
Qwen3-VL-32B (zero-shot)	10.5
Qwen3-VL-32B (SFT, same data)	62.9
Qwen3-VL-32B (GRPO)	26.9
Qwen-Image-Edit-2509 (zero-shot, no fine-tune)	4.0
DiffThinker (Flow Matching, ours)	87.4
DiffThinker++ (Qwen-Image-Edit-2511)	88.5

The gaps are largest at the hard end of each task: at Maze-level-32, DiffThinker hits 56% while every MLLM baseline scores 0%; at Sudoku-level-35, DiffThinker hits 57% vs. ~2% for Qwen3-VL-32B-SFT. The Qwen-Image-Edit base model without fine-tuning scores 4.0 average — so the result is not “diffusion models are magically good at puzzles”, it’s “diffusion models are good substrates for learning these reasoning behaviors from ~30k examples”.

Ablations: 20 inference steps is the elbow; CFG=3.0 is the optimum; with <10k training samples the model is still in “render syntax” mode; collaborative DiffThinker→MLLM-verifier strictly beats either alone on Jigsaw-level-4.

Why it’s interesting

This is the cleanest argument the wiki has seen for the position that reasoning ≠ token generation. The filed papers on reasoning so far (Reasoning RL, Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning, vlm-gym: RL gym for vision language models (JAX)) all stay inside the autoregressive-LM-with-shaped-rewards frame; DiffThinker takes the opposite bet — keep reasoning in the modality the problem actually lives in (pixels), and let the diffusion ODE be the inference-time computation. For Luma, two threads are interesting: the collaborative-mode result suggests image diffusion can act as a “spatial reasoning backend” for an MLLM front-end, which is a natural fit for a video/3D pipeline that wants to keep semantics in tokens and geometry in pixels; and the native-parallel-reasoning observation (multiple candidates coexisting in the early-step clean-latent) is a serious mechanistic claim about why diffusion may beat autoregressive search on combinatorial problems. The DiffThinker-Video negative result is also instructive: video diffusion can reason (the maze video shows a ball navigating to the target), but the compute cost / sample efficiency ratio is currently bad — a useful data point for anyone considering training a “thinking video model”.