InterleaveThinker: Reinforcing Agentic Interleaved Generation

TL;DR

InterleaveThinker is a multi-agent pipeline from CUHK MMLab × Meituan that endows any single-image generator (FLUX.2-klein, Qwen-Image-Edit, OmniGen2) with interleaved text-image generation capabilities — producing coherent multi-step text+image sequences for visual narratives, instructional guidance, and embodied manipulation. A planner agent organizes the input sequence and emits per-step instructions; a critic agent evaluates each generated image, flags deviations, and refines the instruction for regeneration. Both agents are SFT-cold-started on Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k; the critic is then RL-trained on Interleave-Critic-RL-13k with GRPO using a step-wise reward, sidestepping the impractical >25-generator-call full-trajectory optimization. On interleaved generation benchmarks the framework matches Nano Banana and GPT-5; on FLUX.2-klein the wrapped pipeline substantially lifts performance on the reasoning-based WISE and RISE benchmarks.

Key claims

Existing single-image generators and even open-source UMMs are architecturally constrained to single-image output and cannot natively produce coherent interleaved text-image sequences across dependent steps [Abstract, §1].
A multi-agent pipeline (planner + critic + frozen image generator) endows any existing generator with interleaved generation capabilities without retraining the generator itself [Abstract].
The planner organizes the multimodal input sequence and decomposes it into executable per-step generation instructions; the critic evaluates each output, identifies deviations from the planned instruction, and refines the instruction for regeneration [Abstract, Method].
Format cold-start is done with Interleave-Planner-SFT-80k (planner) and Interleave-Critic-SFT-112k (critic); RL training of the critic uses Interleave-Critic-RL-13k under GRPO [Abstract].
A single interleaved generation trajectory can involve more than 25 generator calls, making whole-trajectory RL optimization computationally impractical; the paper proposes an accuracy reward + step-wise reward so that single-step RL guides the full trajectory [Abstract].
On interleaved generation benchmarks the framework achieves performance comparable to Nano Banana and GPT-5 [Abstract].
Wrapping a base image generator with InterleaveThinker substantially enhances the base model on reasoning-based benchmarks: on 4-step FLUX.2-klein, gains on WISE and RISE are highlighted as “substantial” [Abstract].

Method

InterleaveThinker decouples the interleaved-generation workflow into three roles: a Planner Agent that ingests the user’s image-text input sequence, organizes it, and emits per-step instructions specifying what the image generator should produce; a frozen Image Generator (e.g. FLUX.2-klein, Qwen-Image-Edit, OmniGen2) that executes each step’s instruction; and a Critic Agent that scores the generator’s output against the planned instruction, identifies deviations, and refines the instruction for re-execution if needed. The loop iterates per step until the critic accepts the output, then advances to the next planned step.

Training is two-stage. SFT cold-start uses Interleave-Planner-SFT-80k to teach the planner the decomposition format and Interleave-Critic-SFT-112k to teach the critic the evaluate-and-refine format. RL fine-tuning then targets the critic on Interleave-Critic-RL-13k using GRPO. Because a complete interleaved trajectory can require >25 generator calls — and optimizing through that many sampled rollouts is impractical — the authors design two reward terms that let single-step RL guide the whole trajectory: an accuracy reward (does the refined instruction produce an image matching the planned step?) and a step-wise reward (per-step quality signal). This lets GRPO operate at step granularity while still propagating credit toward end-to-end trajectory quality.

The authors also note that they convert their data into two real interleaved-data variants — simple (without reflection) and hard (with reflection) — and release both alongside the planner and critic checkpoints.

Results

Reaches performance comparable to Nano Banana and GPT-5 on interleaved generation benchmarks (specific benchmark names and numbers not stated in the abstract) [Abstract].
“Substantial gains” on WISE and RISE reasoning-based benchmarks when wrapping 4-step FLUX.2-klein as the base image generator [Abstract].
The framework is generator-agnostic: the abstract claims improvement “across various image generators,” consistent with the same Critique-Refine-Repeat pattern that EditThinker: Unlocking Iterative Reasoning for Any Image Editor (by overlapping authors) demonstrated for single-turn image editing across Flux-Kontext, OmniGen2, and Qwen-Image-Edit. Quantitative numbers per generator were not in the fetched abstract; refer to the full PDF.

(The paper page summarizes only what the abstract and project page assert; precise tables, per-step ablations, and the data-construction pipeline details require the full paper.)

Why it’s interesting

InterleaveThinker is the direct generalization of EditThinker: Unlocking Iterative Reasoning for Any Image Editor — same authorship cluster (Dian Zheng, Manyuan Zhang, Kaituo Feng, Hongsheng Li) and the same Critique-Refine-Repeat skeleton — from single-image editing to interleaved multi-step generation. EditThinker wrapped a single editor; InterleaveThinker stacks a planner on top so the trajectory itself is decomposed into steps the critic then refines individually. This is the cleanest filed instance of the “external planner + frozen generator + RL-trained critic” recipe for the Unified Multimodal Models interleaved-output problem — sitting in the Agentic corner of that page’s E2E / Decoupled / Agentic taxonomy and offering a third route alongside the architectural recipes (DuoGen’s video-DiT-as-image-head DuoGen: Towards General Purpose Interleaved Multimodal Generation, OmniWeaving’s Decoupled→E2E OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning, NEO-unify’s encoder-free MoT NEO-unify: Building Native Multimodal Unified Models End to End) and the architectural-extension recipes for video output (Bernini: Latent Semantic Planning for Video Diffusion). The fact that it claims Nano-Banana/GPT-5-level interleaved performance with a frozen base generator is the bet to verify: it would mean the gap between open generators and proprietary unified models on interleaved tasks is largely a scaffolding gap, not a base-model gap.

The accuracy+step-wise reward design is the directly transferable engineering contribution. Long-horizon agentic trajectories with expensive per-step environment calls (here: >25 image-generation rollouts) are the rule, not the exception, across Tool-Use Agents and Reasoning RL; the InterleaveThinker recipe (“compose single-step rewards so single-step GRPO is provably aligned with trajectory quality”) is the same problem the closed-loop tool-use literature is grinding on with verification gates and rollout pruning. The step-wise reward primitive is also a sibling of EditThinker’s differential edit reward (positive only when the refined instruction strictly improves on the previous step per a GPT-4.1 expert) — which validates the pattern across at least two settings.

For Thinking with Modalities, InterleaveThinker is a (c)-style external-tool Think-Act-Observe loop but where the “tool” is a powerful image generator and the loop ranges over a planned sequence of generations rather than a single output. This sharpens the substrate axis: Agentic Vision (Gemini 3 Flash) invokes Python for crops; EditThinker invokes a single editor for one-shot edits; InterleaveThinker invokes the editor across an entire planned narrative. The “frozen generator + trained orchestrator(s)” pattern lets a small MLLM (8B-class for EditThinker; planner+critic for InterleaveThinker) lift much larger frozen image models on reasoning-heavy benchmarks they cannot solve alone — which is a clean variant of the “pixel-space generator proposes; token-space reasoner verifies” pattern the concept page already tracks.