Diffusion Distillation

TL;DR

Diffusion distillation compresses a multi-step pretrained diffusion / flow teacher into a few-step (1–4 NFE) student. The 2025–2026 thread on the wiki spans four orthogonal axes: (1) objective design — linear-shortcut (DMD, consistency, rectified flow) vs non-linear (ArcFlow’s analytic mixture-of-momentums); (2) role separation across student steps — DP-DMD’s v-prediction first step + DMD later steps for diversity preservation; TMD’s outer/inner backbone-flow-head decoupling; RMD’s resolution-stage separation (low-res for high-noise semantic stage, high-res for low-noise refinement); (3) train-vs-inference alignment — TMD’s flow-head rollout inside DMD, Causal Forcing’s AR-teacher ODE init to satisfy frame-level injectivity, RMD’s logSNR-invariance mapping that distills over identical denoising states regardless of resolution; (4) what to post-train with after distillation — TDM-R1’s non-differentiable-reward RL using deterministic-trajectory unbiased per-step reward estimates. Common substrate: distillation is now the practical production path to 1–4 NFE on production-scale models (SD3.5-M, FLUX, Qwen-Image-20B, Wan2.1 14B, Z-Image 6B), while teacher-free recipes like SoFlow / MF hold the from-scratch frontier on smaller image benchmarks.

Key claims

• Linear-shortcut distillation underfits the teacher’s tangent variation across timesteps (ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation). Replacing the linear discrete update with an analytic mixture-of-continuous-momentum velocity gives a closed-form ODE integral over arbitrary [t,s] intervals; on FLUX.1-dev and Qwen-Image-20B this beats pi-Flow / TwinFlow / Qwen-Image-Lightning / SenseFlow at 2 NFE with <5% of parameters trained (ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation §1, §3.1, §3.2; Tables 1–2). The win is geometric, not loss-design.
• DMD’s reverse-KL objective is mode-seeking and costs diversity (Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis). Role-separating the few-step student so step 1 uses v-prediction (forward-KL-flavored, preserves diversity) while steps 2..k use DMD (refines quality), with a gradient stop preventing DMD pressure from overriding step 1, restores teacher-class diversity without LPIPS or GAN add-ons (Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis §3, Fig. 1).
• Asymmetric distillation (bidirectional teacher → AR student) has a structural injectivity gap that prior work ignored (Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation). ODE distillation is fundamentally flow-map matching and requires frame-level injectivity; distilling an AR student from a bidirectional teacher violates this and collapses to a conditional-expectation solution. Fix: insert a teacher-forced AR diffusion fine-tune before causal ODE distillation, so the teacher is AR and its PF-ODE naturally satisfies injectivity. +19.3% Dynamic Degree, +8.7% VisionReward, +16.7% Instruction Following vs Self-Forcing at matched compute (Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation §2.2, §3.2, Prop. 3.4, §4).
• Standard DMD has a train/inference gap when the student is multi-step (Transition Matching Distillation for Fast Video Generation). DMD supervises the student at a single sampled timestep, but at inference the student rolls out a full N-step trajectory. TMD closes the gap by rolling out the student’s flow head over its full inner-step schedule during DMD gradient computation; combined with MeanFlow-style stage-1 init this is SOTA on Wan2.1 1.3B / 14B few-step T2V (Transition Matching Distillation for Fast Video Generation §3, §4). DiagDistill (Streaming Autoregressive Video Generation via Diagonal Distillation §3.2) attacks the AR-specific version of the same gap by injecting the exact noise level the next-chunk KV cache will see at inference into the conditioning during training (Diagonal Forcing, t_d≈100 optimal out of 1000).
• Non-differentiable rewards work on few-step distilled students if and only if the sampling trajectory is deterministic (TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward). For a Dirac posterior $p(x_0 \mid x_t)$ (ODE sampling), a single clean reward $r(x_0)$ is an unbiased single-sample estimate of the marginal reward at every intermediate $x_t$ — variance vanishes, so per-step RL signal is recoverable from one endpoint evaluation (TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward §3.1, Eq. 6). This formally explains Mix-GRPO’s earlier observation that ODE-izing RL rollout steps helps, and it is the structural reason TDM (not DMD or consistency models) is the chosen base for TDM-R1.
• The naive baseline of stacking standard diffusion RL on top of a few-step distillation loss fails (TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward). Denoising-style RL losses (which most diffusion RL methods reduce to) produce blurry outputs at few steps and degrade in late training. A diffusion-parameterized Surrogate Reward, trained group-wise (BT on positive/negative noisy groups, EMA reference) to provide a fine-grained differentiable per-step signal, fixes this — TDM-R1 reaches GenEval 0.92 at 4 NFE on SD3.5-M vs Flow-GRPO 0.95 at 80 NFE (TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward §3.2, §3.3, Table 1, Figs. 4–5).
• Improving a model as a reward signal does not require improving it as a generator (TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward). TDM-R1 trains a Surrogate Reward parameterized by a diffusion model; that diffusion model at 80 NFE is a strong generator, but the 4-NFE TDM-R1 student trained against its reward outperforms it on both in-domain and out-of-domain metrics. The reward’s quality as gradient signal is independent of its quality as samples (TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward §4.2).
• Causal consistency distillation can substitute for offline ODE trajectory generation in causal AR distillation (Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation). Causal Forcing needed offline PF-ODE trajectories from the AR teacher; Causal Forcing++ replaces this with causal consistency distillation, pushing to 1- and 2-step frame-wise interactive video models (Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation).
• Step-reduction has saturated; cascading per-step resolution is the next axis (Cross-Resolution Distribution Matching for Diffusion Distillation). Aggressive step reduction to 1–3 NFE catastrophically degrades quality, so further efficiency comes from running early high-noise steps at low resolution. Naive resolution drop introduces cross-resolution distributional inconsistencies because the same diffusion model yields distinct data distributions per resolution; RMD bridges this by folding a logSNR-invariance mapping and a predicted-noise + Gaussian upsampling transformation directly into the DMD objective. On Wan2.1-T2V-14B this delivers 25.6× speedup at VBench Total 82.51 — beating 6-NFE DMD2 (80.30) and 6-NFE TDM (80.48) at

  50% less compute (Cross-Resolution Distribution Matching for Diffusion Distillation §1, §3.1, §3.3, Tables 2, 3).

• For AR video, a per-chunk-asymmetric step schedule beats a uniform one at matched total NFE (Streaming Autoregressive Video Generation via Diagonal Distillation). DiagDistill allocates more denoising steps to early chunks (which establish structural priors) and fewer to later chunks (which inherit those priors via the KV cache), e.g. 4-3-2-2-2-2-2 over 7 chunks of a 5-second video. On Wan2.1-T2V-1.3B this hits 31 FPS / 0.37 s first-frame latency / VBench Total 84.48 — beating Self-Forcing (17 FPS / 0.69 s / 84.31) and matching the undistilled Wan2.1 teacher (84.26) at 277.3× speedup (Streaming Autoregressive Video Generation via Diagonal Distillation §3.2, Table 1). This is the chunk-axis analogue of RMD’s resolution-axis cascade — “more compute where it matters most” applied to the temporal dimension instead of the noise dimension.
• Motion amplitude is the dominant failure mode of few-step AR video distillation (Streaming Autoregressive Video Generation via Diagonal Distillation). Truncated noise-estimation paths attenuate motion; standard DMD’s spatial reverse-KL doesn’t capture this. Flow Distribution Matching adds a flow-field reverse-KL using a lightweight learnable latent-frame-difference module (teacher updated via EMA), restoring motion under strict step constraints without an external optical-flow network. Weight λ=1.0 optimal (Streaming Autoregressive Video Generation via Diagonal Distillation §3.3, Fig. 5(b)).
• DMD2 can be applied to the decoder of a latent pipeline, not just the main denoise loop. PiD distills a pixel-diffusion decoder (operating on either VAE or SigLIP/DINOv2 latents at megapixel resolutions) from multi-step to 4 NFE, reaching ~210 ms on GB200 / under 1 s on RTX 5090 for a 512²→2048² decode+upsample (PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion §3, Abstract). This widens the surface DMD-family distillation is deployed across — beyond the generator itself, into the decode step that often sits between the LDM and the user.

Recent contributions

• [2026-07-20] Introducing Cosmos 3 Edge: Alongside Cosmos 3 Edge, NVIDIA releases a Cosmos 3 Super 4-Step Distillation checkpoint (T2I + I2V) — distribution-matching distillation reducing 35–50 denoising steps to 4 for up to 25× faster inference, with the post-training script released so developers can rerun the distillation on their own domains.
• [2026-07-16] FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space: FlowDAgger ports the flow-matching inversion primitive (reverse-time ODE integration + local refinement to recover a source noise from a target sample) out of the image-editing setting and into robot-policy adaptation: each human corrective action a* is mapped to the noise ẑ* that the frozen flow-matching policy would have integrated forward into it, and those (o, ẑ*) pairs supervise a small latent policy that steers the base sampler at deployment. A wiki-first datapoint that noise-space inversion is a stable supervision signal beyond diffusion editing — with the frozen base policy playing the role of the frozen image DiT and the latent policy playing the role of a tiny steering adapter.
• [2026-07-10] LingBot-VA 2.0: Native Video-Action Pretraining for Generalizable Robot Control: Few-step consistency distillation is applied to both the video expert and the action expert in a Mixture-of-Transformers video-action model, and stacked with quantization, sparse MoE routing, and asynchronous execution to reach a reported 225 Hz peak inference frequency on real robot hardware — one of the strongest filed cases of distillation as a load-bearing lever for real-time closed-loop VLA/WFM deployment (not just for image/video generation latency).
• [2026-07-09] LingBot-World 2.0 / LingBot-World-Infinity — Infinite Worlds with Versatile Interactions: LingBot-World 2.0 distills a causal-fast real-time student from a causal-pretrained base to hit 720p / 60 fps interactive throughput — a step up from v1’s 480p / 16 fps LingBot-World-Fast. Method details not yet disclosed (tech report URL unresolvable at filing); positioned as the throughput datapoint that lets the open Wan-based interactive-WFM stack quote higher fps/resolution than DeepMind’s closed Project Genie surface.
• [2026-07-04] RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation: Step Forcing distills an AR video diffusion world model to a 4-step schedule by supervising denoising from one-step self-forwarded priors under the same schedule used at inference, avoiding the expense and action-following degradation of Self Forcing.
• [2026-06-29] Qwen-Image-2.0-RL Technical Report: Qwen-Image-2.0-RL uses on-policy distillation with trajectory-level velocity matching as the final merge step of a multi-task RL pipeline — consolidating a T2I GRPO specialist and an edit GRPO specialist into a single deployable student. New use case for OPD in the diffusion stack: not step-count reduction or quality lift over a single larger teacher, but multi-policy consolidation after task-specific RL.
• [2026-06-23] Causal World Modeling for Robot Control (LingBot-VA): Noisy History Augmentation is a non-distillation route to inference-step reduction: instead of training a separate few-step student, the model is trained with video history corrupted to a sampled flow-time t (using the same flow-matching interpolation x_t = (1-t)·noise + t·x_0) so the inverse-dynamics action decoder learns to read actions from partially denoised video latents. At inference the video need only be denoised to t* (not 0), roughly halving per-step denoising work — without retraining a student. Structurally analogous to Causal Forcing’s distillation trick but applied at the action-decoder readout rather than the video velocity field.
• [2026-06-18] Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation: DFD (NVIDIA + Michigan + Toronto): one-line post-training fix for DMD/DMD2 that mixes real-data samples into the teacher-score evaluation roughly half the time, turning DMD’s reverse-KL student-only gradient into a teacher-score discrepancy that pulls the student toward missed modes and away from spurious ones — restores diversity and resolves over-saturation in 100–300 finetune steps on Wan2.1-1.3B and Cosmos-Predict2.5-2B (T2V, I2V, and AR video), with the claim that explicit real-data regularization is stronger than DMD2’s implicit GAN regularization. A pure-objective counterpart to DP-DMD’s role-separated-student fix for the same DMD failure mode.
• [2026-06-03] Bootstrap Your Generator: Unpaired Visual Editing with Flow Matching: BYG (NVIDIA, ICML 2026): bootstraps an unpaired editor out of a pretrained flow-matching generator by manipulating its velocity field around the target-style manifold, rather than fitting on paired (source, edit) data — reframes editing as a flow-trajectory bootstrap problem.
• [2026-06-02] Stable-Layers: Fine-Tuning Image Layer Decomposition Models with VLM-Scored Reinforcement Learning: Stable-Layers post-trains a diffusion-based layer decomposer with Flow-GRPO + LoRA using VLM-only rewards — a flow-matching RL fine-tune (no distillation step shortening) but in the same Flow-GRPO family that ties this cluster to RL alignment for flow models.
• [2026-06-01] Pre-training under infinite compute: Stanford CRFM — LLM-pretraining analog of the ensemble-and-distill recipe. Distilling an 8-ensemble of 300M models into a single 300M student via sequence-level KD retains 83% of the ensemble benefit, outperforming the regularized recipe’s $N\to\infty$ asymptote at finite inference cost. Self-distillation (300M teacher → 300M student of identical architecture, real + synthetic token mixture) beats the teacher — concrete recipe for the Distillation Scaling Laws thesis that distillation pushes the loss-vs-parameter curve to the left.
• [2026-06-01] RenderFlow: Single-Step Neural Rendering via Flow Matching: RenderFlow (Disney Research, CVPR 2026): single-step neural rendering trained directly with a flow-matching objective on (G-buffer, photoreal RGB) pairs — a from-scratch single-step generator targeting graphics rendering rather than image synthesis. Complements teacher-student distillation recipes (TMD, DMD, ArcFlow) with a domain where deterministic single-step inference is the requirement, not a speedup target, because stochastic samplers produce temporal jitter when applied frame-by-frame.
• [2026-05-30] Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE: Mamoda2.5 reports a joint few-step distillation + RL framework compressing a 30-step video-editing teacher to a 4-step student with up to 95.9× inference speedup over open-source baselines, applied to a 25B/3B-active DiT-MoE.
• [2026-05-29] MRT: Masked Region Transformer for Layered Image Generation and Editing at Scale: MRT distills its 20B masked region diffusion model for multi-layer transparent image generation down to 8 steps with “minimal quality degradation,” contributing a 20B-scale data point for distillation in the layered-image regime.
• [2026-05-25] PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion (PiD, NVIDIA): DMD2-distill a pixel-diffusion decoder (not a generator) from multi-step to 4 NFE. The decoder operates in pixel space at the target upsampled resolution (e.g., 2048×2048) and conditions on a noise-corrupted latent via a sigma-aware adapter. Distillation outcome: 4-step inference, ~210 ms on GB200 / sub-1 s on a consumer RTX 5090 at 13 GB peak memory for the 512² → 2048² decode, ~6× faster than cascaded diffusion-based super-resolution pipelines (and 5.9× over SeedVR2 on the same task) with better visual fidelity per project-page side-by-sides. New target for DMD2 distillation: the decode+upsample step rather than the generative diffusion itself.
• [2026-05-23] Streaming Autoregressive Video Generation via Diagonal Distillation (DiagDistill, ICLR 2026): per-chunk asymmetric step schedule for AR video distillation (more steps early, fewer late, e.g. 4-3-2-2-2-2-2 over 7 chunks). Three pieces: (a) Diagonal Denoising schedule itself; (b) Diagonal Forcing — conditioning the next chunk on a noised version of the previous chunk’s output during training (t_d≈100 / 1000 optimal) to match inference-time KV-cache state and eliminate the implicit “next-noise-level prediction” that drives over-saturation drift; (c) Flow Distribution Matching — a flow-field reverse-KL on top of DMD using a learnable latent-frame-difference module, restoring motion amplitude lost at low NFE. On Wan2.1-T2V-1.3B: 31 FPS, 0.37 s first-frame latency, VBench Total 84.48 — 277.3× faster than the Wan2.1 teacher while slightly exceeding it on Total and Semantic. Ablations show Diagonal Denoising buys the speedup (quality stays at Self-Forcing level when removed), while Diagonal Forcing + Flow Distribution Matching buy the quality.
• [2026-05-23] Cross-Resolution Distribution Matching for Diffusion Distillation (RMD, Huawei Cloud + NJUST): cross-resolution DMD — partitions the teacher’s logSNR trajectory into resolution-specific segments (low-res for high-noise / semantic stage, high-res for low-noise / refinement), derives student timestep intervals via an SD3-style logSNR-shift compensation so distillation occurs over matched denoising states across resolutions, and upsamples the student state into the teacher’s high-res space with a mixture of predicted noise (inherits teacher ODE trajectory) and Gaussian noise (bridges large resolution gaps). Two-stage (lo+hi) recipe on SDXL @ 4 NFE: 33.4× speedup and beats DMD2 / TDM / SDXL-Turbo / SDXL-Lightning across HPS/AeS/CLIP. On Wan2.1-T2V-14B: 25.6× speedup and VBench 82.51 vs 6-NFE TDM 80.48 / 6-NFE DMD2 80.30 — the first wiki datapoint where cascading resolution beats fixed-resolution distillation at fewer effective NFE on a 14B video teacher. The component ablation shows the cross-resolution RM objective (not the cascaded upsampling alone) is what does the lifting; UP without RM underperforms the baseline.
• [2026-05-23] TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward (TDM-R1, ICML 2026): RL post-training of a TDM few-step student with non-differentiable rewards (GenEval / OCR / HPSv3). Hinges on the unbiased-per-step- reward property of deterministic ODE trajectories; introduces a diffusion-parameterized Surrogate Reward with group-BT learning and EMA reference, and a marginal-level (not trajectory-level) reverse-KL to the TDM teacher. SD3.5-M @ 4 NFE → GenEval 0.92; Z-Image @ 4 NFE strictly dominates 100-NFE Z-Image and Z-Image-Turbo on every reported metric.
• [2026-05-23] Self-Flow: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis: internal self-distillation (EMA teacher inside the same generator, Dual- Timestep Scheduling) as an alternative to external DINO-based REPA. Not a teacher-student distillation between two diffusion models, but the same conceptual lever applied within one model; filed here because the “self-distillation as supervision” framing overlaps with the multi-step→few-step thread.
• [2026-05-23] ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation (ArcFlow): non-linear-flow 2-step distillation; analytic mixture-of-continuous- momentum velocity gives closed-form ODE integration. <5% of parameters trained (rank-256 LoRA on FFN + output head). Beats pi-Flow / TwinFlow / SenseFlow / Qwen-Image-Lightning at 2 NFE on FLUX.1-dev and Qwen-Image-20B; +85.7% diversity over Qwen-Image- Lightning on OneIG-Bench.
• [2026-05-23] Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis (DP-DMD): role-separated student where step 1 uses v-prediction (diversity) and steps 2..k use DMD (quality), gradient stop between them. No perceptual / adversarial / extra-real-data machinery — beats LPIPS-DMD and GAN-DMD on the diversity-quality Pareto at 4 NFE.
• [2026-05-23] Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation (Causal Forcing++): replaces offline PF-ODE trajectory generation with causal consistency distillation; scales the Causal Forcing pipeline to 1- and 2-step frame-wise real-time interactive video.
• [2026-05-23] Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation (Causal Forcing): theoretical correction to Self-Forcing / CausVid — bidirectional teacher → AR student violates frame-level injectivity required by ODE distillation. Three-stage fix: teacher-forced AR diffusion fine-tune → causal ODE distillation from the AR teacher → asymmetric DMD. +19.3% Dynamic Degree on real-time-streaming T2V at matched compute.
• [2026-05-22] Transition Matching Distillation for Fast Video Generation (TMD, NVIDIA + NYU): distills Wan2.1 1.3B / 14B T2V into ≤4-NFE students. Two ideas: (a) decouple the student into a heavy semantic backbone (run once per outer transition) and a recurrent flow head (run for several inner flow updates); (b) close DMD’s train/inference gap by rolling out the flow head during DMD gradient computation. MeanFlow-style stage-1 init for the flow head outperforms vanilla flow-matching init.

Open questions

• Is the “deterministic trajectory” property the actual gating condition for non-differentiable-reward RL, or just the cleanest one? TDM-R1 leans on TDM specifically. ArcFlow, TMD, DP-DMD all use deterministic ODE sampling at inference — can the TDM-R1 recipe drop in on top of each? Predicted yes (Eq. 6 of TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward is generic in the trajectory’s determinism), but untested. RMD’s cascaded trajectory adds resolution transitions to this question.
• What is the right marginal-KL anchor for from-scratch few-step models (SoFlow: Solution Flow Models for One-Step Generative Modeling, Self-Evaluation Unlocks Any-Step Text-to-Image Generation)? TDM-R1’s KL is to a pretrained TDM teacher; from-scratch generators have no such anchor. Either the recipe needs a teacher-free regularizer (the generator’s own EMA?) or it doesn’t transfer cleanly.
• Linear vs non-linear vs role-separated vs resolution-separated vs chunk-asymmetric objective — which matters most at each NFE? ArcFlow (non-linear), DP-DMD (role-separated across student steps), RMD (resolution-separated across trajectory segments), and now DiagDistill (asymmetric across AR chunks) all report wins on FLUX-class or Wan-class models at 2–4 effective NFE. None of them compare against the others directly; the field is missing a clean ablation that decouples trajectory parameterization from per-step objective from resolution schedule from per-chunk schedule.
• Are the cascade ideas composable? RMD cascades resolution along the noise axis; DiagDistill cascades step count along the AR-chunk axis. The two cascades are orthogonal and should compose trivially on Wan2.1-T2V-14B; no one has tried.
• Architectural-gap fixes (Causal Forcing’s three-stage AR teacher), train/inference-gap fixes (TMD’s flow-head rollout in DMD; DiagDistill’s Diagonal Forcing for the AR-chunk-specific version), trajectory-parameterization fixes (ArcFlow’s closed-form non-linear ODE), and resolution-cascade fixes (RMD’s logSNR-invariance mapping) all appear to attack independent failure modes. Are they composable? In principle yes; nobody has combined two yet. RMD + TMD’s flow head on Wan2.1-14B is an obvious next target; DiagDistill + Causal Forcing/++ on AR video is another.
• Does TDM-R1 generalize to video? Z-Image is image-class, 6B. The natural target is Wan2.1 14B via TMD’s flow head + a video reward (VBench, ViewCrafter geometry score). Open whether the unbiased-per-step-reward argument survives the much higher-dimensional reward landscape of video. DiagDistill’s Flow Distribution Matching is a non-RL prior for the motion component of that reward and may compose.
• Distillation + RL vs distillation-of-RL. TDM-R1 ablates against “first RL the many-step teacher with DGPO, then distill with TDM”: it converges fast but plateaus because the teacher caps the student. The clean composition appears to be RL during distillation, with a jointly-trained reward — but this is only one ablation, on SD3.5-M.
• Does distilling the decoder (PiD) compose with distilling the generator? PiD’s 4-step DMD2 decoder runs on top of an otherwise normal LDM. The total wall-clock saving is largest if the generator is also distilled (DMD2 / pi-Flow / ArcFlow). PiD additionally allows the LDM to exit early via the sigma-aware adapter, which interacts nontrivially with the generator’s distilled step schedule. Whether the combined system beats either component in isolation, and where the right exit point is, has not been measured.

Papers

2026-05

• PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion — PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion (published 2026-05-22)
• Chatterbox Turbo — 350M Zero-shot TTS (Resemble AI) — Chatterbox Turbo — 350M Zero-shot TTS (Resemble AI) (published 2026-05-25)
• TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times — TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times (published 2026-05-25)
• AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation — AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation (published 2026-05-23)
• Stable Audio 3 — Stable Audio 3 (published 2026-05-18)
• LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation — LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation (published 2026-05-18)
• Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation — Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation (published 2026-05-14)
• HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer — HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer (published 2026-05-10)
• Learning the integral of a diffusion model (flow maps) — Learning the integral of a diffusion model (flow maps) (published 2026-05-06)

2026-04

• Representation Fréchet Loss for Visual Generation — Representation Fréchet Loss for Visual Generation (published 2026-04-30)
• Introducing Odyssey-2 Max: Scaled World Simulation — Introducing Odyssey-2 Max: Scaled World Simulation (published 2026-04-21)
• Continuous Adversarial Flow Models — Continuous Adversarial Flow Models (published 2026-04-13)
• LPM 1.0: Video-based Character Performance Model — LPM 1.0: Video-based Character Performance Model (published 2026-04-09)

2026-03

• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory — Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory (published 2026-03-27)
• Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model (daVinci-MagiHuman) — Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model (daVinci-MagiHuman) (published 2026-03-23)
• FLUX.2 [klein] 9B-KV: KV-cache optimized variant for accelerated multi-reference editing — FLUX.2 [klein] 9B-KV: KV-cache optimized variant for accelerated multi-reference editing (published 2026-03-13)
• OmniForcing: Unleashing Real-time Joint Audio-Visual Generation — OmniForcing: Unleashing Real-time Joint Audio-Visual Generation (published 2026-03-12)
• Streaming Autoregressive Video Generation via Diagonal Distillation — Streaming Autoregressive Video Generation via Diagonal Distillation (published 2026-03-10)
• A Unified View of Drifting and Score-Based Models — A Unified View of Drifting and Score-Based Models (published 2026-03-08)
• TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward — TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward (published 2026-03-08)
• Cross-Resolution Distribution Matching for Diffusion Distillation — Cross-Resolution Distribution Matching for Diffusion Distillation (published 2026-03-06)
• Self-Flow: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis — Self-Flow: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis (published 2026-03-06)
• CausalWan2.2-I2V-A14B-Preview — FastVideo causal I2V model on Wan2.2-A14B — CausalWan2.2-I2V-A14B-Preview — FastVideo causal I2V model on Wan2.2-A14B (published 2026-03-02)

2026-02

• ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation — ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation (published 2026-02-09)
• Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis — Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis (published 2026-02-03)
• Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation — Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation (published 2026-02-02)
• FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching — FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching (published 2026-02-01)

2026-01

• Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield — Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield (published 2026-01-26)
• Gumbel Distillation for Parallel Text Generation — Gumbel Distillation for Parallel Text Generation (published 2026-01-26)
• Transition Matching Distillation for Fast Video Generation — Transition Matching Distillation for Fast Video Generation (published 2026-01-14)
• DUO-VSR: Dual-Stream Distillation for One-Step Video Super-Resolution — DUO-VSR: Dual-Stream Distillation for One-Step Video Super-Resolution (published 2026-01-01)
• HY-SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models — HY-SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models (published 2026-01-01)

2025-12

• fal open-sources FLUX.2 [dev] Turbo — distilled FLUX.2 with sub-second generation — fal open-sources FLUX.2 [dev] Turbo — distilled FLUX.2 with sub-second generation (published 2025-12-29)
• Self-Evaluation Unlocks Any-Step Text-to-Image Generation — Self-Evaluation Unlocks Any-Step Text-to-Image Generation (published 2025-12-26)
• Qwen-Image-Edit-2511-Lightning — Step-Distilled 4-Step LoRA for Qwen-Image-Edit-2511 — Qwen-Image-Edit-2511-Lightning — Step-Distilled 4-Step LoRA for Qwen-Image-Edit-2511 (published 2025-12-23)
• Yume-1.5: A Text-Controlled Interactive World Generation Model — Yume-1.5: A Text-Controlled Interactive World Generation Model (published 2025-12-22)
• Kling-Omni Technical Report — Kling-Omni Technical Report (published 2025-12-18)
• SoFlow: Solution Flow Models for One-Step Generative Modeling — SoFlow: Solution Flow Models for One-Step Generative Modeling (published 2025-12-17)
• HY-World 1.5 (WorldPlay): A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency — HY-World 1.5 (WorldPlay): A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency (published 2025-12-17)
• Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model — Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model (published 2025-12-15)
• EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editing — EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editing (published 2025-12-05)
• TwinFlow: Realizing One-step Generation on Large Models with Self-adversarial Flows — TwinFlow: Realizing One-step Generation on Large Models with Self-adversarial Flows (published 2025-12-03)
• Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models (F2D2) — Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models (F2D2) (published 2025-12-02)
• Improved Mean Flows: On the Challenges of Fastforward Generative Models — Improved Mean Flows: On the Challenges of Fastforward Generative Models (published 2025-12-01)
• RELIC: Interactive Video World Models with Long-Horizon Memory — RELIC: Interactive Video World Models with Long-Horizon Memory (published 2025-12-01)

2025-11

• Adversarial Flow Models — Adversarial Flow Models (published 2025-11-27)
• Test-time scaling of diffusions with flow maps — Test-time scaling of diffusions with flow maps (published 2025-11-27)
• STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows — STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows (published 2025-11-25)
• SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation — SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation (published 2025-11-24)
• FreeFlow: Flow Map Distillation Without Data — FreeFlow: Flow Map Distillation Without Data (published 2025-11-24)
• Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation — Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation (published 2025-11-19)
• Distribution Matching Distillation Meets Reinforcement Learning — Distribution Matching Distillation Meets Reinforcement Learning (published 2025-11-17)
• Neodragon: Mobile Video Generation using Diffusion Transformer — Neodragon: Mobile Video Generation using Diffusion Transformer (published 2025-11-08)
• MotionStream: Real-Time Video Generation with Interactive Motion Controls — MotionStream: Real-Time Video Generation with Interactive Motion Controls (published 2025-11-03)
• FlashVSR v1.1 — Real-Time Diffusion-Based Streaming Video Super-Resolution (HF model release) — FlashVSR v1.1 — Real-Time Diffusion-Based Streaming Video Super-Resolution (HF model release) (published 2025-11-01)

2025-10

• Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals — Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals (published 2025-10-31)
• Eigen-Banana-Qwen-Image-Edit: Lightning-Fast Instruction-Based Image Editing with Pico-Banana-400K — Eigen-Banana-Qwen-Image-Edit: Lightning-Fast Instruction-Based Image Editing with Pico-Banana-400K (published 2025-10-30)
• FARMER: Flow AutoRegressive Transformer over Pixels — FARMER: Flow AutoRegressive Transformer over Pixels (published 2025-10-27)
• The Principles of Diffusion Models — The Principles of Diffusion Models (published 2025-10-24)
• Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset (Ditto / Editto) — Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset (Ditto / Editto) (published 2025-10-17)
• pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation — pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation (published 2025-10-16)
• Learning an Image Editing Model without Image Editing Pairs — Learning an Image Editing Model without Image Editing Pairs (published 2025-10-16)
• FlashWorld: High-quality 3D Scene Generation within Seconds — FlashWorld: High-quality 3D Scene Generation within Seconds (published 2025-10-15)
• Krea Realtime 14B: Real-Time, Long-Form AI Video Generation — Krea Realtime 14B: Real-Time, Long-Form AI Video Generation (published 2025-10-15)
• SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenization — SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenization (published 2025-10-07)
• Sora 2 Pro takes 10-15 minutes for a 15s HD video — David Attisaas first-impressions thread — Sora 2 Pro takes 10-15 minutes for a 15s HD video — David Attisaas first-impressions thread (published 2025-10-06)
• ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation — ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation (published 2025-10-05)
• Self-Forcing++: Towards Minute-Scale High-Quality Video Generation — Self-Forcing++: Towards Minute-Scale High-Quality Video Generation (published 2025-10-02)

2025-09

• Rolling Forcing: Autoregressive Long Video Diffusion in Real Time — Rolling Forcing: Autoregressive Long Video Diffusion in Real Time (published 2025-09-29)
• Seedream 4.0: Toward Next-generation Multimodal Image Generation — Seedream 4.0: Toward Next-generation Multimodal Image Generation (published 2025-09-24)
• Hyper-Bagel: A Unified Acceleration Framework for Multimodal Understanding and Generation — Hyper-Bagel: A Unified Acceleration Framework for Multimodal Understanding and Generation (published 2025-09-23)
• HunyuanImage 2.1 — Efficient 2K Text-to-Image with 32× VAE + DINOv2-Aligned REPA and MeanFlow Distillation — HunyuanImage 2.1 — Efficient 2K Text-to-Image with 32× VAE + DINOv2-Aligned REPA and MeanFlow Distillation (published 2025-09-08)

2025-08

• Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models — Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models (published 2025-08-13)
• FastWan: Generating a 5-Second Video in 5 Seconds via Sparse Distillation — FastWan: Generating a 5-Second Video in 5 Seconds via Sparse Distillation (published 2025-08-04)

2025-07

• Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis — Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis (published 2025-07-24)
• SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modeling — SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modeling (published 2025-07-22)
• MirageLSD: The First Live-Stream Diffusion AI Video Model — MirageLSD: The First Live-Stream Diffusion AI Video Model (published 2025-07-17)

2025-06

• Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition — Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition (published 2025-06-20)
• Align Your Flow: Scaling Continuous-Time Flow Map Distillation — Align Your Flow: Scaling Continuous-Time Flow Map Distillation (published 2025-06-17)
• Seedance 1.0: Exploring the Boundaries of Video Generation Models — Seedance 1.0: Exploring the Boundaries of Video Generation Models (published 2025-06-12)
• The Diffusion Duality — The Diffusion Duality (published 2025-06-12)
• Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation — Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation (published 2025-06-11)
• Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation — Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (published 2025-06-11)
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion — Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (published 2025-06-09)
• TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models — TalkingMachines: Real-Time Audio-Driven FaceTime-Style Video via Autoregressive Diffusion Models (published 2025-06-03)

2025-05

• Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models — Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models (published 2025-05-27)
• Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking — Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking (published 2025-05-24)
• Mean Flows for One-step Generative Modeling — Mean Flows for One-step Generative Modeling (published 2025-05-19)

2025-04

• Autoregressive Distillation of Diffusion Transformers — Autoregressive Distillation of Diffusion Transformers (published 2025-04-15)

2024-12

• Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection — Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection (published 2024-12-14)
• CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models — CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models (published 2024-12-13)

Unknown date

• Lucy — Decart's real-time image-to-video DiT (lucy.decart.ai) — Lucy — Decart’s real-time image-to-video DiT (lucy.decart.ai) (published unknown)
• Fish Audio S2 Pro — SOTA Open-Source TTS (Dual-AR + GRPO) — Fish Audio S2 Pro — SOTA Open-Source TTS (Dual-AR + GRPO) (published unknown)
• Qwen-Image — Image foundation model with text rendering and editing (QwenLM/Qwen-Image) — Qwen-Image — Image foundation model with text rendering and editing (QwenLM/Qwen-Image) (published unknown)
• Awesome Diffusion Categorized — Awesome Diffusion Categorized (published unknown)