Skip to content
Bud's Wiki

Diffusion Distillation

TL;DR

Section titled “TL;DR”

Diffusion distillation compresses a multi-step pretrained diffusion / flow teacher into a few-step (1–4 NFE) student. The 2025–2026 thread on the wiki spans four orthogonal axes: (1) objective design — linear-shortcut (DMD, consistency, rectified flow) vs non-linear (ArcFlow’s analytic mixture-of-momentums); (2) role separation across student steps — DP-DMD’s v-prediction first step + DMD later steps for diversity preservation; TMD’s outer/inner backbone-flow-head decoupling; RMD’s resolution-stage separation (low-res for high-noise semantic stage, high-res for low-noise refinement); (3) train-vs-inference alignment — TMD’s flow-head rollout inside DMD, Causal Forcing’s AR-teacher ODE init to satisfy frame-level injectivity, RMD’s logSNR-invariance mapping that distills over identical denoising states regardless of resolution; (4) what to post-train with after distillation — TDM-R1’s non-differentiable-reward RL using deterministic-trajectory unbiased per-step reward estimates. Common substrate: distillation is now the practical production path to 1–4 NFE on production-scale models (SD3.5-M, FLUX, Qwen-Image-20B, Wan2.1 14B, Z-Image 6B), while teacher-free recipes like SoFlow / MF hold the from-scratch frontier on smaller image benchmarks.

Key claims

Section titled “Key claims”

Recent contributions

Section titled “Recent contributions”
  • [2026-11-05] DreamFree: Future Representation Distillation for Imagination-free World Action Models: DreamFree distills future-aware representations (not samples or scores) from a frozen video-diffusion teacher into a student visual encoder that only observes the current frame — a distinct distillation target from both step-distillation (DMD, F3A) and sample-level distillation (DreamZero-Flash’s single-step action rollout). Teacher sees current + future frames; student sees only current; alignment loss on internal features; both are the same video-diffusion WAM family (shared-author lineage with DreamZero). Combines with standard video generative pre-training on the student. At inference, student encoder features feed a readout + action decoder — teacher and rollout are both dropped. Delivers 17.8× end-to-end speedup over the World Action Models are Zero-shot Policies (DreamZero) DreamZero full-WAM baseline at the same 14B backbone (1,632 → 91 ms per action chunk on H200, 11 Hz closed-loop), with real-world DROID success improving 56.0 → 64.0% rather than degrading as typical step/sample distillation does. First filed WAM-regime datapoint where a representation-distillation target, not a step or sample target, is the serving-latency lever.
  • [2026-11-04] AIRoA-0: Long-Horizon, Precise, Cross-Embodiment Manipulation from Handheld Data: AIRoA-0’s two-pass async denoising buffer is a distillation-adjacent inference-time lever on a flow-matching π0.5-shape action head: amortize a ten-step denoising process across queries by running a 2-chunk rolling buffer where each query emits one single-step denoise (level 1.0 → 0.1) on a fresh chunk and refines the prior chunk to clean prediction, so each query only triggers two network passes. Sibling in spirit to the training-time few-step distillation line but moves the count-reduction into the scheduler across queries rather than the model’s learned denoising trajectory.
  • [2026-10-30] Alex Ksendzovsky — Mischief 1 biocompute image-to-video model launch: Mischief 1 (BioComputingCo) claims “at least 5× faster than its base” for a commercial image-to-video deployment; base model and acceleration recipe are undisclosed, so whether this is step distillation, sparse attention, hardware substrate (biocompute), or a mix is open — filed as a market-side datapoint that the 5× speedup bar is now the public pricing wedge on fal/OpenRouter.
  • [2026-10-29] Pika H3 Flex — open-weight joint audio-video model post-trained on MiniMax H3: Pika H3 Flex (Pika Labs) — faster-than-realtime generation (15 s + audio in ~7 s) on an open-weights MiniMax H3 derivative. Launch tweet does not disclose which acceleration lever (step reduction, sparse attention, low-precision kernels) produces the speedup; filed here speculatively alongside the other H3 accelerators (FastVideo FastH3 V1 — 4-Step Sparse Distilled MiniMax H3 for 14x Blackwell Speedup, TaoMate-H3 — Low-Latency Streaming LoRA Distillation of MiniMax H3 for Joint Audio-Video) pending further technical disclosure.
  • [2026-10-22] Interactive World Simulation with Cosmos for Physical AI: Cosmos Simulation (NVIDIA) stacks causal consistency distillation (Stage 3) and DMD-with-Self-Forcing (Stage 4) in a single post-training pipeline on Cosmos 3, matching the DMD/CD split that Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation names as the right staging (CD for mode-covering initialization, DMD for mode-seeking polish). First filed open-weight release of the full four-stage pipeline (bidirectional adapt → teacher-forced AR → causal CD → DMD-with-self-forcing) at frontier WFM scale with three specialized endpoints.
  • [2026-10-19] World Action Models are Zero-shot Policies (DreamZero): DreamZero-Flash (companion to the main DreamZero WAM) demonstrates single-step inference without performance collapse on a 14B autoregressive video-diffusion backbone emitting joint video + action predictions, enabling 7 Hz closed-loop robot control. Early datapoint that distillation of video-diffusion WAMs down to few/single-step inference preserves control performance even at 14B scale — complementary to the broader diffusion-distillation line but in the uncommonly difficult setting of a jointly-trained video+action backbone deployed as a real-time policy.
  • [2026-10-16] Are We Overthinking One-Step Generation?: Priming Models (Park, J. Kim, Du, Chen, Han) is explicitly positioned as the teacher-free counterpart to the DMD / TMD / TDM / RMD / consistency-style distillation lineage: no diffusion teacher, no flow map, no CFG, no JVP — just reconstruction pretraining at a single fixed noise level, then feature-space distributional post-training. ImageNet 256² one-step FID 1.19 at ~1/10 the training compute of comparable one-step generators, with the central claim that reconstruction pretraining alone eliminates the historical difficulty of training distributional objectives (GAN / Sinkhorn / Wasserstein) from scratch. Sharpens the from-scratch-vs-distillation Pareto for anyone evaluating whether to distill an existing teacher or train one-step from scratch at image-class scale.
  • [2026-10-13] Velocity Scaling in Flow Matching: Velocity Scaling in Flow Matching (Saied & Fleuret, arXiv 2610.10823) is an inference-time alternative to distillation for lowering NFE quality loss: a scalar gain γ(t) multiplying the velocity, chosen to minimize population time lag measured by a tiny time-probe, cuts SiT-XL/2 FID at NFE 25 from 28.0 to 12.2 without any student training; pushing γ past lag correction sharpens covariance further to 8.7. Importantly argues the prior “MSE underestimates velocity magnitude” motivation is wrong — the real effect is discretization-induced distribution-time lag.
  • [2026-10-12] Few-Step Generation via Data-Space Iteration: Lin, Peng, Wang, Fan (ByteDance) replace flow-space iteration inside DMD with data-space iteration: a shared generator refines its data-space prediction directly across N steps starting from noise, with no timestep schedule at all. Drops into DMD with no training-code changes, so it isolates iteration method as the sole variable; on class-conditional ImageNet 256×256 it matches or beats discretized flow-space baselines — including variants selected via schedule search — without doing any schedule search. A different (and more foundational) attack on the discretization axis than ArcFlow’s closed-form non-linear velocity or TMD’s flow-head rollout, both of which still iterate in flow space.
  • [2026-10-08] Meet Odyssey-3: Our Most Powerful Foundation World Model: Odyssey discloses its real-time interactive-WFM path uses distribution-matching + adversarial distillation to compress a multi-step video DiT (teacher-forced autoregressively with causal masking) into a few-step variant that supports real-time environment generation in the research preview. First filed closed-flagship interactive-WFM to explicitly name the DMD + adversarial recipe (joining Self-Forcing++, LongLive-2.0, Causal Forcing, and Phased DMD on the cluster’s recipe-template board for interactive-WFM distillation).
  • [2026-10-06] RealtimeWAM: One-Step Asynchronous World Action Models: RealtimeWAM’s Teacher-Anchored Consistency Distillation (TACD) attacks the local–global error gap of one-step consistency distillation on the action branch of a Mixture-of-Transformers World Action Model: the standard local consistency objective is supplemented by explicit supervision from a frozen multi-step teacher’s rollout endpoint, so single-step student actions are anchored against the actual multi-step inference-time target rather than only against local trajectory consistency. Sibling in framing to Transition Matching Distillation for Fast Video Generation TMD’s flow-head rollout inside DMD (which closes the same train/inference gap for video via multi-step student rollout) but attacks the dual: anchor against the teacher’s multi-step endpoint instead of simulating the student’s. Composed with inter-expert pipelining (CEWP), delivers ~25× speedup at <1% task-success drop across LIBERO, LIBERO-Plus, and RoboTwin on Fast-WAM / Faster-WAM — first filed application of the TMD-style train/inference-gap framing to WAM action distillation specifically.
  • [2026-10-05] Rho-1: Collapsing the Multimodal Stack: Rho-1 (Reka Labs, 19B) uses a 99→8 step distillation on its generation-stream denoising trajectory as the specific lever that moves a 0.79× real-time base rollout into the interactive-steering regime — the distillation payoff here is not throughput for batch video synthesis but the latency floor required to let text instructions arriving mid-rollout update the shared KV cache and have the generation stream render the new trajectory without a cut. Also claimed to let the system return a 5.3-second video clip in ~1 second with “minimal quality loss”; distillation recipe details, step-count ablation, and quality numbers are not disclosed.
  • [2026-10-02] Veda: Scalable Video Diffusion via Distilled Sparse Attention: Veda extends what distillation targets inside a video DiT — not step count (DMD/TMD/RMD) or decode (PiD), but the tile-level attention distribution itself. The estimator is supervised with row-wise KL against max-pooled full attention before top-K thresholding, aligning tile-score geometry on the simplex rather than Euclidean scores; oracle-tile recall jumps 34.2% → 66.4% at matched 90% sparsity budget vs average pooling.
  • [2026-09-30] Sol-H3: Recursive Self-Improvement for MiniMax-H3 Inference Acceleration on Sol-Engine across Cloud and Edge: Sol-H3 is not a distillation recipe but its cross-resolution two-stage scheduler with a learned latent-to-latent mapping is the closest inference-time cousin to RMD’s Cross-Resolution Distribution Matching Distillation (Cross-Resolution Distribution Matching for Diffusion Distillation) — same coarse-to-fine cascade motivation, same VAE-round-trip elimination, but the resolution bridge is trained as a mapping module and composed with kernel-level RSI at inference rather than folded into a few-step distillation objective. Filed as a pointer for future work that might merge Sol-H3’s latent bridge with RMD-style step distillation on the H3 base.
  • [2026-09-23] FLUX 3 Action: A 7B World Action Model for Robot Control: FLUX 3 Action stacks guidance distillation (student trained to produce guided prediction directly, 1.8–2× speedup at 0.6–1.08 pp gain — outperforms teacher) with step distillation (4→1 sampling step from guidance-distilled checkpoint, 3.15–4× additional speedup at 3.51–4.32 pp cost) on top of FP8 (+0.05 to -0.76 pp) to convert a 7B joint video+action WAM into a real-time robot policy. End-to-end result: 32.29 ms per prediction in FP8 vs 246.42 ms for the BF16 4-step guidance-on baseline (~7.6× wall-clock), while preserving joint video+action denoising and setting SOTA on the RoboLab leaderboard at 42.92% (beats Cosmos 3 Nano 36.8% at less than half the parameters). Concrete existence proof that the diffusion-distillation stack that shipped for T2I/T2V (DMD, Self-Forcing, etc.) transfers cleanly to action-conditional world models — the WAM’s imagination path survives 4→1 step compression at only 3.51–4.32 pp task-success cost.
  • [2026-09-22] WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory: WorldCrafter uses few-step distillation as the load-bearing component that turns its jointly-trained video-generator-plus-implicit-3D-memory stack into a streaming minute-scale world model — another instance of few-step distillation as an enabler for interactive/long-horizon video WFM inference (sibling to Self-Forcing++, Rolling Forcing, PackForcing).
  • [2026-09-14] TaoMate-H3 — Low-Latency Streaming LoRA Distillation of MiniMax H3 for Joint Audio-Video: TaoMate-H3 distills MiniMax H3 (33B dense single-stream joint-AV DiT) via a rank-128 / alpha-128 LoRA adapter (step-3000 generator EMA) that runs three Stage3 denoising intervals per chunk. Third distillation recipe filed on the H3 base, sitting between FastH3’s 4-step VSA data-free distillation and VideoDeltaNet’s 8-step DMD2 — and the first LoRA-only distilled variant (base weights untouched, only rank-128 adapter shipped). Combined with streaming chunked emission for a 27.66× first-chunk latency speedup vs the dense denoising trajectory.
  • [2026-09-03] VideoDeltaNet on MiniMax H3 — Near-Lossless Hybrid Attention for Video Models: VideoDeltaNet composes 8-step DMD2 LoRA distillation (from LarryVRH’s community MiniMax-H3 turbo LoRA) with a hybrid-attention architectural retrofit rather than distilling more aggressively. Explicitly avoids 4-step distillation (contrast: FastH3’s 4-step FastVideo FastH3 V1 — 4-Step Sparse Distilled MiniMax H3 for 14x Blackwell Speedup) to prioritize quality over latency, arguing the architectural speedup (2.65× per-layer via VDA + kernels + FP8) shoulders most of the compounding — leaving distillation to buy the last few multipliers while keeping per-sample fidelity high.
  • [2026-09-03] FastVideo FastH3 V1 — 4-Step Sparse Distilled MiniMax H3 for 14x Blackwell Speedup: FastH3 Preview v1 is the first filed 4-step data-free sparse-attention distillation on a frontier open joint text-to-video-and-audio model (MiniMax H3, 33B dense single-stream Omni-Transformer). Data-free means the student is trained from prompts without target videos. Distilled weights are calibrated to the VSA-H3 (Video Sparse Attention) kernel backend — dense attention is not a drop-in substitute. Recommended 4-step VSA / Data-Free checkpoint is the primary release; dense-attention and synthetic-data ablation checkpoints ship alongside. Extends the FastWan sparse distillation lineage from Wan-family V-only models to H3’s joint-AV modality. Whether 4 steps preserve audio-visual sync is not quantitatively answered.
  • [2026-09-03] SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models: SolarWM applies distribution matching distillation as the third stage of a shared causal-video pipeline across four heterogeneous backbones (Wan2.2, LTX-2.5, MiniMax-H3, 5B–33B) — a datapoint that DMD as the final polish step transfers across backbones once earlier stages have converted a bidirectional teacher into a causal AR teacher.
  • [2026-08-26] On-Policy Self-Distillation in Diffusion Models: DiffusionOPSD merges distillation and RL alignment into a single outer loop: reward gradients construct bounded positive/negative targets around a frozen behavior policy’s clean-output prediction, the trainable policy fits them under detached supervision, then EMA refresh. Notably works on top of an already step-distilled backbone (Z-Image-Turbo) with larger relative wins (63% GPU-hour reduction vs 40% on SD 3.5-M), suggesting distillation-then-alignment composes additively rather than substituting.
  • [2026-08-14] DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation: DreamX-Phi uses distribution-matching distillation to compress its multi-step action-conditioned video WM teacher into a few-step deployable student — DMD applied at the robotic-manipulation-WM layer of the stack.
  • [2026-08-10] Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models: Dyna-2 reformulates one-step video distillation as a control problem: the teacher target is not fixed but tracks the student’s own reachable band throughout training, addressing the “locally straight, globally curved” teacher-flow failure that breaks image-side one-step generators when transplanted to video (MeanFlow etc. produce static videos). Reports 1 NFE at 110 ms on one H100 for a 3s 3-view robot manipulation video (93× teacher wall-clock speedup) with FVD 121 vs teacher 80 and DMD2-1step 599; motion coverage 75% vs teacher 94% and DMD2 56%.
  • [2026-08-07] Wan-Animate-2: End-to-End Character Animation with Text-Driven Viewpoint Control and Real-Time Streaming: Wan-Animate-2-Lite applies a three-stage recipe — teacher-forcing pretraining with an error-buffer mechanism, then Self-Forcing distillation with chunk-wise backpropagation — to reach real-time streaming for driving-video-driven character animation. Chunk-wise backprop is presented as the ingredient that makes Self-Forcing tractable at streaming granularity for this setting.
  • [2026-07-31] Towards Video World Models: Xun Huang position essay for video world models argues few-step distilled diffusion is not just a serving-speed optimization but the only path to real-time interactive video WMs — a non-causal T2V model generating t-second segments has minimum latency t regardless of throughput. Names the CausVid → Self-Forcing → APT2 lineage as the load-bearing recipe: AR-DiT trained under DMD (CausVid), further improved by simulating the inference-time autoregressive rollout with KV caching during training (Self-Forcing, 17 FPS + sub-second latency, single GPU), then interactive finetuning of the distilled base (APT2).
  • [2026-07-29] Wonder: Video World Model Done Better: Wonder identifies control-signal drift and diversity collapse as the specific failure modes of self-forcing distillation, and rectifies them with two additions: a Mixture-of-Students design (broadens student capacity to match teacher diversity) and a GAN Control Regularization loss (keeps camera-control fidelity from decaying during distillation).
  • [2026-07-24] A2A: Action-to-Action Flow Matching: A2A (RSS 2026) is a from-scratch counter-recipe to teacher-based few-step distillation for robot policies: rather than compress a many-step flow-matching teacher (TMD / TDM-R1 / DP-DMD / RMD / DiagDistill), shorten the flow trajectory itself by initializing from the previous proprioceptive action instead of Gaussian noise. Reports 0.56 ms single-step inference at ≥90% success and beats multi-step DDPM baselines at 40–100 NFE on 5 Robosuite tasks. Same “adjacent latents are close, so the flow between them is short” principle Causal Forcing / DiagDistill exploit for AR video — now applied to actions via proprioceptive continuity.
  • [2026-07-23] Lucy 2.5 Real-Time AI Video Editing: Decart's 40ms Live Stream Revolution: Lucy 2.5 (Decart, community writeup) — real-time video editor at sub-40 ms/frame / 30 FPS / 720p. Recipe details not disclosed, but latency budget implies few-step distillation on top of the MirageLSD Shortcut-Distillation lineage now applied to editing (source-conditional) rather than pure stream generation.
  • [2026-07-22] Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing: Mage-Flow (Microsoft, 4B) ships a 3-stage post-training recipe applied identically to generation and editing branches: rectified-flow-matching Base → Diffusion-NFT RL alignment → 4-step Turbo distilled with adversarial perceptual guidance. Datapoint that a 4-step distilled student at 4B (Mage-Flow-Turbo) can beat 9B–20B teachers (Qwen-Image 20B, FLUX.2-Klein 9B) on GenEval (0.88) and CVTG-2K (0.873), reinforcing the “compact + distilled” over “large + few-step” line at production quality.
  • [2026-07-21] AlayaWorld: Interactive Long-Horizon World Modeling: AlayaWorld’s Stage-3 post-training combines three distillation objectives in one pipeline — DMD, Self-Forcing++, and consistency distillation — reducing inference from ~30 to 4 steps per chunk while preserving the complete control + memory stack. First filed system combining all three, and a concrete answer to Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation‘s DMD-vs-CD split (CD for initialization, DMD for polish) as co-objectives rather than staged alternatives.
  • [2026-07-21] DriftWorld: Fast World Modeling through Drifting: DriftWorld (Yilun Du et al.) is the first filed application of Kaiming-lab drifting to a robotics world model — teacher-free one-step generation extended from images to action-conditioned video, reporting 17× faster average rollout vs diffusion-based WFM baselines and 1–2 GPU training cost. Complements the DMD/phased-DMD teacher-based branch by demonstrating drifting scales to video-sequence outputs.
  • [2026-07-20] Introducing Cosmos 3 Edge: Alongside Cosmos 3 Edge, NVIDIA releases a Cosmos 3 Super 4-Step Distillation checkpoint (T2I + I2V) — distribution-matching distillation reducing 35–50 denoising steps to 4 for up to 25× faster inference, with the post-training script released so developers can rerun the distillation on their own domains.
  • [2026-07-16] FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space: FlowDAgger ports the flow-matching inversion primitive (reverse-time ODE integration + local refinement to recover a source noise from a target sample) out of the image-editing setting and into robot-policy adaptation: each human corrective action a* is mapped to the noise ẑ* that the frozen flow-matching policy would have integrated forward into it, and those (o, ẑ*) pairs supervise a small latent policy that steers the base sampler at deployment. A wiki-first datapoint that noise-space inversion is a stable supervision signal beyond diffusion editing — with the frozen base policy playing the role of the frozen image DiT and the latent policy playing the role of a tiny steering adapter.
  • [2026-07-10] LingBot-VA 2.0: Native Video-Action Pretraining for Generalizable Robot Control: Few-step consistency distillation is applied to both the video expert and the action expert in a Mixture-of-Transformers video-action model, and stacked with quantization, sparse MoE routing, and asynchronous execution to reach a reported 225 Hz peak inference frequency on real robot hardware — one of the strongest filed cases of distillation as a load-bearing lever for real-time closed-loop VLA/WFM deployment (not just for image/video generation latency).
  • [2026-07-09] LingBot-World 2.0 / LingBot-World-Infinity — Infinite Worlds with Versatile Interactions: LingBot-World 2.0 distills a causal-fast real-time student from a causal-pretrained base to hit 720p / 60 fps interactive throughput — a step up from v1’s 480p / 16 fps LingBot-World-Fast. Method details not yet disclosed (tech report URL unresolvable at filing); positioned as the throughput datapoint that lets the open Wan-based interactive-WFM stack quote higher fps/resolution than DeepMind’s closed Project Genie surface.
  • [2026-07-04] RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation: Step Forcing distills an AR video diffusion world model to a 4-step schedule by supervising denoising from one-step self-forwarded priors under the same schedule used at inference, avoiding the expense and action-following degradation of Self Forcing.
  • [2026-06-29] Qwen-Image-2.0-RL Technical Report: Qwen-Image-2.0-RL uses on-policy distillation with trajectory-level velocity matching as the final merge step of a multi-task RL pipeline — consolidating a T2I GRPO specialist and an edit GRPO specialist into a single deployable student. New use case for OPD in the diffusion stack: not step-count reduction or quality lift over a single larger teacher, but multi-policy consolidation after task-specific RL.
  • [2026-06-23] Causal World Modeling for Robot Control (LingBot-VA): Noisy History Augmentation is a non-distillation route to inference-step reduction: instead of training a separate few-step student, the model is trained with video history corrupted to a sampled flow-time t (using the same flow-matching interpolation x_t = (1-t)·noise + t·x_0) so the inverse-dynamics action decoder learns to read actions from partially denoised video latents. At inference the video need only be denoised to t* (not 0), roughly halving per-step denoising work — without retraining a student. Structurally analogous to Causal Forcing’s distillation trick but applied at the action-decoder readout rather than the video velocity field.
  • [2026-06-18] Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation: DFD (NVIDIA + Michigan + Toronto): one-line post-training fix for DMD/DMD2 that mixes real-data samples into the teacher-score evaluation roughly half the time, turning DMD’s reverse-KL student-only gradient into a teacher-score discrepancy that pulls the student toward missed modes and away from spurious ones — restores diversity and resolves over-saturation in 100–300 finetune steps on Wan2.1-1.3B and Cosmos-Predict2.5-2B (T2V, I2V, and AR video), with the claim that explicit real-data regularization is stronger than DMD2’s implicit GAN regularization. A pure-objective counterpart to DP-DMD’s role-separated-student fix for the same DMD failure mode.
  • [2026-06-03] Bootstrap Your Generator: Unpaired Visual Editing with Flow Matching: BYG (NVIDIA, ICML 2026): bootstraps an unpaired editor out of a pretrained flow-matching generator by manipulating its velocity field around the target-style manifold, rather than fitting on paired (source, edit) data — reframes editing as a flow-trajectory bootstrap problem.
  • [2026-06-02] Stable-Layers: Fine-Tuning Image Layer Decomposition Models with VLM-Scored Reinforcement Learning: Stable-Layers post-trains a diffusion-based layer decomposer with Flow-GRPO + LoRA using VLM-only rewards — a flow-matching RL fine-tune (no distillation step shortening) but in the same Flow-GRPO family that ties this cluster to RL alignment for flow models.
  • [2026-06-01] Pre-training under infinite compute: Stanford CRFM — LLM-pretraining analog of the ensemble-and-distill recipe. Distilling an 8-ensemble of 300M models into a single 300M student via sequence-level KD retains 83% of the ensemble benefit, outperforming the regularized recipe’s N→∞N\to\infty asymptote at finite inference cost. Self-distillation (300M teacher → 300M student of identical architecture, real + synthetic token mixture) beats the teacher — concrete recipe for the Distillation Scaling Laws thesis that distillation pushes the loss-vs-parameter curve to the left.
  • [2026-06-01] RenderFlow: Single-Step Neural Rendering via Flow Matching: RenderFlow (Disney Research, CVPR 2026): single-step neural rendering trained directly with a flow-matching objective on (G-buffer, photoreal RGB) pairs — a from-scratch single-step generator targeting graphics rendering rather than image synthesis. Complements teacher-student distillation recipes (TMD, DMD, ArcFlow) with a domain where deterministic single-step inference is the requirement, not a speedup target, because stochastic samplers produce temporal jitter when applied frame-by-frame.
  • [2026-05-30] Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE: Mamoda2.5 reports a joint few-step distillation + RL framework compressing a 30-step video-editing teacher to a 4-step student with up to 95.9× inference speedup over open-source baselines, applied to a 25B/3B-active DiT-MoE.
  • [2026-05-29] MRT: Masked Region Transformer for Layered Image Generation and Editing at Scale: MRT distills its 20B masked region diffusion model for multi-layer transparent image generation down to 8 steps with “minimal quality degradation,” contributing a 20B-scale data point for distillation in the layered-image regime.
  • [2026-05-25] PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion (PiD, NVIDIA): DMD2-distill a pixel-diffusion decoder (not a generator) from multi-step to 4 NFE. The decoder operates in pixel space at the target upsampled resolution (e.g., 2048×2048) and conditions on a noise-corrupted latent via a sigma-aware adapter. Distillation outcome: 4-step inference, ~210 ms on GB200 / sub-1 s on a consumer RTX 5090 at 13 GB peak memory for the 512² → 2048² decode, ~6× faster than cascaded diffusion-based super-resolution pipelines (and 5.9× over SeedVR2 on the same task) with better visual fidelity per project-page side-by-sides. New target for DMD2 distillation: the decode+upsample step rather than the generative diffusion itself.
  • [2026-05-23] Streaming Autoregressive Video Generation via Diagonal Distillation (DiagDistill, ICLR 2026): per-chunk asymmetric step schedule for AR video distillation (more steps early, fewer late, e.g. 4-3-2-2-2-2-2 over 7 chunks). Three pieces: (a) Diagonal Denoising schedule itself; (b) Diagonal Forcing — conditioning the next chunk on a noised version of the previous chunk’s output during training (t_d≈100 / 1000 optimal) to match inference-time KV-cache state and eliminate the implicit “next-noise-level prediction” that drives over-saturation drift; (c) Flow Distribution Matching — a flow-field reverse-KL on top of DMD using a learnable latent-frame-difference module, restoring motion amplitude lost at low NFE. On Wan2.1-T2V-1.3B: 31 FPS, 0.37 s first-frame latency, VBench Total 84.48 — 277.3× faster than the Wan2.1 teacher while slightly exceeding it on Total and Semantic. Ablations show Diagonal Denoising buys the speedup (quality stays at Self-Forcing level when removed), while Diagonal Forcing + Flow Distribution Matching buy the quality.
  • [2026-05-23] Cross-Resolution Distribution Matching for Diffusion Distillation (RMD, Huawei Cloud + NJUST): cross-resolution DMD — partitions the teacher’s logSNR trajectory into resolution-specific segments (low-res for high-noise / semantic stage, high-res for low-noise / refinement), derives student timestep intervals via an SD3-style logSNR-shift compensation so distillation occurs over matched denoising states across resolutions, and upsamples the student state into the teacher’s high-res space with a mixture of predicted noise (inherits teacher ODE trajectory) and Gaussian noise (bridges large resolution gaps). Two-stage (lo+hi) recipe on SDXL @ 4 NFE: 33.4× speedup and beats DMD2 / TDM / SDXL-Turbo / SDXL-Lightning across HPS/AeS/CLIP. On Wan2.1-T2V-14B: 25.6× speedup and VBench 82.51 vs 6-NFE TDM 80.48 / 6-NFE DMD2 80.30 — the first wiki datapoint where cascading resolution beats fixed-resolution distillation at fewer effective NFE on a 14B video teacher. The component ablation shows the cross-resolution RM objective (not the cascaded upsampling alone) is what does the lifting; UP without RM underperforms the baseline.
  • [2026-05-23] TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward (TDM-R1, ICML 2026): RL post-training of a TDM few-step student with non-differentiable rewards (GenEval / OCR / HPSv3). Hinges on the unbiased-per-step- reward property of deterministic ODE trajectories; introduces a diffusion-parameterized Surrogate Reward with group-BT learning and EMA reference, and a marginal-level (not trajectory-level) reverse-KL to the TDM teacher. SD3.5-M @ 4 NFE → GenEval 0.92; Z-Image @ 4 NFE strictly dominates 100-NFE Z-Image and Z-Image-Turbo on every reported metric.
  • [2026-05-23] Self-Flow: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis: internal self-distillation (EMA teacher inside the same generator, Dual- Timestep Scheduling) as an alternative to external DINO-based REPA. Not a teacher-student distillation between two diffusion models, but the same conceptual lever applied within one model; filed here because the “self-distillation as supervision” framing overlaps with the multi-step→few-step thread.
  • [2026-05-23] ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation (ArcFlow): non-linear-flow 2-step distillation; analytic mixture-of-continuous- momentum velocity gives closed-form ODE integration. <5% of parameters trained (rank-256 LoRA on FFN + output head). Beats pi-Flow / TwinFlow / SenseFlow / Qwen-Image-Lightning at 2 NFE on FLUX.1-dev and Qwen-Image-20B; +85.7% diversity over Qwen-Image- Lightning on OneIG-Bench.
  • [2026-05-23] Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis (DP-DMD): role-separated student where step 1 uses v-prediction (diversity) and steps 2..k use DMD (quality), gradient stop between them. No perceptual / adversarial / extra-real-data machinery — beats LPIPS-DMD and GAN-DMD on the diversity-quality Pareto at 4 NFE.
  • [2026-05-23] Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation (Causal Forcing++): replaces offline PF-ODE trajectory generation with causal consistency distillation; scales the Causal Forcing pipeline to 1- and 2-step frame-wise real-time interactive video.
  • [2026-05-23] Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation (Causal Forcing): theoretical correction to Self-Forcing / CausVid — bidirectional teacher → AR student violates frame-level injectivity required by ODE distillation. Three-stage fix: teacher-forced AR diffusion fine-tune → causal ODE distillation from the AR teacher → asymmetric DMD. +19.3% Dynamic Degree on real-time-streaming T2V at matched compute.
  • [2026-05-22] Transition Matching Distillation for Fast Video Generation (TMD, NVIDIA + NYU): distills Wan2.1 1.3B / 14B T2V into ≤4-NFE students. Two ideas: (a) decouple the student into a heavy semantic backbone (run once per outer transition) and a recurrent flow head (run for several inner flow updates); (b) close DMD’s train/inference gap by rolling out the flow head during DMD gradient computation. MeanFlow-style stage-1 init for the flow head outperforms vanilla flow-matching init.

Open questions

Section titled “Open questions”
  • Is the “deterministic trajectory” property the actual gating condition for non-differentiable-reward RL, or just the cleanest one? TDM-R1 leans on TDM specifically. ArcFlow, TMD, DP-DMD all use deterministic ODE sampling at inference — can the TDM-R1 recipe drop in on top of each? Predicted yes (Eq. 6 of TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward is generic in the trajectory’s determinism), but untested. RMD’s cascaded trajectory adds resolution transitions to this question.
  • What is the right marginal-KL anchor for from-scratch few-step models (SoFlow: Solution Flow Models for One-Step Generative Modeling, Self-Evaluation Unlocks Any-Step Text-to-Image Generation)? TDM-R1’s KL is to a pretrained TDM teacher; from-scratch generators have no such anchor. Either the recipe needs a teacher-free regularizer (the generator’s own EMA?) or it doesn’t transfer cleanly.
  • Linear vs non-linear vs role-separated vs resolution-separated vs chunk-asymmetric objective — which matters most at each NFE? ArcFlow (non-linear), DP-DMD (role-separated across student steps), RMD (resolution-separated across trajectory segments), and now DiagDistill (asymmetric across AR chunks) all report wins on FLUX-class or Wan-class models at 2–4 effective NFE. None of them compare against the others directly; the field is missing a clean ablation that decouples trajectory parameterization from per-step objective from resolution schedule from per-chunk schedule.
  • Are the cascade ideas composable? RMD cascades resolution along the noise axis; DiagDistill cascades step count along the AR-chunk axis. The two cascades are orthogonal and should compose trivially on Wan2.1-T2V-14B; no one has tried.
  • Architectural-gap fixes (Causal Forcing’s three-stage AR teacher), train/inference-gap fixes (TMD’s flow-head rollout in DMD; DiagDistill’s Diagonal Forcing for the AR-chunk-specific version), trajectory-parameterization fixes (ArcFlow’s closed-form non-linear ODE), and resolution-cascade fixes (RMD’s logSNR-invariance mapping) all appear to attack independent failure modes. Are they composable? In principle yes; nobody has combined two yet. RMD + TMD’s flow head on Wan2.1-14B is an obvious next target; DiagDistill + Causal Forcing/++ on AR video is another.
  • Does TDM-R1 generalize to video? Z-Image is image-class, 6B. The natural target is Wan2.1 14B via TMD’s flow head + a video reward (VBench, ViewCrafter geometry score). Open whether the unbiased-per-step-reward argument survives the much higher-dimensional reward landscape of video. DiagDistill’s Flow Distribution Matching is a non-RL prior for the motion component of that reward and may compose.
  • Distillation + RL vs distillation-of-RL. TDM-R1 ablates against “first RL the many-step teacher with DGPO, then distill with TDM”: it converges fast but plateaus because the teacher caps the student. The clean composition appears to be RL during distillation, with a jointly-trained reward — but this is only one ablation, on SD3.5-M.
  • Does distilling the decoder (PiD) compose with distilling the generator? PiD’s 4-step DMD2 decoder runs on top of an otherwise normal LDM. The total wall-clock saving is largest if the generator is also distilled (DMD2 / pi-Flow / ArcFlow). PiD additionally allows the LDM to exit early via the sigma-aware adapter, which interacts nontrivially with the generator’s distilled step schedule. Whether the combined system beats either component in isolation, and where the right exit point is, has not been measured.

Papers

Section titled “Papers”

2026-05

Section titled “2026-05”

2026-04

Section titled “2026-04”

2026-03

Section titled “2026-03”

2026-02

Section titled “2026-02”

2026-01

Section titled “2026-01”

2025-12

Section titled “2025-12”

2025-11

Section titled “2025-11”

2025-10

Section titled “2025-10”

2025-09

Section titled “2025-09”

2025-08

Section titled “2025-08”

2025-07

Section titled “2025-07”

2025-06

Section titled “2025-06”

2025-05

Section titled “2025-05”

2025-04

Section titled “2025-04”

2024-12

Section titled “2024-12”

Unknown date

Section titled “Unknown date”