Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis

TL;DR

DP-DMD is a step-distillation recipe for fast diffusion samplers that explicitly assigns different objectives to different distilled steps: the first step is supervised by a target-prediction (v-prediction) loss to preserve diversity, while subsequent steps use the standard DMD reverse-KL loss to refine quality. Gradients from the DMD loss are stopped at the first step, preventing its mode-seeking pressure from collapsing the first step’s role. The result is a 4-NFE student that matches DMD on quality while restoring much of the teacher’s sample diversity — without LPIPS, GANs, auxiliary networks, or extra ground-truth images.

Key claims

DMD’s reverse-KL objective inherently encourages mode-seeking, so a student distilled with DMD alone exhibits substantially reduced sample diversity vs its multi-step teacher under identical text conditioning [§1, Fig. 1].
Existing remedies (LPIPS perceptual regularizers, adversarial/GAN losses) work but add compute and training-stability cost, and the cost is often not justified by the diversity gain [§1; cf. discussion via Yin et al. 2024a, Chadebec et al. 2025, Lu et al. 2025].
A role-separated framework — first step = v-prediction (target-prediction) for diversity, later steps = DMD for quality — restores diversity without auxiliary networks [§3, abstract].
Stopping the gradient of the DMD loss at the first step is required: without it, reverse-KL pressure overrides the first step’s diversity-preserving supervision [§3].
DP-DMD has no perceptual backbone, no discriminator, no auxiliary networks, and no additional ground-truth images, making it cheaper and more stable than LPIPS/GAN-augmented DMD variants [abstract].
Across backbones, metrics, and a user study, DP-DMD reaches a better diversity-quality trade-off than LPIPS-augmented and GAN-augmented DMD variants at 4 NFEs, with SD3.5-M (30 steps, 60 NFEs) as the teacher upper bound [Fig. 1, experiments].

Method

DMD trains a few-step student generator by minimizing reverse KL between the student’s distribution and the teacher’s, via a score-distillation-style gradient that uses two score networks (real-data score and fake-data score on student samples). The mode-seeking bias of reverse KL is what costs diversity. DP-DMD keeps DMD as the backbone but decomposes the multi-step student trajectory by role:

Step 1 (diversity anchor): trained with a target-prediction objective — given a noisy latent and a sampled noise/data target, regress to the teacher’s v-prediction target. This is a forward-KL-flavored supervision that does not punish mass on non-teacher modes.
Steps 2…k (quality refinement): trained with the standard DMD reverse-KL distillation loss. These steps refine the diverse-but-coarse output from step 1 toward teacher-quality samples.
Gradient stop: the DMD loss’s backward pass is stopped before reaching step 1’s parameters, so the role separation is enforced architecturally rather than just by loss weighting.

Training runs entirely in latent space with a frozen VAE/teacher (SD3.5-M class), so memory and compute overhead is modest. No discriminator, perceptual model, or extra real-data terms are introduced.

Results

Headline qualitative result: under identical prompt + varied seeds, 4-NFE DMD produces near-identical faces while 4-NFE DP-DMD produces a visibly diverse population, with image quality competitive with both DMD and the 60-NFE teacher [Fig. 1]. Quantitatively, DP-DMD reports a better diversity-quality trade-off than LPIPS-augmented and GAN-augmented DMD on SD3.5-M-class backbones across the paper’s metrics and user studies. The authors also note that LPIPS adds modest gains at non-trivial cost and that GAN add-ons help on some axes but dent quality and training stability on others — positioning DP-DMD as a simpler operating point that dominates on the Pareto frontier.

Why it’s interesting

This is a clean instance of role-separated distillation: the insight is not that DMD is broken, but that the same student model can be trained with different objectives at different stages of its own sampling trajectory, and that this decomposition can substitute for the heavy machinery (perceptual losses, discriminators) the field has been bolting onto DMD. For Luma, the practical hook is in the slack pointer — “different kinds of algorithms to distill at different stages” generalizes beyond DMD: any few-step diffusion/flow distillation pipeline could in principle separate a coarse-diversity stage from a refinement stage and assign them different losses. Worth tracking against the from-scratch few-step line (SoFlow, MeanFlow, Shortcut Models): distillation-based DP-DMD targets the same 1–4 NFE regime but starts from a strong multi-step teacher rather than training a single-step model end-to-end.