Ambient Diffusion Omni: Training Good Models with Bad Data

TL;DR

Ambient-o is a principled framework for training diffusion models on heterogeneous data without discarding the low-quality or out-of-distribution (OOD) tail. The trick is to route each sample to the noise-level range where it is indistinguishable from the target distribution: low-quality (blurred / JPEG / motion-blurred / synthetic) data is supervised only at high diffusion times, where additive Gaussian noise contracts distribution distances, and OOD data is used only at low diffusion times via crops, where locality of natural-image statistics means small patches from cats can still teach a dog model how to refine high-frequency details. The cutoffs are set by training noise/crop classifiers on the labeled clean/corrupted subsets. Headline result: SOTA ImageNet-512 FID with EDM2-XXL backbone, plus improved diversity in text-to-image at no fidelity cost.

Key claims

Addition of Gaussian noise provably contracts the distance between the clean distribution $p_0$ and any corrupted distribution $\tilde{p}_0$ — formally $W_2(p_t, \tilde{p}_t) \leq W_2(p_0, \tilde{p}_0) \cdot D / (\sigma_t \sqrt{2})$ — so at sufficiently high diffusion times the two distributions are interchangeable for denoiser training [Theorem 4.2, §4].
The optimal noise level above which a low-quality sample is safe to use is found by training a time-conditional clean-vs-corrupted classifier under additive Gaussian noise, and reading off the minimum $t$ at which classifier accuracy drops to chance (50%) [§3.1, Eq. 3.1–3.2].
Sample-dependent annotation (per-sample noise threshold) beats fixed-threshold annotation, because the corrupted “set” is itself a mixture of clean and corrupt and per-sample assignment recovers info that fixed thresholds lose [§5, Table 2(a); Appendix Table 7].
A second key property — locality of natural images — lets OOD data be used at low diffusion times via crops: there is a mapping between diffusion time and the minimal receptive-field size needed for optimal denoising, so small patches from $\tilde{p}_0$ that match $p_0$ marginally can train the low-noise regime without global distribution match [§3.2, Eq. 3.3].
The same classifier idea applies in crop-space: train a classifier between size- $k$ crops of clean vs OOD images, find the maximum $k$ at which classifier is confused, use OOD crops up to that size at low diffusion times [§3.2, Eq. 3.4].
The “donut paradox”: each low-quality / OOD sample is usable for $t > T_{\text{anneal}}$ (after the corrupted distribution has merged into the clean one under noise) and for $t < T_{\text{crop}}$ (where local crops suffice) but not in the middle band where neither condition holds — there is a hole in the available training data along the diffusion trajectory [§3.2, Fig. 15].
Empirically validates the framework on synthetic Gaussian-blur / JPEG / motion-blur corruptions on CIFAR-10 and FFHQ, outperforming both “filter to clean only” and “use everything as equal” baselines across all corruption strengths [§5, Tables 2(a), 3, 4].
Cat images measurably improve a dog generator (and vice-versa) via crop-level training; procedurally-generated Shader images give a smaller but positive lift, demonstrating that the locality recipe extracts signal from data with no semantic overlap with the target [§5, Table 2(b)].
On ImageNet-512 the high-quality / low-quality split is not given and is constructed via CLIP-IQA (top vs bottom percentile by CLIP image quality score), then Ambient-o is run on that split. Ambient-o-XXL+crops sets new SOTA FID — better on Train FID and substantially better on Test FID, indicating less memorization from the added noise during annotation [§5, Table 1].
Limitation: the contraction argument is most effective for high-frequency corruptions (blur, JPEG, motion blur) because Gaussian noise suppresses high frequencies first by the power-law spectrum. Low-frequency corruptions (color shifts, contrast reduction, fog occlusion, large masks) require much more noise to merge with clean distribution, degenerating to filtering in the limit [§3.1].

Method

Ambient-o builds on Ambient Diffusion’s loss for learning from known-degradation noisy data [Eq. 2.2], but generalizes to unknown, arbitrary corruptions and to OOD data. Given a dataset partitioned into a small clean subset $\mathcal{D}_g$ (drawn from the target $p_0$ ) and a large corrupted/OOD subset $\mathcal{D}_b$ (drawn from a tilted measure $\tilde{p}_0$ ), the method has two stages.

High-noise regime (low-quality data). Train a time-conditional classifier $h_\phi(x, t)$ that takes noisy inputs and predicts clean-vs-corrupted, jointly over diffusion times $t \in [0, T]$ (Eq. 3.1). For each corrupted sample $x_b$ , compute the per-sample annotation time $T(x_b) = \min\{t : h_\phi(x_b + \sigma_t \varepsilon, t) \approx 0.5\}$ (Eq. 3.2). The sample is then re-noised to time $T(x_b)$ and inserted into the training set for $t \geq T(x_b)$ only, with the loss being the noisy-data Ambient Diffusion objective (Eq. 2.2 generalized — predict the clean denoiser using the partially-noised low-quality sample as the input).

Low-noise regime (OOD data via crops). Train a crops classifier that takes random size- $k$ patches and predicts clean-vs-OOD. Find the largest $k^*$ for which the classifier cannot distinguish the marginals (Eq. 3.4). For diffusion times $t \leq T_{\text{crop}}(k^*)$ — where $T_{\text{crop}}(k)$ is the empirically-measured noise level at which a $k \times k$ receptive field is sufficient for optimal denoising — OOD images can be used with the regular diffusion objective on crops of size $\leq k^*$ .

The mapping $t \leftrightarrow k$ is measured by ablating crop size against denoising loss at fixed $t$ on a pre-trained denoiser and reading off where the loss plateaus (Figs. 16–18). The combined training schedule has the “donut” structure: clean data is used at all $t$ ; low-quality data at high $t$ ; OOD crops at low $t$ .

Results

ImageNet-512 (unconditional generation, EDM2 backbone). Ambient-o-XXL+crops reaches FID 1.80 with CFG (vs EDM2-XXL 1.81) and FIDv2 32.63 with CFG (vs EDM2-XXL 33.09). Headline win is on Test FID: 2.53 with CFG (vs EDM2-XXL 2.88), a 0.35 point absolute improvement, and FIDv2 Test 45.78 vs 46.22 — explicitly attributed to less memorization because of the annotation-stage noise injection [Table 1].
Synthetic Gaussian blur on CIFAR-10. At all four corruption strengths $\sigma \in \{1.0, 0.8, 0.6, 0.4\}$ , Ambient-o beats both “Only Clean (10%)” filtering and “All data” naive training. Headline numbers (FID, smaller is better): at $\sigma=0.6$ , Ambient-o reaches 5.34 vs All-data 11.42 and Only-Clean baseline higher still [Table 2(a)].
JPEG compression and motion blur (Appendix Table 3, Fig. 12; FFHQ Table 4). Same pattern: Ambient-o beats both baselines across corruption strengths and datasets.
OOD generation, cats↔dogs. Using only 10% clean dog images, augmenting with classifier-annotated cat crops gives FID 8.92 (vs dog-only baseline higher); the symmetric cats-from-dogs experiment gets FID 4.89 using Wildlife OOD crops [Table 2(b)].
Procedural Shader images (zero semantic overlap with ImageNet) still produce a measurable FID improvement (10.98 vs filtered-only baseline) on the dog-generation task, demonstrating that even synthetic non-natural OOD data carries usable low-noise information at the crop level [Table 2(b)].
Text-to-image: authors report improvements in both quality and diversity for T2I generative modeling; full per-prompt evaluations are in the appendix (figures referenced but not in the fetched extract).

Why it’s interesting

This paper is the constructive twin of A Bitter Lesson for Data Filtering filed earlier today: both argue against filtering, but the bitter-lesson paper says “just throw scale at it and the curation gate stops mattering” while Ambient-o says “the curation gate is the wrong abstraction — assign each sample to the right noise level instead of a binary keep/drop.” The mechanism is sharper and more directly actionable: the contraction-under-noise theorem (Theorem 4.2) gives a principled reason that low-quality samples are strictly informative at high diffusion times, and the crops/locality argument supplies an analogous principle for OOD data at low times. For Luma’s data pipeline this turns a discrete filter into a continuous routing function — and re-frames synthetic data as just another kind of “low-quality data” that gets used at high noise levels (Alex’s framing). Concretely, this is a candidate recipe for using degraded video, JPEG-compressed web video, and generator-synthesized video alongside studio footage without the model collapsing to the worst-quality mode. The page complements Diffusion training efficiency‘s existing recipes (REPA/RAE/Self-Flow/UL): those address efficiency given a clean curated dataset; Ambient-o addresses efficiency given a heterogeneous one. It also stands in productive tension with L2P: Unlocking Latent Potential for Pixel Generation, which uses pure-synthetic data for transfer — Ambient-o would predict that pure-synthetic is suboptimal compared to a synthetic+real blend with synthetic routed to high-noise times.