The Geometry of Noise: Why Diffusion Models Don't Need Noise Conditioning

TL;DR

A Google-Research theory paper that explains why “blind” / noise-conditioning-free diffusion models (Equilibrium Matching, blind DDPM, Flow Matching without t-embedding) can work at all despite the fact that the optimal denoising direction is supposed to depend on the noise level. The authors formalize a Marginal Energy landscape and show that autonomous (time-invariant) vector fields implement a Riemannian gradient flow on it whose local conformal metric perfectly cancels the energy’s singularity at the data manifold. From the same framework they derive a sharp practical claim: among the three standard parameterizations, only velocity prediction (Flow Matching / EqM) is structurally stable when noise conditioning is removed; noise-prediction (DDPM/DDIM) is provably unstable because a “Jensen Gap” gets amplified by a divergent gain, and signal-prediction (EDM) is stable but for a more delicate reason.

Key claims

Autonomous (noise-agnostic) generative models implement a Riemannian gradient flow on the Marginal Energy $\mathcal{E}_M(x) = -\log \int p_t(x)\,p(t)\,dt$ , with the learned vector field equal to the natural gradient of this energy plus a transport correction term [§4, §5].
The Marginal Energy landscape has a genuine geometric singularity near the data manifold: its gradient diverges at rate $\propto 1/\sigma_t^2$ , forming an infinitely deep potential well that classical gradient descent could not stably follow [§4.1, Eq. 12, Fig. 1].
The autonomous field’s “effective gain” vanishes at exactly the rate needed to cancel this singularity, so the product (gain × singular gradient) stays bounded — the Riemannian preconditioning is what makes blind diffusion converge stably [§5.1, §5.3].
The transport-correction term vanishes in two asymptotic regimes: (a) high ambient dimension with low intrinsic data dimension — high-D concentration makes the input $x$ a deterministic proxy for the noise level $t$ (§5.2); (b) the near-manifold limit $\sigma_t \to 0$ — the noise-level posterior collapses to a Dirac at the smallest scale (§5.3). Note (b) does not require high ambient dimension.
Noise-prediction parameterizations are structurally unstable under autonomous (blind) generation: the effective gain scales as $1/\sigma_t$ , which amplifies a finite “Jensen Gap” between the harmonic mean of noise levels and the true level, so the drift perturbation error diverges as $\sigma_t \to 0$ [§6, Table 2].
Velocity-prediction (Flow Matching, EqM) is inherently stable: the effective gain is $\Theta(1)$ — bounded — so posterior uncertainty over noise levels is absorbed into a smooth geometric drift and the perturbation error stays bounded [§6, Table 2].
Signal-prediction (EDM) is also stable, but via a different mechanism: the gain diverges polynomially ( $1/\sigma_t^2$ ) while the signal-estimation error vanishes exponentially fast near discrete data, so exponential decay beats polynomial divergence [§6, Table 2].
The empirical predictions match: a DDPM-blind ResNet-UNet on CIFAR-10 / SVHN / Fashion-MNIST fails to generate coherent samples (dominated by high-frequency artifacts), while a Flow-Matching-blind model with the same backbone produces samples qualitatively comparable to its noise-conditional counterpart [§7.1].
On a controlled 2D-embedded-in- $\mathbb{R}^D$ toy experiment, the predicted three regimes appear as $D$ grows: (i) low-D blind failure for both DDPM and FM, (ii) moderate-D where FM-blind resolves first while DDPM-blind shows residual noise from gain amplification, (iii) absolute-concentration regime where even DDPM-blind succeeds because estimation error drops to zero faster than the gain diverges [§7.2, Fig. 5].
On CIFAR-10 against the Sun et al. blind-diffusion benchmark, DDIM-blind hits FID 40.90 (structurally broken) while velocity-based blind models (EqM, uEDM) reach state-of-the-art — confirming the parameterization, not the expressivity, is the failure mode [§7.3, Table 3].

Method

The paper is theoretical. The unified affine schedule $x_t = \alpha_t x_0 + \sigma_t \varepsilon$ covers DDPM, EDM, and Flow Matching as parameter choices for $(\alpha_t, \sigma_t)$ and for the prediction target (noise $\varepsilon$ , signal $x_0$ , or velocity $v = \dot\alpha_t x_0 + \dot\sigma_t \varepsilon$ ). For a noise-agnostic network (no $t$ input), MSE training drives the model to the posterior expectation of the target given only $x_t$ — i.e. a time-average of the optimal conditional denoiser weighted by the posterior $p(t \mid x_t)$ .

The core technical move is the Energy-Aligned Decomposition (§5.1): the optimal autonomous vector field factors into exactly three geometric pieces — a natural gradient on $\mathcal{E}_M$ scaled by an effective gain, a transport-correction term involving the posterior covariance of $t$ given $x_t$ , and a linear drift coming from the schedule. Tweedie’s formula lets them write the conditional score as a function of the optimal denoiser, and then the divergence in the gradient (proven in Appendix B for both discrete data and data on a $d$ -dim manifold in $\mathbb{R}^D$ with $d < D$ ) is shown to be canceled by a matching $\sigma_t \to 0$ vanishing of the gain.

Stability analysis (§6) then treats the sampler velocity as a time-varying field and computes the drift perturbation error between the autonomous and the noise-conditional (“oracle”) update. Subtracting eliminates the schedule’s linear part and exposes parameterization-specific gain × estimation-error products. Three asymptotic regimes appear: noise-prediction (unstable, gain dominates), signal-prediction (stable, exponential error decay), velocity-prediction (stable, bounded gain).

Results

The empirical section is small and confirmatory rather than a benchmark push. On CIFAR-10 / SVHN / Fashion-MNIST with a ResNet-UNet trained for 10k steps (EMA 0.999, batch 128), DDPM-blind generates noisy non-coherent samples while FM-blind matches the FM-conditional baseline qualitatively. On a 2D-concentric-circles dataset embedded in $\mathbb{R}^D$ for $D \in \{2, \dots, 1024\}$ , the predicted three-regime behavior appears in the projected samples (Fig. 5). On the Sun et al. CIFAR-10 benchmark (Table 3), DDIM-blind sits at FID 40.90 while EqM and uEDM (both velocity-based) achieve SOTA among blind models — matching the theoretical prediction that the failure is the parameterization, not the absence of $t$ .

Why it’s interesting

This paper closes a real gap in the noise-conditioning-free diffusion story that has been growing through 2025–2026: Sun et al. and Kadkhodaie et al. showed empirically that blind models can match conditional ones, EqM and Flow Matching put the velocity-target on a practical footing, but no one had clearly explained which parameterizations are stable when you drop the time embedding, why they’re stable, and what landscape is being optimized. The clean prediction — that DDPM noise-prediction structurally fails under autonomous generation while velocity-prediction structurally succeeds — is directly actionable: it tells the field which parameterizations to use whenever simplifying the conditioning signal (blind models, but also potentially in time-saving training regimes, robust restoration, and self-consistent loss designs).

The connection to existing wiki threads is unusually dense. SoFlow: Solution Flow Models for One-Step Generative Modeling and Generative Modeling via Drifting (the Drifting Models paper from Kaiming He’s group) both build one-step-from-scratch generators on top of velocity-style parameterizations — this paper gives the theoretical reason why those are the right primitives even before you add bi-time / drift-field structure on top. It also sharpens the “velocity-based architectures absorb posterior uncertainty into a stable drift” story underlying SoFlow’s training recipe. And the analysis of the Marginal Energy landscape ties in with Unified Latents (UL): How to train your latents‘s framing that the diffusion prior’s noise-floor is itself a designable knob — same geometric object viewed from different sides.