The Universal Normal Embedding

TL;DR

UNE posits that diffusion-model noise and vision-encoder embeddings are not two unrelated latent spaces but two noisy linear projections of a single underlying approximately-Gaussian latent — the Universal Normal Embedding. The paper backs this up empirically with NoiseZoo, a dataset of per-image DDIM-inverted noises paired with CLIP/DINO embeddings, and shows that on CelebA, linear probes in noise space and embedding space yield strongly aligned attribute predictions. Linear directions in DDIM-noise space (estimated via simple logistic regression on attributes) act as semantically meaningful edit directions — flipping smile / age / gender — without any architectural changes, with orthogonalization sufficient to untangle spuriously correlated edits. The framing is a Platonic-Representation-Hypothesis-style claim aimed specifically at the encoder ↔ generator bridge.

Key claims

Vision-encoder embeddings (CLIP, OpenCLIP, DINOv3) and diffusion latents both empirically exhibit per-coordinate Gaussianity even though only the diffusion side trains toward a Gaussian prior explicitly [§1, Fig. 2].
The Universal Normal Embedding hypothesis: there exists an ideal multivariate-standard-Gaussian latent space whose linear projections approximate both encoder embeddings and DDIM-inverted noise [§1, Fig. 2].
Encoder latent spaces are not perfectly Gaussian but contain a “strong normal core” — many directions are nearly Gaussian, the rest carry redundancy or noise — which is consistent with a noisy-linear-projection model of UNE [§3].
NoiseZoo, a paired dataset of DDIM-inverted diffusion noise and matching CLIP/DINO embeddings, lets the same linear-probing methodology be applied to both spaces head-to-head [§2 / Datasets].
On CelebA, linear attribute probes in DDIM-noise space and in encoder-embedding space achieve strong and aligned accuracy — the same attributes are linearly readable from both [Abstract, §4].
Single linear directions in DDIM-noise space act as faithful, controllable edit directions for attributes (smile, gender, age) — purely linear, no architectural changes, no fine-tuning [Abstract, §4].
A simple orthogonalization of the linear edit directions mitigates spurious attribute entanglement (e.g. gender-smile coupling) [Abstract, §4].
Statistical Gaussianity tests (Anderson–Darling, D’Agostino–Pearson, Shapiro–Wilk applied to random 1D projections) show generative models approach the theoretical 95% acceptance rate of true Gaussians; encoders score high but lower; non-Gaussian baselines score low — consistent with UNE [§3, Table on AD/DP/SW %].

Method

The paper is part hypothesis statement, part empirical study. The hypothesis: there is one ideal Gaussian latent space Z ~ N(0, I), and both an encoder E and a DDIM-inversion map Φ produce “noisy linear views” E(x) ≈ A_E z + ε_E, Φ(x) ≈ A_Φ z + ε_Φ for some image x whose latent representation is z. Under this view the two latent spaces are not different objects, just different projections of the same underlying source — which predicts (a) marginal Gaussianity of coordinates in both spaces, (b) linear semantic structure preserved across both, and (c) aligned linear probes.

To test (a) the authors run univariate Gaussianity tests on random 1D projections of each space’s vectors, reporting the fraction that pass Anderson-Darling / D’Agostino-Pearson / Shapiro-Wilk thresholds. Diffusion noise from generative models scores near 95% (the floor expected for true Gaussian samples under the same tests); encoders score lower but still strongly Gaussian-like; non-Gaussian baselines fail. To build the data they assemble NoiseZoo: a per-image table of (image, DDIM-inverted diffusion noise from a pretrained DM, encoder embedding from CLIP / OpenCLIP / DINOv3). To test (b) and (c) they train linear probes on CelebA attribute labels in both spaces; the probes are compared for accuracy and for direction-alignment. Attribute edits are performed by perturbing the noise (or embedding) along the linear probe direction and then either decoding through the diffusion model (for noise space) or inspecting nearest neighbors (for embedding space). Orthogonalization of probe directions against confounding attributes is used to disentangle edits.

Results

On CelebA, linear probes in DDIM-noise space and encoder-embedding space yield strong and aligned attribute classification — same attributes, comparable accuracy, comparable direction structure across spaces [Abstract, §4].
Gaussianity acceptance rates (AD%, DP%, SW%): generative-model noise approaches the theoretical 95% ceiling for Gaussian samples; encoder embeddings remain high; non-Gaussian references fail the tests [§3].
Linear edits along probe directions in DDIM-noise space produce faithful smile/gender/age modifications on decoded images, with no architectural changes to the generator [Abstract, §4, Figs.].
Orthogonalization against confounding probe directions removes spurious entanglements (e.g. unwanted gender drift when editing smile) [Abstract, §4].

Why it’s interesting

This is the theoretical / explanatory companion to the recent burst of “vision-encoder-as-tokenizer” work the team has been filing. RAEv2 (Improved Baselines with Representation Autoencoders) shows you can train a DiT on top of frozen DINOv3 features and get state-of-the-art numbers; DINO-SAE (DINO-SAE: DINO Spherical Autoencoder for High-Fidelity Image Reconstruction and Generation) shows the encoder latent is best modeled on a hypersphere rather than ambient ℝ^d; UNITE (End-to-End Training for Unified Tokenization and Latent Denoising) collapses tokenizer + denoiser into one network and reports that they intrinsically align. UNE proposes the why: encoder latents and diffusion noise are two noisy linear projections of one shared Gaussian source, which predicts both the empirical Gaussianity DINO-SAE leans on and the encoder-denoiser CKA alignment UNITE observed without enforcing it. The CelebA result that a single linear direction in DDIM-inverted noise edits a semantic attribute is a strong operational test — it implies generator noise space is not opaque, and that any encoder-derived attribute classifier transfers to a generator-side edit primitive for free. For Luma’s stack, the actionable consequence is that “edit by perturbing inverted noise along a CLIP/DINO-readable direction” is a baseline worth measuring against learned-control adapters, and that the shared-Gaussianity argument gives a principled answer to “which encoder geometry should a new tokenizer match” — pick the encoder whose strong normal core has the highest overlap with your generator’s noise distribution. Caveat: the hypothesis is supported on faces (CelebA) plus aggregate Gaussianity tests; whether it survives at the scale and diversity of internet-scale image / video models is the obvious next experiment.