Are We Overthinking One-Step Generation?

TL;DR

Priming Models (Park, J. Kim, Du, Chen, Han) propose a drastically simplified two-stage recipe for training one-step generative models from scratch: first train the network to reconstruct clean images from inputs corrupted at a single fixed noise level (no time embedding, no schedule, no teacher), then run feature-space distributional post-training. On ImageNet 256×256 this reaches one-step FID 1.19 using roughly one-tenth the training compute of comparable one-step generators. The paper’s central scientific claim is that reconstruction pretraining alone is sufficient to eliminate the historical difficulty of training distributional objectives (GAN / Sinkhorn / Wasserstein) from scratch — just 1,250 post-training updates lift Priming-B to FID 2.26 even though the pretrained Stage-1 network itself produces poor samples from Gaussian noise. The recipe scales to text-to-image (0.818 GenEval) and robot world modeling (RT-1 / Bridge-V2).

Key claims

Reconstruction pretraining + distributional post-training beats SOTA one-step from-scratch generators at a fraction of the compute. Priming-L reaches FID 1.19 on ImageNet 256² at 6.2× fewer FLOPs than Drifting-L (FID 1.61); Priming-B hits FID 1.47 at 12.2× fewer FLOPs than Drifting-B (FID 1.74), beating Improved/Pixel MeanFlow, Shortcut, W-Flow, and Drifting at every compute budget [§Abstract, Fig. 1].
Reconstruction at a single fixed noise level suffices as the pretraining task. The simplest instantiation feeds Gaussian noise directly into a network with no time embedding; it learns pixel-space denoising at one noise level and nothing else [§2, Fig. 1].
Stage-1 is a bad generator but a great initialization. The pretrained reconstruction network produces poor samples from Gaussian noise, yet 1,250 post-training updates with a distributional objective bring Priming-B to FID 2.26 [§Abstract, Fig. 1 right].
The pretraining task is flexible. Masking-based reconstruction gives FID 1.87 and RAEv2 feature reconstruction also provides an effective initialization — reconstruction, not the specific corruption, is what matters [§3].
Scales to text-to-image at 0.818 GenEval one-step on the MiniT2i codebase, compared to 0.548 for one-step MeanFlow and 0.842 for four-step MeanFlow [§4]. A two-step variant is also proposed.
Scales to action-conditioned world modeling on RT-1 and Bridge-V2 [§5].
Mechanistic argument via the Jacobian. A distributional loss identifies useful correction directions but the network’s Jacobian determines how parameter updates translate those directions; reconstruction pretraining shapes the Jacobian so distributional gradients become effective [§6].
Known limitation: shared with Drifting. Priming’s distributional post-training stage relies on pretrained feature encoders; whether this dependence can be reduced is left as an open question [§7].

Method

Priming Models have exactly two components. Stage 1 (Reconstruction pretraining) trains a time-embedding-free network to reconstruct clean images from inputs corrupted at a single fixed noise level — in the simplest instantiation, Gaussian noise is fed directly in and the network learns pure pixel-space denoising. Masked reconstruction and reconstruction of pretrained RAEv2 features are shown as drop-in alternatives. Stage 2 (Distributional post-training) fine-tunes the pretrained network with a distributional objective in feature space, treating the Stage-1 weights as an initialization that pre-shapes the Jacobian so distributional gradients translate into useful parameter updates. No diffusion teacher, no flow map, no CFG, and no JVP are used anywhere in the pipeline — the authors explicitly contrast this with consistency-style, Flow Map, and DMD one-step recipes. Code and intermediate reconstruction checkpoints are open-sourced at https://github.com/JaeyeonKim01/priming.

Results

Headline ImageNet 256² one-step FID: Priming-L 1.19 at 6.2× fewer FLOPs than Drifting-L (1.61); Priming-B 1.47 at 12.2× fewer FLOPs than Drifting-B (1.74) [§Abstract, Fig. 1]. Priming beats Improved MeanFlow, Pixel MeanFlow, Shortcut, W-Flow, and Drifting across scales. Even an under-trained Stage-1 → short Stage-2 run (1,250 updates for Priming-B) hits FID 2.26 — the reconstruction init is load-bearing. Text-to-image on MiniT2i: one-step Priming GenEval 0.818 vs 0.548 for one-step MeanFlow and 0.842 for four-step MeanFlow [§4]. World modeling results reported on RT-1 and Bridge-V2 [§5]. Ablations: masked reconstruction init → FID 1.87; RAEv2 feature reconstruction also works. State-of-the-art on ImageNet 256² among one-step generators at ~1/10 training compute of prior methods with comparable FID [§Abstract].

Why it’s interesting

Lands directly in Few-Step Generation (from scratch) as the fourth from-scratch one-step recipe filed on the wiki — alongside SoFlow’s bi-time solution function (SoFlow: Solution Flow Models for One-Step Generative Modeling), DriftWorld’s action-conditioned drifting (DriftWorld: Fast World Modeling through Drifting), and Explorative Modeling (Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation) — but with a strikingly simpler training recipe: just reconstruction pretraining followed by distributional post-training, no JVP, no flow map, no CFG. The claimed 6–12× FLOPs reduction vs Drifting at better FID is a concrete datapoint for the open question on Few-Step Generation (from scratch) about whether from-scratch few-step can scale to production-quality video — Priming does not yet reach Wan-class scale but does scale to T2I and robot world modeling. Also complements the Diffusion Distillation thread by sharpening the from-scratch vs distillation Pareto: Priming matches/beats one-step distillation baselines on ImageNet while skipping the teacher entirely. Shares a limitation with Drifting (reliance on pretrained feature encoders), tying it to Representation Autoencoders.