Self-Evaluation Unlocks Any-Step Text-to-Image Generation

TL;DR

Self-E (Self-Evaluating Model) is a from-scratch text-to-image pretraining framework that supports any-step inference. It augments a standard Flow Matching loss with a self-evaluation loss: the in-training network uses its own current score estimate (via Tweedie’s formula) as a dynamic teacher for global distribution matching, so no pretrained teacher is required. The result is a single 2B model that beats FLUX.1-dev, SDXL, SANA-1.5, LCM, SDXL-Turbo, SD3.5-Turbo and the concurrent TiM on GenEval Overall at 2, 4, 8, and 50 steps — and whose quality improves monotonically with more steps within one unified checkpoint.

Key claims

The training objective is a two-component hybrid: a Conditional Flow Matching loss for local velocity supervision plus a self-evaluation loss for global distribution matching, weighted by a function of two sampled timesteps (s, t) [§3.3, Eq. 16].
The self-evaluation term is the gradient of the reverse-KL between the model’s implicit one-shot distribution and the data distribution, but the real-data score is replaced by the in-training model’s own conditional-expectation estimate via Tweedie’s formula — i.e. the model is its own teacher [§3.2, Eq. 10–11].
Only the classifier score term of the score difference is needed in early training; the fake/auxiliary score term is omitted to avoid co-training a second network and is reintroduced only near convergence to remove oversaturation artifacts [§3.2, Fig. 6, Tab. 2].
An “energy-preserving normalization” rescales the implicit regression target to preserve the clean sample’s norm, mitigating color/saturation bias when the self-evaluation weight is large [§3.3, Eq. 18–20].
A 2B parameter Self-E trained at 512² outperforms FLUX.1-dev, SDXL, SANA-1.5, LCM, SDXL-Turbo, SD3.5-Turbo and the concurrent any-step TiM on GenEval Overall at 2 / 4 / 8 / 50 inference steps (0.753 / 0.781 / 0.785 / 0.815) [Tab. 1].
At 2 steps Self-E beats the second-best baseline (TiM) by +0.12 GenEval Overall (0.753 vs 0.634); at 50 steps it still beats SANA-1.5 (0.815 vs 0.806) within a single checkpoint [Tab. 1].
Performance scales monotonically with inference budget in the same checkpoint, unlike distillation-based few-step models such as LCM and SDXL-Turbo whose 50-step scores collapse below their 4-step scores [Tab. 1].
Controlled 0.5B ablations on identical data show Self-E beats Flow Matching and Inductive Moment Matching (IMM) at every step count, with the gap largest in the few-step regime (0.61 vs 0.25 / 0.26 at 2 steps after 300k iters) [Tab. 2].

Method

A single network g_θ(x_t, t, s) takes a noisy latent at time t and predicts a clean sample as if jumping to time s. Two timesteps, (s, t), are sampled per training step. When s = t the model receives only the standard Conditional Flow Matching loss against the marginal velocity (local supervision, identical to a flow model). When s ≠ t an additional self-evaluation loss fires.

The self-evaluation loss is derived as follows. Treating g_θ at (t, s) as a sampler from an implicit distribution q_θ, minimizing reverse-KL KL(q_θ || p_data) gives a per-sample gradient that is the difference of two intractable scores: the data score ∇log p_data and the model’s own fake score ∇log q_θ. Tweedie’s formula relates each score to a conditional expectation; the paper’s key move is to estimate the data-score expectation with the in-training model itself, treating it as a dynamic self-teacher. Empirically only the classifier-score component (the CFG-induced difference between conditional and unconditional expectations) is used in early training; the auxiliary (fake-score) term is added back later for refinement. The loss is implemented as MSE between the model’s prediction and a stop-gradient pseudo-target built from two extra forward passes of the same network [Fig. 2(c), Eq. 14–15].

An energy-preserving normalization of the implicit regression target (inspired by EP-CFG) rescales the target so its norm matches the clean sample’s, which the ablation shows is necessary to avoid color bias when the self-evaluation weight is non-trivial. Architecture is a latent transformer in the FLUX/SD3 family, modified to accept the second timestep s alongside t. Inference is the same iterative predict-and-step procedure as a diffusion/flow model, except the “next-step” target time can be set freely — enabling any-step sampling from a single trained checkpoint, with EP-CFG at inference time.

Results

GenEval Overall on a 2B model trained at 512² (higher better), single checkpoint evaluated at multiple step counts [Tab. 1]:

Steps	Self-E	TiM	SANA-1.5	FLUX.1-dev	SD3.5-Turbo	LCM
2	0.753	0.634	0.166	0.100	0.364	0.262
4	0.781	0.687	0.573	0.320	0.719	0.328
8	0.785	0.714	0.779	0.589	0.707	0.340
50	0.815	0.780	0.806	0.797	0.611	0.330

Self-E is the only method in the table whose GenEval Overall improves monotonically across all four step counts — distillation-based methods (LCM, SDXL-Turbo, SD3.5-Turbo) plateau or regress past their target step count, and Flow Matching collapses below 4 steps. Ablations at 0.5B with 300k iters under matched conditions: Self-E 0.610 / 0.712 / 0.749 / 0.754 vs Flow Matching 0.252 / 0.608 / 0.716 / 0.731 vs IMM 0.262 / 0.599 / 0.711 / 0.747 at 2 / 4 / 8 / 50 steps [Tab. 2]. Removing target normalization mostly hurts (except marginally at 2 steps); including the auxiliary term from scratch significantly hurts (0.331 at 2 steps vs 0.544 baseline).

Why it’s interesting

This is the clearest existence proof so far that teacher-free any-step text-to-image generation can match or beat both pure flow matching and distillation pipelines at every inference budget — and crucially, in a single checkpoint rather than one model per step regime. The trick is small: feed Tweedie back into reverse-KL using the same network that’s still learning the score. For a generative-models team that ships diffusion/flow stacks, the practical implication is that one could collapse the “train a big slow flow model, then distill a fast student” two-stage pipeline into a single pretraining run that exposes the full quality/latency Pareto. It also pairs naturally with SoFlow: Solution Flow Models for One-Step Generative Modeling (SoFlow): both are from-scratch alternatives to MeanFlow/IMM/consistency models, but SoFlow targets one-step via a bi-time solution function and Taylor-based consistency, while Self-E targets any-step via reverse-KL self-distillation. The contrast — solution-function + consistency vs flow-matching + reverse-KL — is a useful axis to track.