Self-Refining Video Sampling

TL;DR

A training-free inference-time recipe that boosts physical realism and motion coherence of a pretrained video flow-matching model by repeatedly perturbing and re-denoising the latent at a fixed noise level before advancing the ODE step. The justification is a DAE reinterpretation of flow matching: the trained velocity model doubles as a time-conditioned denoising autoencoder, so iterated Predict-and-Perturb (P&P) at a single $t$ drives the latent toward higher-density regions of the data manifold. With CFG, more than ~2 P&P iterations oversaturate static regions; the fix is uncertainty-aware P&P that only updates voxels whose prediction is sensitive to the perturb step. Demonstrated on Wan2.2-A14B (T2V) and Cosmos-Predict2.5-2B (I2V robotics) with ~50% extra NFEs, beating same-budget NFE×2 and best-of-4 rejection sampling against Cosmos-Reason1-7B.

Key claims

The flow-matching loss equals a standard DAE reconstruction objective up to a time-dependent weighting, so a trained flow-matching model can be repurposed as a time-conditioned denoising autoencoder at inference [§Methods, “Flow Matching as Denoising Autoencoder”].
Iterated Predict-and-Perturb at a fixed noise level $t$ (the “P&P loop”: $z_t \to \hat z_1 \to z_t' \to \dots$ ) gradually steers latents toward the data manifold; 2–3 iterations are sufficient to improve temporal coherence and physical plausibility [§Methods, “Predict-and-Perturb”].
Naive P&P with CFG at $K \geq 3$ causes oversaturation and simplification of static regions (e.g. backgrounds) [§Methods, “Uncertainty-aware P&P”].
Uncertainty for region selection is estimated as the L1 distance between consecutive $\hat z_1$ predictions across P&P iterations, $\mathbf{U} = \|\hat z_1^{(k)} - \hat z_1^{(k+1)}\|_1$ , and requires no additional NFEs because both terms are already computed inside the P&P loop [§Methods, “Uncertainty-aware P&P”].
Applied to Wan2.2-A14B T2V, the method visibly improves complex human motion (sprinting, gymnastics, sword parries, basketball physics) vs. base sampling, vs. matched-compute NFE×2, and vs. the FlowMo baseline [§Results, “Motion Enhanced Video Generation with Wan2.2-A14B”].
Applied to Cosmos-Predict2.5-2B I2V on robotics scenes, the method reduces unstable-grasp and implausible-interaction artifacts and produces more consistent motion than rejection sampling with Cosmos-Reason1-7B at best-of-4 [§Results, “Image-to-Video Robotics”].
Total cost is ~50% additional NFEs over base sampling, with no external verifier, no training, and no extra dataset [§TL;DR].

Method

The trained flow-matching velocity field is reinterpreted as a denoising autoencoder: at a given noise level $t$ , Predict maps the noisy state $z_t$ toward a clean estimate $\hat z_1$ , and Perturb corrupts it back to $z_t'$ by interpolating with fresh Gaussian noise at the same level $t$ . Repeating Predict-and-Perturb at fixed $t$ for $K$ iterations forms a corrupt-reconstruct loop that pulls the latent toward higher-density regions of the data distribution. The refined latent $z_t^* := z_t^{(K)}$ then replaces $z_t$ before passing to the next ODE solver step, so the recipe is solver-agnostic and drop-in.

Multiple P&P updates combined with CFG (which already pushes samples away from the prior) tend to oversaturate static regions. The fix is Uncertainty-aware P&P: between consecutive iterations, compute $\mathbf{U} = \|\hat z_1^{(k)} - \hat z_1^{(k+1)}\|_1$ as a per-region confidence proxy (regions whose $\hat z_1$ is stable across perturbation are already on-manifold), and apply the next P&P update only to regions where $\mathbf{U}$ exceeds a threshold $\tau$ . The uncertainty signal is free in NFEs because both $\hat z_1^{(k)}$ predictions are already produced inside the loop.

Results

The project page only reports qualitative comparisons (no FVD / VBench numbers shown), against four conditions: Ours, Base, NFE×2 (matched-compute baseline that just takes more ODE steps), and FlowMo (prior refinement-style baseline) on Wan2.2-A14B T2V; and against Base and Rejection Sampling (best-of-4) with Cosmos-Reason1-7B as external verifier on Cosmos-Predict2.5-2B and Wan2.2-A14B I2V robotics. Stated cost overhead is ~50% additional NFEs. Quantitative numbers are deferred to the arXiv preprint (2601.18577).

Why it’s interesting

This is a clean instance of Inference-Time Scaling applied to generation rather than reasoning: spend extra NFEs at the same noise level instead of spending them on more noise levels, and the same total compute buys better physics. Two specific results are worth flagging for the team: (1) P&P at fixed compute beats NFE×2, which is a non-trivial claim about where the marginal value of an extra denoiser call lives in the schedule — at the current $t$ , not in finer discretization. (2) It beats best-of-4 rejection sampling with a 7B verifier, suggesting that for video, the trained denoiser already encodes a stronger manifold prior than a separately-trained reasoner-as-verifier — relevant for any pipeline considering an external VLM judge as a quality lever. The uncertainty-masking detail is also worth internalizing: any iterative-refinement scheme on a CFG-trained generator probably needs a similar gate, or it will oversaturate.