Inference-time Physics Alignment of Video Generative Models with Latent World Models

TL;DR

WMReward treats physics plausibility of video generation as an inference-time alignment problem: it re-uses a pretrained latent world model (VJEPA-2) as an off-the-shelf reward by measuring the surprise between VJEPA’s predicted future latents and those of a generated video, then samples from the reward-tilted distribution via best-of-N selection, ∇-guidance, or both combined. Applied on top of MAGI-1 24B and an internal 5B video latent-DiT, it wins 1st place on the ICCV 2025 PhysicsIQ Challenge with a 62.0% final score on V2V (+6.78 over MAGI-1 baseline) and 36.56 on I2V (+6.79), with the gain transferring to T2V on VideoPhy and to human-preference studies. Most relevant to Luma: the recipe is purely inference-time, model-agnostic, and shows clear scaling with the number of particles — i.e. you can buy physics by spending compute, no retraining.

Key claims

Physics implausibility in video generation is partly an inference-time problem, not only a pre-training one: the same base model can be steered to physically more plausible samples without any weight updates [§1, Abstract].
VJEPA-2’s predictive surprise (latent-space MSE between predicted-from-context and full-video representations over a sliding window of T context + H future frames) is a stronger physics-plausibility reward than VideoMAE pixel reconstruction or Qwen2.5/3-VL VQA scores [§3.1, Table 1].
WMReward(BoN) with 16 particles on MAGI-1 V2V improves PhysicsIQ score 55.22 → 60.34 (+5.12); adding ∇-guidance (∇+BoN) pushes to 62.00 (+6.78), the 1st-place ICCV 2025 entry [§3.1, Table 1].
The same recipe improves I2V on the smaller vLDM 5B baseline 27.76 → 33.44 (+5.68 with ∇+BoN) and on MAGI-1 29.77 → 36.56 (+6.79 with BoN alone) [§3.1, Table 1].
VLM-based rewards (Qwen2.5-VL, Qwen3-VL) perform near chance and can even regress baseline performance (e.g. Qwen2.5-VL(BoN) on MAGI-1 I2V: −4.78), while VideoMAE pixel-surprise gives smaller gains than WMReward [§3.1, Table 1].
Scaling holds: PhysicsIQ score increases monotonically with particle count, with variance shrinking; ∇+BoN sharpens the high-score tail of the distribution more than BoN alone [§3.3, Fig. 1, Fig. 4].
Gains transfer to T2V on VideoPhy: physics-consistency pass-rate +6.9 (vLDM) and +8.1 (MAGI-1), with a small drop in semantic-adherence that the authors attribute to VJEPA’s text-agnostic reward [§3.2, Table 3].
Human raters confirm: on PhysicsIQ, WMReward(∇+BoN) wins physics-plausibility 54.9% (MAGI-1) and 53.1% (vLDM), with smaller but positive wins on visual quality and prompt alignment [Table 2].
WMReward does not regress general perceptual quality on VBench (subject/background consistency, motion smoothness, temporal flickering, imaging/aesthetic) — most metrics improve slightly [§3.4, Table 5].
Compute cost is predictable: WMReward(BoN) memory scales linearly with particle count and runs in parallel; ∇-guidance adds backprop cost proportional to the number of guidance steps [§3.3, Table 4].
The method is sampler-agnostic — SMC and SVDD also yield gains over vanilla sampling but underperform BoN / ∇+BoN at matched particle budgets [§3.4, Table 6].

Method

The pipeline reuses VJEPA-2 untouched. Over a generated video, a window of T context + H predicted frames slides across all valid positions. At each position, VJEPA produces two latent representations: one from context-only (with future positions masked), one from the full window. The WMReward signal is the negative MSE between the future portions of these two representations, summed over all window positions — high reward means VJEPA’s “prediction from context” matches what was actually generated, i.e. the generated future is the kind of thing a physics-aware world model would have predicted.

Three sampling schemes draw from the reward-tilted distribution q(x) ∝ p(x) · g(R(x)):

BoN: generate N independent samples from the base video model, return argmax-reward. Embarrassingly parallel; memory scales linearly with N.
∇-guidance: at each denoising step, add λ · ∇_x R(x̂_0) to the score, using Tweedie’s formula to estimate the clean sample x̂_0 from the current noisy iterate. Adds backprop cost per guided step.
∇+BoN: run N guided trajectories, return argmax-reward. Combines stronger tilting with selection-based filtering of approximation error.

Applied to MAGI-1 24B (autoregressive chunk-by-chunk DiT) and an internal 5B holistic vLDM. The ICCV 2025 submission used MAGI-1 + ∇+BoN with 16 particles.

Results

The headline numbers — PhysicsIQ scores at 16 particles, ICCV 2025 challenge submission:

Model + method	I2V	V2V
vLDM baseline	27.76	—
vLDM + WMReward(∇+BoN)	33.44	—
MAGI-1 baseline	29.77	55.22
MAGI-1 + WMReward(BoN)	36.56	60.34
MAGI-1 + WMReward(∇+BoN) *	36.28	62.00
VLIPP (prev. SOTA I2V)	34.6	—
Kling 1.6	23.64	—
Sora	10.00	—

(* = ICCV 2025 PhysicsIQ Challenge submission, +6.78 over MAGI-1 baseline on V2V, +6.51 on I2V.) [§3.1, Table 1]

Reward ablations on MAGI-1 V2V (16 particles): WMReward(BoN) +5.12; VideoMAE-surprise(BoN) −0.51; Qwen2.5-VL(BoN) −0.51; Qwen3-VL(BoN) −0.80. The world-model surprise signal is the only reward that consistently moves the needle, and the alternatives can actively hurt.

T2V on VideoPhy (8 particles): MAGI-1 physics-consistency pass-rate 25.0 → 33.1 with ∇+BoN; semantic-adherence drops 54.4 → 44.8 (text-agnostic reward). Human study still shows a net 55.7% overall-quality win for MAGI-1 + WMReward(∇+BoN) on VideoPhy [§3.2, Table 2-3].

VBench perceptual-quality (PhysicsIQ generations, MAGI-1 + ∇+BoN vs. baseline): subject 95.82 → 95.88, background 97.25 → 97.30, motion smoothness 99.70 → 99.71, temporal flickering 99.82 → 99.85, imaging quality 69.07 → 69.25, aesthetic 47.05 → 46.94. No regressions outside aesthetic noise [Table 5].

Why it’s interesting

This paper is the clean inference-time counterpart to the existing wiki strand on long-horizon AR video — where Context Forcing: Consistent Autoregressive Video Generation with Long Context and Pretraining Frame Preservation in Autoregressive Video Memory Compression attack drift / forgetting by training better context machinery, WMReward shows that a frozen MAGI-1 already has physically plausible samples in its support, you just have to find them. The fact that VJEPA-2 surprise works as a reward while VLM judges do not is a non-obvious empirical claim — it pins down “physics understanding” as a property of predictive latent dynamics models, not of general-purpose multimodal LLMs, and it supplies a free, off-the-shelf reward signal that any video generator can stack on at sampling time. For Luma specifically: the recipe is model-agnostic, requires no fine-tuning, and the scaling-with-particles plot is the kind of compute-for-quality knob that’s straightforward to expose to power users. The author list (FAIR + Meta Superintelligence Labs + Mila + Oxford) overlaps with the VJEPA-2 group and with the inference-time alignment / image-generation-reward line at Mila — worth tracking as a recruiting signal for that cluster.