Wonder: Video World Model Done Better

TL;DR

Wonder is a general-purpose video world model from Adobe Research and Johns Hopkins that turns a single image (or a conditional video) into a playable, camera-controllable world at 16 FPS with constant latency for up to a minute of rollout. It co-designs three systems: a dense-coordinate-field camera conditioning that renders 6-DoF camera motion into a spatially aligned visual signal, a sparse-attention memory that adaptively retrieves relevant full-fidelity history KV entries at constant cost as context grows, and a “rectified” self-forcing distillation pipeline (Mixture-of-Students + GAN Control Regularization) that preserves both control fidelity and long-term consistency in the student. The result covers both image-to-video world exploration and video-conditioned re-shooting under new camera trajectories.

Key claims

• Rendering camera pose through a 3D scaffold + environment map turns 6-DoF translation and rotation into dense, frame-aligned visual evidence the model can interpret directly, rather than an abstract parameter vector [§Method / Control Signal].
• A sparse-attention memory over full-fidelity history KV caches gives constant-latency retrieval independent of actual context length, supporting return-to-previously-observed-regions at up to one-minute rollouts [§Method / Memory Mechanism, Abstract].
• Self-forcing-style distillation degrades control and memory in the student; a Mixture-of-Students design + GAN Control Regularization is needed to rectify camera drift and preserve teacher diversity/long-term memory [§Method / Training Strategy].
• Wonder synthesizes 16 FPS, minute-scale videos with coherent geometry, appearance, dynamics, and interactive control across both I2V and V2V regimes [Abstract].
• Beyond I2V world exploration, the same model supports V2V — existing dynamic scenes can be re-shot in real time under new camera trajectories [Abstract].

Method

Wonder is a Wan2.1-based video world model with three co-designed components. Control signal: the camera pose is turned into a dense coordinate field by rendering a synthetic camera space with a 3D scaffold and environment map, so that translation and rotation are exposed to the DiT as frame-aligned visual evidence rather than injected as a numeric conditioning vector. Memory mechanism: during rollout, past latents are retained as full-fidelity KV caches; at each new chunk, a sparse attention module selects a small set of relevant entries, giving constant per-step latency regardless of context length. This is what lets a user leave a room and come back to find it unchanged. Training strategy: the student is distilled from a bidirectional teacher via a self-forcing pipeline, but two additions rectify the usual student-side degradations: a Mixture-of-Students architecture broadens the student’s expressive capacity, and a GAN Control Regularization loss keeps camera control from drifting during distillation. Together these yield 16 FPS 6-DoF-controllable I2V/V2V generation with 0.5 s constant latency.

Results

Wonder reports 16 FPS throughput at 0.5 s constant latency for up to one minute of rollout, with 6-DoF camera control, on both I2V (game / cartoon / real worlds) and V2V (real-scene re-shooting, cartoon/game re-shooting). The project page demonstrates complex camera trajectories and side-by-side comparisons with SOTA models; specific benchmark numbers are not disclosed in the abstract at filing time.

Why it’s interesting

Wonder sits at the intersection of three concepts the wiki has been tracking. As a camera-control recipe on Wan2.1 it is a new datapoint on the Camera-Controlled Video Diffusion Pareto — dense coordinate-field rendering is a distinct signal shape from the depth/RGB control maps used by VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control or the reference-video branch of OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfer, and closer in spirit to point-cloud-as-ControlNet approaches. Its sparse-attention KV memory is a fifth-plus operating point on Context Length / Quality Trade-off in Video Generation alongside PFP, Context Forcing, PackForcing, MALT, and Mirage’s latent cache (Latent Spatial Memory for Video World Models) — same “revisit-previous-region at constant latency” target as Mirage, but working over full-fidelity KV rather than warping latents through a depth cache. And the Mixture-of-Students + GAN Control Regularization rectification is a control-preserving Diffusion Distillation recipe, making explicit that self-forcing distillation trades control fidelity for speed unless the training loss is redesigned.

See also

• Camera-Controlled Video Diffusion — dense-coordinate-field camera conditioning is a new signal shape on the frozen-Wan camera-control Pareto.
• Context Length / Quality Trade-off in Video Generation — sparse-attention selection over full-fidelity KV cache is a distinct operating point from partitioning (Context Forcing / PackForcing) and latent-warping (Mirage).
• 4D Scene Generation — “playable world from one image with revisit-consistent geometry” is the WonderJourney-lineage 4D goal, now realized in real-time interactive video.
• Latent Spatial Memory for Video World Models — Mirage; closest sibling on the “constant-latency long-horizon revisit” target, via depth-guided latent warping rather than sparse KV selection.
• WonderJourney: Going from Anywhere to Everywhere — WonderJourney’s static perpetual-scene lineage; Wonder adds dynamics and interactive camera control.
• OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfer — OmniTransfer; sibling Wan-adapter recipe using reference-video conditioning instead of a rendered control map.
• Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation — Voyager; long-range explorable 3D scene generation from video diffusion with a bounded 3D cache.
• PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference — PackForcing; short-video training → long-video inference with partitioned + compressed KV cache.