Hallucination in World Models is Predictable and Preventable (MMBench2)

TL;DR

Nicklas Hansen and Xiaolong Wang argue that hallucination in generative world models — visually fluent rollouts that drift away from ground-truth dynamics — is fundamentally a data-coverage problem and is therefore both predictable at runtime (via three label-free metrics computed from quantities the model already produces) and preventable (via coverage-aware reweighting and curiosity-driven autonomous data collection). They build MMBench2, a 210-task, 10-domain open benchmark with ground-truth actions, rewards, language instructions, and live simulators per task, and use it to train a 350M-parameter world model (Dreamer 4-style: tokenizer + block-causal Transformer dynamics + decoder). Three failure modes (encoder-snap, action-marginalization, dynamics-divergence) are each traced to a specific component, predicted at Spearman ρ ≈ 0.8 against realized rollout error, and substantially reduced by uniform-per-task resampling and by finetuning on just 50 trajectories per unseen task — collected autonomously by rolling out candidates inside the world model and executing the highest-hallucination one.

Key claims

Hallucination is decomposed into three component-specific failure modes — encoder snap onto a familiar scene, action marginalization (same trajectory regardless of action), and abrupt dynamics divergence in low-coverage regions — each traceable to the tokenizer, dynamics model, or decoder of a Dreamer-4-style stack [§Three types of hallucination].
Three label-free predictors track realized rollout error at Spearman ρ ≈ 0.8 across 9k held-out 24-frame sequences: tokenizer round-trip residual u_r = ||ẑ − Enc(Dec(ẑ))||, flow instability (clean-frame movement across Euler substeps), and inter-seed variance of next-latent predictions [§Yes! Across 9k held-out sequences].
All three predictors are normalized by per-step scene motion (u_norm = u/m) so they track uncertainty relative to scene dynamism rather than absolute change [§dynamism-normalized variants].
High predictor values spatially co-locate with low training-data state density, making hallucination predictable from the same coverage map that explains it [§If we visualize hallucination as measured by our predictors].
Resampling the corpus to be uniform across the 210 tasks (rather than uniform across frames) simultaneously improves model quality and reduces all three hallucination predictors at no additional compute cost [§one data-centric lever should move all three at once].
Curiosity-based data collection that scores candidate rollouts by predicted hallucination and executes the most hallucination-prone one closes most of the gap to privileged human/expert data collection strategies in closed-loop MPC, using only 50 trajectories per unseen task for finetuning [§closed-loop planning (MPC) performance with each of 6 finetuned models].
MMBench2 spans 210 tasks across 10 domains (locomotion, manipulation, navigation, arcade) with equal trajectory counts per task but a heavy-tailed frame distribution from 25 (ManiSkill3) to 1,000 (Atari) steps — the per-task median is 65,260 frames — and ships with live simulators and full action/reward/language ground truth [§MMBench2 includes 210 tasks].

Method

The 350M base world model largely follows the Dreamer 4 recipe: a video tokenizer encodes each frame into a continuous latent z (trained jointly with the decoder via masked autoencoding); a block-causal Transformer dynamics model predicts the next latent from past latents and an action token (trained with shortcut flow-matching, encoder/decoder frozen); a decoder renders latents back to pixels. Because the stages compose sequentially, an early hallucination (e.g. a snap in the encoder) propagates downstream, so the failure-attribution step matters before any fix.

Three hallucination predictors are derived from already-computed quantities, with no additional training: the tokenizer round-trip residual u_r, the flow-instability metric (next-frame oscillation across denoising substeps), and the inter-seed variance of the next-latent prediction. Each is dynamism-normalized by per-step scene motion m. The mitigation pipeline has two stages: (1) reweight the existing corpus to be uniform across tasks (not frames), retraining tokenizer + dynamics; (2) at test time on an unseen task, roll out candidate trajectories inside the world model, score by predicted hallucination, execute the highest-hallucination candidate in the real environment, collect 50 trajectories, and finetune. MPC is then used to evaluate closed-loop downstream task performance against five other data-collection baselines (random, human, expert, etc.).

Results

Three label-free predictors achieve Spearman ρ ≈ 0.8 against realized 24-frame rollout error on 9k held-out sequences without any labels or extra training [§Across 9k held-out sequences].
Coverage-aware (uniform-per-task) retraining improves held-out model quality across 200 tasks while reducing all three hallucination predictors simultaneously [§Mean change vs. the base model].
Finetuning on 50 trajectories per task qualitatively recovers correct dynamics on multiple unseen tasks (Point Maze unseen layout, Dungeon Explorer transfer task, Cup Catch unseen variant, Reacher Easy unseen task) where the base model otherwise quietly drifts to a familiar scene [§targeted data collection].
In closed-loop MPC across 6 finetuned models varying only the data-collection policy, curiosity-based collection using the hallucination predictors approaches privileged human/expert data collection [§closed-loop planning (MPC) performance].
Coverage analysis on a central-bottleneck maze shows curiosity targets walls (where the base model penetrates them), confirming the predictors flag exactly the regions targeted exploration should visit [§coverage].

Why it’s interesting

This is the clearest filed statement of the coverage-gap diagnosis for world-model failure — an explanation that cuts across the three poles of the World Foundation Models cluster (pixel-rollout WFMs like Genie 3 / Cosmos 3, latent-predictive WFMs like V-JEPA 2.1, and language WFMs like Qwen-AgentWorld). It complements the wiki’s existing benchmark-as-probe entries — Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation (PhyGenBench) (PhyGenBench), RISE-Video: Can Video Generators Decode Implicit World Rules? (RISE-Video), and Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation? (Dream.exe) — by being the first filed entry where the benchmark, the model, and the failure-mode taxonomy ship together as one open artifact, with live simulators rather than a held-out video set. Compared to Robots Need More than VLA and World Models‘s position that scaling VLA + WMs is insufficient because of missing interfaces, this paper takes the opposite bet: scaling can be enough if data coverage is treated as a first-class engineering target with runtime predictors and curiosity-driven collection. The interactive 350M model embedded in the page (live keyboard control with per-frame hallucination indicators) is also notable as a research-comms artifact in the The flavor of the bitter lesson for computer vision spirit.