FACT: Failure-Aware Causal Training for World-Action Models

TL;DR

FACT (Peng, Liang, Yan, Hansen, Wang; UCSD Xiaolong Wang group) trains World-Action Models to predict future video and task-progress conditioned on the executed action — flipping the standard “generate future video, recover action via IDM or goal-conditioning” WAM interface so that failed rollouts become valid supervision rather than discarded data. The model predicts action first and then imagines its consequence, so both successful and failed action outcomes shape the video predictor and the progress head. Reports 81.8% → 85.6% → 87.5% on 50 RoboTwin tasks (no video co-training → FACT → FACT + failures) and monotone scaling of success in failure-data fraction (32.7% → 44.7% → 57.3% at 0 / 50 / 100% failure rollouts). Real YAM robots: seen 82 → 89 → 92 and unseen 67 → 77 → 82 with FACT + failures + scoring, approaching Motus / LingBot-VA levels. First filed WAM recipe to make failure-video supervision a first-class training signal at the world-model layer rather than folding failure into policy-side value learning (π*0.6: a VLA That Learns From Experience (RECAP) RECAP) or as a retrieval-side memory (FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models FailSafe).

Key claims

Most WAMs generate future videos and recover actions via IDMs or use predicted futures as goal conditions, so the world model is trained mostly on successful demonstrations and has no reason to predict the consequences of bad actions [Abstract].
FACT’s causal interface is action-first, video-second: predict the action, then condition the future-video + task-progress prediction on the executed action, turning failed rollouts into valid supervision targets [§Method / Tweet 3/n].
Failure-aware training makes the progress predictor discriminate successful vs failed action outcomes; the progress head can optionally rescore sampled action candidates at inference [Abstract].
RoboTwin (50 tasks): 81.8 → 85.6 → 87.5 % avg success (no video co-training → FACT → FACT + failures) — success-only WAM training is the ablation the paper’s causal interface unlocks [Tweet 5/n].
Failure-data scaling. Sweeping failure-rollout fraction 0 → 50 → 100% lifts success 32.7 → 44.7 → 57.3 %, evidence failure supervision scales rather than saturating [Tweet 6/n].
Real YAM robots. Seen tasks 82 → 89 → 92% (base → FACT + failures → +scoring); unseen tasks 67 → 77 → 82% — first filed WAM result showing failure supervision lifts held-out real-world tasks and approaches Motus / LingBot-VA levels [Tweet 5/n].
Reduced success-biased hallucination. Given a failed action, a success-only WAM still imagines successful futures; FACT predicts the observed failure. Future-frame PSNR on failure futures rises 19.51 → 25.92 dB while success-future prediction is preserved [Tweet 4/n].

Method

FACT re-orders the standard WAM training objective. Where prior WAMs (video-first: RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation RynnWorld-4D, Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model Xiaomi-U0) generate futures under the marginal video distribution and recover actions post-hoc via an IDM (or use predicted futures as goals), FACT trains the world-action model to first emit an action a, then produce the video + task-progress prediction conditioned on a. This action-consequence factorization has two consequences: (i) failure rollouts become valid supervision — the ground-truth “bad action → observed failure” trajectory is now a legal target of the video branch; (ii) the progress head, trained jointly, learns to discriminate imagined outcomes and can score sampled action candidates at inference. The training set mixes successful demonstrations with failure rollouts, and the paper reports the failure-fraction ablation directly. Real-robot evaluation uses YAM bimanual arms with seen and unseen task splits and an inference-time scoring pass over N action samples from the WAM.

Results

RoboTwin 50-task avg success: 81.8% (no video co-training) → 85.6% (FACT) → 87.5% (FACT + failures) [Tweet 5/n].
Failure-data scaling law: 32.7% → 44.7% → 57.3% success at 0% → 50% → 100% failure-rollout fraction [Tweet 6/n].
Real YAM bimanual — seen tasks: 82 → 89 → 92% (base → +failures → +scoring); unseen tasks: 67 → 77 → 82% [Tweet 5/n].
Future-frame PSNR on failure futures: 19.51 → 25.92 dB when trained with failure supervision, with prediction of successful futures preserved [Tweet 4/n].
Approaches Motus / LingBot-VA scores on real YAM without matching their data scale [Tweet 5/n].

Why it’s interesting

FACT is the first filed WAM training recipe that treats failed rollouts as first-class supervision at the world-model layer, and it sharpens the World Foundation Models cluster’s open question about which axis of WFM-ness carries downstream policy gains. Where Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2 identifies joint video co-training on unlabeled human video as the load-bearing lever at 1M+ hours, FACT identifies a complementary lever in the supervision distribution itself — feed the video predictor the ground-truth outcomes of bad actions, not just good ones. The success-biased-hallucination diagnostic (a success-only WAM imagines success even under a failed action; PSNR 19.51 → 25.92 dB after failure supervision) is the sharpest quantitative version yet of the “world model as physically faithful predictor rather than optimistic completion engine” argument.

The paper also lands in the middle of an active thread on the VLA Models page about how to reuse failure data. π*0.6: a VLA That Learns From Experience (RECAP) π*0.6 / RECAP folds failure into a policy-side advantage-conditioning signal via a binarized value token; FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models FailSafe treats failure as a companion-VLM’s runtime recovery signal; EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstration EgoRecovery treats it as a data-collection problem tackled with egocentric human recovery clips; and OopsieData — a multi-lab dataset of real robot manipulation failures OopsieData pools the real-robot failure and suboptimal rollouts that policy evals already produce. FACT stakes out the fifth position: consume failure at the video-prediction training layer via causal action-first conditioning, so the WAM is not policy-agnostic (like Dyna-2) but action-consequence-shaped in a way that survives use as either a data engine or an action-selection scorer.

The name collision with FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT (Force-Aware Contact-rich manipulation via Timestep modulation) is unfortunate but semantic — that FACT is a training-side flow-matching fix for the low-noise regime; this FACT is a WAM training-objective fix for the success-only-supervision regime. Different authors, different attack surfaces.