R²-WAM: Data-Free Continual Learning for World Action Model with Rectified Generative Replay

TL;DR

R²-WAM attacks catastrophic forgetting in World Action Models (WAMs) when their training data is unavailable — the standard “weights-only deployment” setting in robotics. The key observation is that a WAM already is a generative model of its own past: it can autoregressively regenerate rollouts of previously learned tasks and replay them against sequential fine-tuning. Raw generative replay already recovers most of the gap vs real-data experience replay (ER) on LIBERO (70.4% vs 94.6% SR), and two “rectification” steps — truncating at task completion with a VLM agent, and filtering rollouts whose generated actions are physically inconsistent with the generated video via optical-flow matching — close it almost completely (92.7% SR, 1.9 pp below real-data ER). The analysis finds forgetting concentrates in the WAM’s inverse-dynamics channel, not its video-prediction channel, which is what makes self-regeneration viable in the first place.

Key claims

Sequential fine-tuning of a WAM on four LIBERO suites in order collapses average success to 12.5% while per-stage plasticity stays above 94%, i.e. the policy can learn each new task but forgets prior ones [Table on LIBERO 4-suite benchmark].
Experience replay with 20 real demonstrations per prior task is the upper bound at 94.6% SR / 0.09 NBT across the 4 LIBERO suites [Table on LIBERO 4-suite benchmark].
Raw generative replay (R-WAM — the WAM regenerates 20 rollouts per prior task from language instructions alone, no real data) already lifts SR to 70.4% [Table on LIBERO 4-suite benchmark].
The two rectification steps are jointly necessary and individually insufficient: truncation-only hits 70.8% SR on LIBERO-Spatial, filtering-only 80.8%, both together 94.0% [Table — LIBERO-Spatial ablation].
An agentic VLM evaluator truncates each rollout at the step where it judges the task completed, and discards rollouts judged non-successful; removing truncation drops LIBERO-Spatial SR 94.0 → 70.8% [Table — LIBERO-Spatial ablation].
Physical-consistency filtering computes an optical-flow discrepancy s_flow between the actions executed in a robot-only simulator and a 3D tracker’s flow on the generated video, keeping the most consistent rollouts; removing it drops LIBERO-Spatial SR 94.0 → 80.8% [Table — LIBERO-Spatial ablation].
On a real 5-task sandwich-making sequence (steak → luncheon meat → cheese → fried egg → meat slice) on an arm + dexterous hand, R²-WAM scores 89.5% SR vs ER’s 91.8% and Seq-FT’s 28.5%, with no real data from prior tasks [Table — Real-world sandwich-making].
A teacher-forced probe isolates the mechanism: as later tasks arrive the WAM’s inverse-dynamics (action-prediction) accuracy on earlier tasks collapses, while its video-prediction on the same tasks stays visually plausible — forgotten WAMs still imagine the scene but can no longer act in it [§ Mechanism / teacher-forcing probe].
With 20 rectified rollouts per prior task R²-WAM reaches 92.7% average LIBERO SR, within 1.9 points of real-data ER (94.6%) while using no previous-task real demonstrations [Table on LIBERO 4-suite benchmark].

Method

A WAM is a video-generation model fine-tuned to co-emit future frames and future actions, trained sequentially on tasks 1..K. R²-WAM runs between tasks: before learning task k, the current model WAM_{k-1} is prompted with the language instructions of each previously learned task and generates candidate rollouts from scratch — complete (observation, action) sequences, with no real data from prior tasks required. These raw rollouts suffer from two systematic failure modes that make them worse than real demonstrations: (i) the WAM has no notion of being done and keeps generating purposeless motion after the task is completed (e.g. placing the bowl, then returning to re-grasp it); (ii) under open-loop generation the generated actions gradually drift away from the generated video — the arm pose implied by the actions and the pose shown in the frames disagree.

Two rectification steps address these in sequence. Truncation uses an agentic VLM evaluator: given the task instruction and annotated frames of a candidate rollout, the VLM locates the last successful step and the rollout is cut there; rollouts the VLM judges as not accomplishing the task are discarded entirely. Physical-consistency filtering then scores the surviving rollouts: the generated actions are rolled out in a robot-only simulator (no scene) to obtain the optical flow those actions imply, a 3D tracker (TAPIP-3D) estimates the flow actually visible in the generated frames, and the discrepancy between the two gives a per-rollout s_flow score; the most consistent rollouts are kept. The surviving rectified rollouts R_{k-1} join the cumulative replay set, and training on task k interleaves fresh task-k data with sampled rollouts from the replay set as in standard ER — only the “experience” is self-generated and rectified, not real.

Results

Benchmarked across four LIBERO suites (Spatial, Object, Goal, 10) learned sequentially with three seeds, 20 episodes per prior task. Headline: R²-WAM averages 92.7% SR / 2.33 NBT / 94.7% plasticity, within 1.9 points of real-data ER’s 94.6% / 0.09 / 94.6%, and 80.2 points above sequential fine-tuning’s 12.5% / 80.19 / 94.4%. Raw generative replay R-WAM reaches 70.4% / 16.46 / 89.1% — i.e. ~57 of the 82 points the rectifications contribute come from the two targeted interventions on top of the base self-regeneration mechanism. Classical continual-learning baselines (EWC 13.3%, LwF 14.9%, Seq-LoRA 11.1%) do not materially improve on Seq-FT in this setting [Table on LIBERO 4-suite benchmark].

The LIBERO-Spatial ablation isolates which rectification matters where: R-WAM 76.1 → R²-WAM w/o truncation 70.8 → w/o filtering 80.8 → full 94.0% SR. Dropping truncation makes R²-WAM worse than raw R-WAM, which the paper attributes to the purposeless-motion tail interfering with the physical-consistency score and making filtering unreliable [Table — LIBERO-Spatial ablation].

On a real xArm + dexterous-hand platform executing 5 sandwich-making tasks sequentially (each with 4 stages, scored by progress rate), R²-WAM scores 89.5% SR vs ER’s 91.8% and Seq-FT’s 28.5%, retaining task 1 (steak) through the full curriculum without any steak data in replay. A Seq-FT ablation on task 1 across checkpoints shows the forgetting curve directly: 4/4 progress immediately after learning the task → 0/4 after four more tasks [Table — Real-world sandwich-making].

The teacher-forcing probe decomposes the WAM’s two internal capabilities. On LIBERO-Spatial, as later tasks are added, the inverse-dynamics channel (next-action prediction given ground-truth observations) collapses on prior tasks while the video-prediction channel (next-frame generation) stays visually plausible. This is the empirical ground for the central claim: raw generative replay works at all because the WAM still imagines the past correctly; what breaks is only its ability to act in it, and that is exactly what the replay data needs to repair [§ Mechanism / teacher-forcing probe, Fig.].

Why it’s interesting

R²-WAM gives the cleanest filed answer to a question the WAM lineage (Causal World Modeling for Robot Control (LingBot-VA) LingBot-VA, Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2, Long-WAM: Scaling the Context of World-Action Models Long-WAM, InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data InternW0-Δ) has implicitly deferred: how to extend a shipped WAM to new tasks when its training data is private or unavailable, which is the typical weights-only deployment setting. The structural insight — that the two heads of a joint video+action model degrade asymmetrically under sequential fine-tuning, with inverse dynamics failing first and video prediction retained — complements the EgoWAM target-ablation finding (EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data) that the WAM head is the channel through which human/human-like data enters the policy, now applied to the WAM’s own generated past. It also sharpens the “WAM as self-generating data engine” framing against earlier WFM-as-data-engine entries like Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model XR-U0 (world model augments real demos) and Argus: An Open-Source Annotator for Robotics Data Argus (external VLM annotator audits a static corpus) — here the WAM audits and filters its own regenerated rollouts via a VLM, with no real data touched.