World Action Models are Zero-shot Policies (DreamZero)

TL;DR

DreamZero (NVIDIA; Ye, Ge, Zheng et al.) is the paper that introduced World Action Models (WAMs) as a distinct recipe from VLAs: a 14B pretrained video-diffusion backbone jointly predicts future world states and actions, using video as a dense representation of how the world evolves rather than only as upstream vision for an action head. Reports >2× generalization over SOTA VLAs on new tasks/environments in real-robot experiments; 7 Hz closed-loop control at 14B via model+system optimization; +42% improvement on unseen tasks from 10–20 min of video-only human or other-robot demonstrations; adapts to an entirely new robot (YAM) from 30 min of play data while retaining zero-shot task generalization. Direct predecessor of Zero-Shot to New Robots: Introducing Dream-1 Dream-1, from the same team.

Key claims

A WAM — video-diffusion backbone trained to jointly predict future frames and actions — generalizes to unseen tasks and environments ~2× better than SOTA VLA baselines on AgiBot-pretrained evaluation, with pretrained VLAs defaulting to pick-and-place regardless of the instruction (39.5% task progress vs near-zero for VLAs trained from scratch; 62.2% average on seen-task-unseen-environment vs 27.4% for best pretrained VLA baseline) [§AgiBot Pretraining, project page §1].
Training on DROID (one of the most heterogeneous open robotics datasets) with 20 seen tasks + 20 tasks containing unseen verbs yields 49% task progress on the unseen-verb set vs 25–32% for SOTA VLAs — evidence that the video-dense substrate transfers under heterogeneous data and OOD verb coverage [§DROID results].
Post-training on three downstream tasks (shirt folding, fruit packing, table bussing) retains environment generalization — a counter-datapoint to the “fine-tuning erases pretraining generalization” concern raised against VLM-with-action-head VLAs. Table bussing (5 trash + 5 dishware) reports 81% task progress over 10 consecutive uncut-recording evaluations [§Post-Training].
30 minutes of play data (55 trajectories) adapts DreamZero to the YAM robot while preserving zero-shot generalization to novel objects (pumpkins, teddy bears, paper bags) with strong language-following — characterized as the most efficient embodiment transfer yet (hundreds-of-hours → 30 min) [§New Embodiment Adaptation].
10–20 minutes of video-only demonstrations from humans or other robots yield +42% improvement on unseen tasks — the WAM’s video prediction channel is the interface through which human/other-robot video data enters without paired action labels [§Abstract].
Through model + system + implementation optimization, DreamZero reaches 150 ms per action chunk → 7 Hz closed-loop control at 14B parameters; DreamZero-Flash maintains performance down to single-step inference [§Real-Time Inference].
Asynchronous inference + action-chunk smoothing combine with the step-count reductions to produce smooth, responsive execution — the serving stack is as load-bearing as the modeling choices for 14B real-time deployment [§Real-Time Inference].
Interactive prompting is validated as a workable deployment pattern: operators walk the robot around and verbally prompt new tasks (e.g. “Move forward and press the elevator button with right arm”), with the robot executing new skills zero-shot [§Interactive Prompting].

Method

DreamZero is a 14B autoregressive video-diffusion WAM. The core architectural commitment is that video and actions are jointly predicted on the same backbone — rather than treating video prediction as an auxiliary pretext task for a separate action head, DreamZero makes video prediction the substrate the policy is, with actions emitted alongside future frames. This factorization is what lets heterogeneous data streams (robot teleop, video-only human demonstrations, other-robot videos) enter via the shared video channel without paired-action alignment.

For real-time control, model/system/implementation optimizations collapse per-chunk inference to 150 ms (7 Hz closed-loop) for the full 14B model; DreamZero-Flash reduces to single-step inference while preserving performance. Deployment uses asynchronous inference + action-chunk smoothing to turn the step-rate into smooth continuous behavior.

Pretraining uses the AgiBot corpus; separate validation on DROID confirms the recipe works on heterogeneous open data. Post-training follows a standard fine-tune-per-task pattern (shirt folding / fruit packing / table bussing) with the key finding that environment-generalization is retained post-fine-tune. Embodiment adaptation requires only 30 min of robot play data for YAM.

Results

AgiBot pretraining, seen tasks on unseen environments: DreamZero 62.2% average task progress vs best pretrained VLA baseline 27.4% (>2×). VLAs trained from scratch near-zero. Evaluated on 80 rollouts/checkpoint across 4 robots in different environments with different objects [§1].
AgiBot unseen tasks (untying shoelaces, shaking hands — entirely absent from training): DreamZero 39.5% task progress; VLAs again struggle, defaulting to pick-and-place [§1].
DROID, 20 seen tasks + 20 unseen-verb tasks: DreamZero 49% on unseen verbs vs 25–32% for pretrained-VLA baselines [§2].
Post-training: environment generalization retained on all three downstream tasks (shirt folding, fruit packing, table bussing). Table bussing (5 trash + 5 dishware): 81% task progress over 10 consecutive evaluations in a single uncut recording [§3].
New embodiment adaptation: 30 min of YAM play data (55 trajectories) suffices for zero-shot generalization to novel objects and strong language-following; no other YAM data used [§4].
Video-only cross-embodiment transfer: +42% on unseen tasks from 10–20 min of video-only human or other-robot demonstrations [§Abstract].
Real-time inference: 150 ms/chunk → 7 Hz closed-loop at 14B; DreamZero-Flash preserves performance down to single-step inference; async inference + chunk smoothing yield smooth deployment [§6].
Interactive prompting demonstrates emergent capabilities (fanning burgers, pressing elevator buttons, playing xylophones, shaking tambourines) across 100+ zero-shot task rollouts in the project page gallery [§What’s Next].

Why it’s interesting

DreamZero defined “World Action Model” as a named counter-recipe to VLA on the VLA Models recipe-lever board — the earliest filed instance of the “video-dense prediction is the policy substrate, actions are a side channel” architectural commitment that Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2 later scaled to million-hour human-video pretraining, Causal World Modeling for Robot Control (LingBot-VA) LingBot-VA unified into causal AR flow-matching, and Zero-Shot to New Robots: Introducing Dream-1 Dream-1 productized into a public API with EE-pose shared-action-space for cross-embodiment transfer. The headline 30-min-YAM-adaptation + preserved-zero-shot result is the specific empirical anchor the Dream Labs founders cite as what made them believe the data chicken-and-egg loop was breakable; everything on Dream-1’s recipe inherits from here. The 7 Hz closed-loop at 14B is also the sharpest filed datapoint for “real-time control from a video-diffusion backbone” and sits alongside RealtimeWAM: One-Step Asynchronous World Action Models RealtimeWAM’s cross-expert pipelining as independent evidence that WAM inference latency is a solvable engineering problem rather than a structural blocker.