DreamFree: Future Representation Distillation for Imagination-free World Action Models

TL;DR

DreamFree (KAIST; Jo, Jang, Yu, Ye, Nah, Jung, Lee, Hwang — shared authors with World Action Models are Zero-shot Policies (DreamZero) DreamZero) stakes out an imagination-free position on the WAM recipe board: keep the video-diffusion backbone’s predictive prior as a representation-distillation target but drop future-video generation entirely at inference. A frozen video-diffusion teacher observes current + future frames and supplies future-aware features; an alignment loss trains the student’s current-frame features to predict those teacher representations alongside the standard video generative objective; a readout + action decoder then extracts action chunks from the student’s visual encoder at inference. Reports 11 Hz closed-loop on a single H200 and 17.8× inference speedup over DreamZero at 14B, with real-world DROID success 64.0% / 83.0% progress vs DreamZero’s 56.0 / 71.8 and π0.5’s 16.0 / 33.5 — and the first filed scaling signature on this recipe where larger backbones and video-only extra data both pay off (RoboCasa 58.5 → 71.7 → 74.4% at 5B → 14B → 14B+videos; Fast-WAM gains only +0.3pp across the same backbone sweep).

Key claims

Future-representation distillation enables backbone scaling on the imagination-free WAM recipe where generative training alone does not — RoboCasa success rises 58.5 → 71.7% when the video backbone scales from 5B → 14B under DreamFree, versus 44.4 → 44.7% (+0.3pp) for Fast-WAM under the same scaling [§Evaluation results, Scaling table].
Extra video-only data improves policy performance without additional action labels — RoboCasa 71.7 → 74.4% and Fast-WAM 44.7 → 68.8% at 14B when extra videos are added to the visual encoder pretraining stage [§Evaluation results].
On RoboCasa-Kitchen (24 tasks), DreamFree 14B reaches 71.7% success vs π₀ 62.5, GR00T-N1.5 64.1, FLARE 66.4, Cosmos Policy 67.1; +74.4% with extra videos [§Comparison with VLA and WAM baselines table].
Real-world DROID (10 tasks × 5 trials/policy, 50 trials total): DreamFree 64.0% success / 83.0% task progress vs DreamZero 56.0 / 71.8 and π₀.₅ 16.0 / 33.5 [§DROID real-world evaluation].
Inference cost on H200: DreamFree 91 ms/chunk (11 Hz closed-loop control) vs DreamZero 1,632 ms (17.8× speedup), without video step distillation or asynchronous execution [§Real-time control at 11 Hz].
Memory: without future-video generation or classifier-free guidance at inference, DreamFree fits on a single A6000 48 GB [§TL;DR].
Stage 1 trains the visual encoder’s current-frame features to predict a frozen video-diffusion teacher’s future-aware representations via an alignment loss, alongside the original video generative objective — needs no action labels [§Method, Stage 1].
Stage 2 trains a lightweight readout module to extract action-relevant features from the (now future-aware) visual encoder, then an action decoder emits executable action chunks [§Method, Stage 2].
Composite-task extra pretraining lets the model chain learned atomic skills into unseen sequences without any composite-task action labels (microwave open → put food → close → turn on) [§Composite videos in pre-training video-only fine-tuning].
Known failure modes on sparse-data setups: folding clothes and whiteboard erasing remain unreliable; t-shirt folding failure likely reflects difficulty inferring cloth state. Longer training not yet investigated [§Failure Cases].

Method

DreamFree is a two-stage recipe on top of a video-diffusion WAM backbone. Stage 1 (visual encoder with future representation distillation). A frozen teacher video-diffusion model takes current and future frames and produces future-aware internal representations. A student visual encoder sees only the current frame and is trained under the original video-generation objective plus an alignment loss that pushes its current-frame features toward the teacher’s future-aware features. Because the alignment target is a representation rather than pixels, no action labels are required — training uses unlabeled video. The video backbone is scaled from 5B to 14B; extra composite-task videos can be added here without any action labels. Stage 2 (readout + action decoder). With the visual encoder frozen after Stage 1, action-labeled robot demonstrations train (a) a readout module that extracts action-relevant features from the visual encoder and (b) an action decoder that maps those features to executable action chunks.

Inference. Only the current frame is observed. The visual encoder (whose features now carry future-aware structure from Stage 1 distillation) produces features, the readout extracts action-relevant features, and the action decoder emits an action chunk. No future-video denoising rollout, no CFG at inference — the two levers responsible for DreamZero’s ~1.6s per-chunk latency are both absent. On H200 the full 14B model runs at 91 ms per action chunk / 11 Hz closed-loop.

Results

RoboCasa-Kitchen (24 tasks, 50 demos/task). DreamFree 14B 71.7% (+ extra video: 74.4%) vs π₀ 62.5, GR00T-N1.5 64.1, FLARE 66.4, Cosmos Policy 67.1 [Comparison table].
Backbone scaling. Fast-WAM 5B→14B: 44.4 → 44.7% (+0.3pp). DreamFree 5B→14B: 58.5 → 71.7% (+13.2pp). DreamFree + extra video 5B→14B: 68.8 → 74.4% (+5.6pp) [Scaling table].
DROID real-world (10 tasks × 5 trials, 50 trials total/policy). Success / Progress / Latency: π₀.₅ 16.0 / 33.5 / 81 ms; DreamZero 56.0 / 71.8 / 1,632 ms; DreamFree 64.0 / 83.0 / 91 ms [DROID table].
Inference speedup over DreamZero at 14B: 17.8× on H200, with no video step distillation or asynchronous execution — the per-chunk cost reduction comes from dropping the future-video rollout entirely, not from faster denoising [§Real-time control at 11 Hz].
Composite-task generalization. Composite-task videos added at Stage 1 (video-only) enable chained execution of learned atomic skills at test time without composite-task action labels (microwave open → place food → close door → turn on) [§Composite videos].
Failure cases (DROID + YAM pilot). Cloth folding (t-shirt) and whiteboard erasing remain unreliable with limited training data; long-training ablation not conducted.

Why it’s interesting

DreamFree lands the sharpest filed answer to the imagine-vs-no-imagine debate Fast-WAM: Do World Action Models Need Test-time Future Imagination? Fast-WAM opened and Long-WAM: Scaling the Context of World-Action Models Long-WAM pushed back on: keep the video prior but swap the training-time interface from pixel-level video generation to representation-level distillation. Fast-WAM already established that video co-training as a training-side objective dominates test-time future imagination at matched 5B scale, but its backbone-scaling signature was flat (+0.3pp from 5B → 14B); DreamFree’s +13.2pp over the same sweep is the first filed evidence that the imagination-free recipe can be made to scale, and that the lever is the alignment target, not just removing the rollout. This is a direct instance of Representation Autoencoders‘s central thesis — frozen foundation-model features as the right distillation target for a smaller downstream model — ported from image generation into VLA training, and complements EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data EgoWAM’s finding that DINO-feature prediction targets beat pixel-VAE targets for cross-embodiment transfer by picking a video-diffusion-internal feature target instead.

Shared-author lineage with World Action Models are Zero-shot Policies (DreamZero) DreamZero makes the 17.8× speedup + +8pp real-world success claim unusually credible as a within-group apples-to-apples comparison: same 14B backbone, same DROID evaluation protocol, same lab. The +48pp real-world success gap over π0.5 (16.0 → 64.0%) at ~0.11× the per-chunk latency (91 ms vs 1,632 ms DreamZero, 81 ms π0.5) is the sharpest filed Pareto datapoint on the WAM-as-policy vs VLA axis for a shared-backbone comparison. Sibling in framing to FLUX 3 x mimic: The Next Generation of Video-Action Models FLUX-mimic (frozen video backbone + small action decoder reads intermediate features at ~80 ms) and μ₀: A Scalable 3D Interaction-Trace World Model µ₀ (frozen trace WM + action expert) on the “do the imagination in training, skip it at inference” axis, but keeps the imagination channel in-backbone rather than factored out, and is the only member of the family with a published backbone-scaling signature of this shape.