Skip to content
Bud's Wiki

Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models

TL;DR

Section titled “TL;DR”

Dyna-2 is Dyna Robotics’s world-action model (WAM) pre-trained on >1,000,000 hours of egocentric human video (~170 human-years) — a 50× jump over EgoScale’s 20,854 h and the largest filed human-data pre-training run for a manipulation policy. Two headline scaling laws come out of the sweep of nested {1k, 10k, 100k, 1M}-hour subsets: (i) a clean scaling law on held-out human action prediction across four orders of magnitude with no plateau, and (ii) — for the first time — a scaling law on never-seen robot data trained with zero robot examples, monotonically improving offline prediction across 39 tasks on two YAM bimanual embodiments as human hours grow. The mechanism is not the action loss but the world-modeling objective: joint denoising of future video + future action beats action-only supervision on 39 of 39 tasks at every scale, and video co-training on additional unlabeled human video is the specific lever that makes the cross-embodiment gain scale with data. Post-training carries the 1M-hour prior through to real hardware (20% → 28% → 45% → 53% of max on 14 tasks) and unlocks tasks unsolvable at smaller scales (Lockbox Key Turning 0/× at ≤100k h → 90% at 1M). A one-step distillation drops the 100-NFE teacher video generator from 10,203 ms to 110 ms on H100 at near-teacher FVD, and Dyna-2 achieves 87% quality/throughput at zero-shot customer deployment vs the production Dyna-1 VLA’s 46%.

Key claims

Section titled “Key claims”
  • Scaling law on human data holds at 1M hours with no plateau; across nested subsets {1k, 10k, 100k, 1M} h, all four metrics (MSE, L1, accuracy@0.5, accuracy@0.1) improve monotonically, with accuracy@0.1 rising 51% and MSE falling 12% end-to-end, fit by power law y = a·N^{-b} in hours [§3, Fig. 5].
  • First filed cross-embodiment transfer scaling law: same human-data-only checkpoints, evaluated zero-shot on 39 held-out YAM bimanual robot tasks (12 internal + 27 xdof ABC), show monotone offline-metric improvement as human hours scale, with an inflection between 10k and 100k h suggesting an emergence-like threshold for cross-embodiment transfer [§3, Fig. 7].
  • The pre-training scaling gain carries through to real hardware after post-training: mean normalized score on 14 real-robot tasks moves 20% → 28% → 45% → 53% of maximum across the {1k, 10k, 100k, 1M} rungs, and Dyna-2@1M is best on 9 of 14 tasks. Several tasks are unsolvable below a threshold: Lockbox Key Turning is 0 at ≤100k h and reaches 90% at 1M h; Bottle Cap Untwisting rises 10% → 40–50% from ~10 min of robot demonstrations [§3, Fig. 8].
  • World modeling is what enables cross-embodiment transfer to scale — action-only supervision loses on 39 of 39 tasks at every action-data scale, and joint denoising (predict future action + future video jointly) is the winning recipe; joint alone overfits less than action-only but does not scale with data unless augmented with additional unlabeled human video prediction [§3, Fig. 10].
  • Video is a first-class scaling axis independent of action data: at two fixed action-data amounts, scaling video-only pre-training improves cross-embodiment robot-data generalization monotonically, while scaling video has no impact (and slight regression) on same-embodiment human evaluation — the video prior’s contribution is specifically the embodiment gap [§3, Figs. 11, 12].
  • Architecture is a Mixture-of-Transformers world-action model on a video-diffusion backbone: per-modality DiT stacks (video, action) cross-attend via shared attention; video tokens use causal masking, action tokens use bidirectional self-attention over observed video only, text conditions video (not action directly). Action transformer is deliberately shallower and joins the video stream only at early layers to cut real-time inference latency [§2, Fig. 3].
  • Training data is 1M+ hours of egocentric human manipulation with derived pseudo-action supervision: 3D hand-pose tracks → wrist end-effector trajectories + continuous grasp signal from thumb–index aperture, no embodiment-specific data processing to close the visual/kinematic gap. Nested exact-hours subsets (1k / 10k / 100k / 1M) preserve source-mixture proportions so scaling-curve differences cannot be explained by distribution shift [§3].
  • Evaluation reports two continuous errors (MSE, L1) and two thresholded accuracies (@0.5 for motion intent, @0.1 for precision) to guard against metric-specific artifacts, following the Schaeffer emergence-is-a-mirage critique [§3].
  • In an apple-to-apple head-to-head under matched data + hyperparameters + three seeds each, an early Dyna-2 (no 1M-hour pretrain, action-only loss) beats production VLA Dyna-1 (Qwen3-VL-3B-based) with 1.55× success rate and 1.12× quality grade across 7 post-training tasks × 3 pre-training checkpoints, winning 65% of head-to-head cells vs 29% for the VLA (6% tied) — despite the pipeline being tuned for the VLA [§4, Fig. 13].
  • Zero-shot deployment at customer sites: Dyna-1 passes the joint quality/throughput/reliability production criterion 46% of the time, Dyna-2 passes 87% — a 41-point gap at identical post-training budget, even though both reach ~100% task completion on in-distribution eval [§4, Fig. 14].
  • Instruction following improves with world-modeling objective + video scale on a controlled benchmark (jenga push/pull, object kitting, piece stacking, napkin manipulation) where the scene is fixed and only the language instruction varies — targeted drink retrieval rises 58% → 83% across the pre-training scale ladder [§3, §4, Fig. 15].
  • One-step video distillation: a control-problem formulation where the student is matched against an evolving target distribution (rather than a fixed teacher) drops NFE from 100 (teacher) to 1 (student) at near-teacher FVD (121 vs 80) with only 1.94 vs 2.69 flicker — DMD2-1step by contrast collapses to FVD 599 / motion 56%. Sampler wall-clock: 10,203 ms → 110 ms on one H100 bf16 for a 3-second, 3-view robot video [§4, Fig. 17, Table].
  • Case study on chopping-celery real hardware: Dyna-2 cuts thinner and more uniform pieces than Dyna-1 on the same post-training config; robust to altered lighting, dark chopping, partial visual-input loss (top-cam covered), and adversarial state perturbation (an observer restoring pieces to the board mid-run — Dyna-2 keeps clearing until the board is actually empty, not for a fixed cycle count) [§4].

Method

Section titled “Method”

Dyna-2 is a video-diffusion Mixture-of-Transformers WAM. Two per-modality DiT stacks (video, action) share attention across every block; video tokens use causal masking, action tokens use bidirectional self-attention and attend only to observed-context video (whether they may attend to predicted future video is a per-experiment objective toggle). Proprioception is a token stream fed into the action transformer; text is a cross-attention condition on the video stream only. The action transformer is deliberately shallower than the video transformer and joins the video stream at early layers, which the paper argues cuts real-time inference latency without hurting performance. Training uses flow matching on straight-line noise paths for both modalities. Pseudo-action supervision on the human data comes from a 3D hand-pose pipeline: retargeted wrist end-effector trajectories plus a continuous grasp signal from thumb–index aperture; no visual or kinematic bridging is applied.

For the scaling-law sweep, four nested subsets of exactly {1k, 10k, 100k, 1M} hours are drawn with identical source-mixture proportions, so a larger budget never exchanges data — it only adds it. A separate 100-hour validation set is held fixed across all runs. Training hyperparameters and configurations are identical across rungs; hours of experience is the sole independent variable. Model size and compute scaling are left to future work.

Post-training uses a modest per-task teleop dataset and re-uses the same MoT architecture. The one-step video distillation formulation reframes distillation as a control problem: the teacher target is not fixed but moves with the student, so the target stays inside the student’s reachable band throughout training — the paper argues this is what fixes the “locally straight, globally curved” teacher-flow problem that breaks image-side one-step distillation recipes when transplanted to video.

Results

Section titled “Results”
  • Human-side scaling law [§3, Fig. 5]: monotone across four orders of magnitude on all four metrics; accuracy@0.1 rises 51%, MSE falls 12% end-to-end, no plateau.
  • Cross-embodiment scaling law [§3, Fig. 7]: monotone on 39 held-out YAM tasks under zero robot data in pre-training; inflection between 10k–100k h.
  • Post-training on real hardware [§3, Fig. 8]: 20% → 28% → 45% → 53% of max normalized score across the ladder; best on 9/14 tasks at 1M h; Lockbox Key Turning 0 → 90% at 1M h; targeted drink retrieval 58% → 83%.
  • Objective ablation [§3, Fig. 10]: joint (video + action) beats action-only on 39/39 tasks at every action-data scale. Only joint + video co-training on extra unlabeled human video keeps scaling as action data grows.
  • Video-axis ablation [§3, Figs. 11, 12]: at fixed action data, video-only scaling improves cross-embodiment robot generalization; has no impact (slight regression) on same-embodiment human evaluation.
  • WAM vs VLA head-to-head [§4, Fig. 13]: early Dyna-2 (pre-1M, action-only) wins 65% vs Dyna-1 29% (6% tied) across 21 task×checkpoint cells; 1.55× success rate, 1.12× quality grade — under a pipeline tuned for the VLA.
  • Customer-site deployment [§4, Fig. 14]: Dyna-2 87% vs Dyna-1 46% on joint quality+throughput+reliability pass criterion.
  • One-step distillation [§4, Table]: 1 NFE, 110 ms sampler on H100, FVD 121, motion 75%, flicker 1.94 — vs 100-NFE teacher (10,203 ms, FVD 80, motion 94%, flicker 2.69) and DMD2 1-step (109 ms, FVD 599, motion 56%, flicker 5.81).

Why it’s interesting

Section titled “Why it’s interesting”

Dyna-2 is the sharpest filed data point yet for World Foundation Models as the substrate for embodied policies scaled with human video — it stakes out the strongest version of the “video is the new pre-training axis for robotics” thesis and delivers the first quantitative cross-embodiment transfer scaling law to back it. It complements EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data on the same axis at 50× the data scale, but takes the opposite side of the objective debate: EgoScale bets on flow-matching action supervision on retargeted 22-DoF joint actions and reports the joint-space channel wins; Dyna-2 instead claims action-only loses on 39/39 tasks at every scale, and video co-training is the specific mechanism that lets the human-data investment cross the embodiment gap. Where EgoScale needs an aligned mid-training stage to get one-shot transfer, Dyna-2 reports zero-shot offline-metric transfer directly from pretraining scale.

It also settles a version of the VLA Models recipe debate on the same lab’s own bench: an early WAM beats a production VLA under a matched training pipeline tuned for the VLA, and the deployment-side quality/throughput gap (46% → 87%) is much larger than the in-distribution completion gap (both ~100%) — evidence that world-modeling structure buys robustness that VLA-tier completion metrics don’t measure. This is a concrete answer to the recurring open question on the VLA Models page about whether world-modeling peers (NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry Cosmos+GR00T, μ₀: A Scalable 3D Interaction-Trace World Model µ₀, Causal World Modeling for Robot Control (LingBot-VA) LingBot-VA) are actually a better default than end-to-end VLM+action-head recipes at scale — and unlike prior work, the head-to-head is apples-to-apples on the same data + hyperparameters.

The two scaling laws also give the Hyperparameter scaling laws concept its second clean embodied-data-axis fit at scale (after EgoScale’s R²=0.9983 log-linear result at 20k h), now extended by 50× and with an explicit cross-substrate extrapolation from human to robot — filling a gap Scaling Laws, Carefully explicitly excluded from the LLM scaling-law canon. On the Human-to-Robot Retargeting axis, Dyna-2 uses only wrist EE + thumb-index aperture (not the 22-DoF joint-space retargeting of EgoScale nor the drop-rotation minimalism of Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots), and pays for the coarseness with pretraining scale + world-modeling supervision rather than with a richer action representation — a distinct point on the interface-location design space.

See also

Section titled “See also”