Rhoda AI — Web-video pretraining scaling laws for robot policy learning (DVA)

TL;DR

Rhoda AI’s Tongzhou Mu reports the first public quantitative scaling-law result for their Direct Video-Action (DVA) model: pre-training a causal video model on general web video (no actions) and then post-training on ~10 h of teleoperated robot demos for one industrial bearing-unpacking task. Across 7 checkpoints spanning 4 model sizes (XS/S/M/L) and 4 compute budgets at the M size (0.08×, 0.18×, 0.37×, 1×), at-speed completion rate on real hardware rises monotonically from 4% → 85% with model size (still rising at L) and rises predictably with pre-training compute at fixed size. The key methodological claim: DINO Fréchet Distance (DINO FD) on held-out web video ranks all 7 checkpoints in the same order as the robot task performance — providing a pre-training-measurable proxy for downstream robot success that avoids the expensive post-training + robot-eval loop.

Key claims

Scaling model size at fixed pre-training + post-training recipe raises real-robot at-speed completion (every requirement met within 100 s) on the bearing-unpacking task from 4% (XS) to 65% (S) to 75% (M) to 85% (L), with L still rising [Figure 2].
Pre-training compute at fixed M model size (0.18×, 0.37×, 1×) helps at every post-training data fraction tested (12.5%, 25%, 50%, 100% of ~11 h robot demos); the three budgets are within 8 points at the full demo set and 31 points apart at a quarter of the demos (p=0.07) — more pre-training compute matters most when robot demos are scarce [Figure 3].
DINO FD on held-out web video ranks all 7 checkpoints in the same order as the real-robot at-speed completion rate, providing a pre-training-measurable proxy for downstream robot policy quality [Figure 4].
The result is stated as the first public quantitative measurement of Rhoda’s core bet — that general web video pretraining (little of it robot-related) transfers to real industrial manipulation performance [thread §5].
Every eval trial was set up to a written protocol; 100 to several hundred robot trials per policy, over 200 real-robot hours total [thread §1].
Bayesian posteriors over true completion rate are reported per policy (violin plots) rather than point estimates, with pairwise 95%-confidence letter groupings [Figures 2, 3, 4].

Method

Two-stage pipeline. Stage 1: causal video model pre-trained from scratch on general web video (no actions) — same architecture family as the DVA blog post (Direct Video-Action Models — Causal Video Models Are Data-Efficient Robot Policy Learners), 4 model sizes (XS/S/M/L), each trained until validation loss flattens (Kaplan-style stopping). At the M size only, three intermediate pre-training-compute checkpoints (0.08×, 0.18×, 0.37×, 1× of the full M-size compute) are also kept for the compute-axis sweep. Stage 2: post-train each checkpoint on tele-op demos of one industrial bearing-unpacking task, then deploy on the real robot for evaluation. For the model-size sweep, all 4 sizes use the same recipe and full post-training data. For the compute-axis sweep, the 3 M-size compute budgets are each post-trained on 12.5%, 25%, 50%, and 100% of the robot demos. The proposed pre-training proxy metric is DINO Fréchet Distance on a held-out web-video set: encode both generated and real videos with a frozen DINO backbone, fit a Gaussian in feature space, and take the Fréchet distance — lower means the pre-trained video model better predicts held-out web video.

Results

Model-size axis (fixed pre-training + post-training recipe): at-speed completion rate 4% → 65% → 75% → 85% across XS/S/M/L, still rising at L [Figure 2].
Compute-axis (fixed M size, 3 budgets): three budgets tie at 100% post-training data, fan apart as post-training data shrinks; at 12.5% data the range is 31 points and p=0.07 [Figure 3].
Pre-training proxy: DINO FD on held-out web video (log-scale, axis reversed so lower FD is to the right) is monotonically correlated with real-robot at-speed completion rate across all 7 checkpoints, including circles (full post-training data) and squares/triangles/diamonds (50%/25%/12.5% post-training data) [Figures 1, 4].
No comparison against non-DVA baselines (VLAs, world-model-as-evaluator, etc.) is reported in the thread.

Why it’s interesting

Sits at the intersection of two threads on the wiki. First, adds a robotics-substrate datapoint to Hyperparameter scaling laws, joining the recent embodied-scale results — Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models‘s 1M-h video scaling curve, EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data‘s R²=0.9983 log-linear egocentric-hours-vs-loss fit, Isaac 0.5: An Open-Weight Embodied Foundation Model‘s 210× video-vs-teleop exchange rate, and Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training‘s UMI-pretraining scaling that transfers through post-training. Rhoda’s contribution is a joint (model size × pre-training compute × post-training data) sweep on a single deployed task with real-robot evaluation and Bayesian posteriors — sharper than most prior VLA scaling reports that use offline validation loss or benchmark scores. Second, the DINO FD proxy is a concrete instance of the VLM-as-Evaluator pattern applied to pre-training oracles rather than post-hoc quality scoring: a video-generation quality metric that ranks checkpoints the same way real-robot deployment does, so pre-training recipe changes can be evaluated without post-training + real-robot eval. Complements Direct Video-Action Models — Causal Video Models Are Data-Efficient Robot Policy Learners (DVA architecture description) with the missing quantitative scaling evidence, and contrasts with 1X World Model (1XWM) — action-controllable world model for humanoid policy evaluation and Evaluating Gemini Robotics Policies in a Veo World Simulator where the world model is used as a policy evaluator rather than as the policy itself.