Is Diversity All You Need for Scalable Robotic Manipulation?

TL;DR

Shi et al. (Shanghai AI Lab / HKU / AgiBot) run a systematic study of which data-diversity axes actually matter for scaling VLA policies, using GO-1 and RDT on AgiBot World. They report three concrete counter-intuitions to the prevailing “more diverse is better” recipe: (1) task diversity dominates per-task demonstration quantity for downstream transfer, and downstream success obeys a clean power law in pre-training data under fixed diversity; (2) cross-embodiment training data is optional — pre-training on a single high-quality embodiment (AgiBot G1) transfers to Franka, Arx, and Piper arms with better fine-tuning scaling than OXE’s multi-embodiment recipe; (3) expert diversity — variation in demonstrator velocity — is a confounder, not a benefit. They propose a Velocity Model that debiases action chunks by choosing an observation-conditional optimal chunk length, and report a +15% average improvement on GO-1-Pro equivalent to 2.5× more pre-training data.

Key claims

Under identical pre-training sample sizes, episode-based sampling (10% of episodes per task, preserving 87-skill diversity) beats task-based sampling (10% of highest-relevance tasks) by +0.1 average score across four evaluation tasks, with the largest gains on the most semantic/spatial tasks (Make Sandwich +0.26, Pour Water +0.14) despite covering fewer on-target atomic-skill episodes (59.2% vs 71.1%) [§III-B, Fig. 4].
Under fixed task diversity, pre-training-scale to downstream-score obeys a power law $y = A - Bx^{-C}$ with Pearson $r = 0.9998$ across {no pre-train, 100K, 250K, 1M} demos, with GO-1 average score rising 0.28 → 0.47 → 0.53 → 0.58 [§III-C, Fig. 5].
Single-embodiment pre-training on AgiBot G1 (RDT-AWB) reaches parity with OXE-pretrained RDT-OXE on ManiSkill Franka at 250 fine-tuning samples/task and exceeds it thereafter, with the gap growing under a power-law relationship in fine-tuning data — despite OXE containing the Franka embodiment and matching ManiSkill data [§IV-B, Fig. 6, Fig. 7].
On real-world Agilex Piper (100 demos/task), RDT-AWB averages 0.45 vs RDT-OXE 0.40 across four tasks (Package Product 0.57 vs 0.40, Fold Shorts 0.48 vs 0.65, Clean Trash 0.47 vs 0.33, Industrial Sorting 0.27 vs 0.23) [Table I].
Expert velocity multimodality is separable from expert spatial multimodality: two demonstrations following the identical spatial trajectory A→D but at different speeds produce different chunk representations ([A→B] vs [A→B→C→D]), making spatially-equivalent motions appear as distinct training samples [§V-A, Fig. 9].
Uniform-velocity normalization and episode-level temporal normalization both fail as debiasing recipes: the former destroys task-specific velocity requirements (plug-insertion alignment, pour-water pauses); the latter cannot eliminate intra-episode speed heterogeneity across trajectory segments [§V-A].
A per-observation Velocity Model $\mathcal{V}(\mathbf{o})$ trained with MSE on a scalar velocity metric (L1 end-effector displacement normalized to $[0,1]^D$ ) predicts an optimal chunk length that is used at training time to interpolate demonstrations into a canonical-velocity representation, constrained to a bounded search range around the base chunk size [§V-A, Eqs. 1–3, Fig. 10].
The Velocity Model uses a minimal architecture — SigLIP encoder over three input images + MLP head to a scalar velocity — deliberately keeping it lightweight for training-time preprocessing [§V-A].

Method

The three studies share one experimental spine: two-phase training (pre-train on a candidate diversity mix, then fine-tune on the same target task set), evaluated with normalized per-action scores across in-domain / object-generalization / visual-distraction scenarios. Task-diversity is manipulated by two contrasting samplers over AgiBot World Beta (task-based vs episode-based) at matched dataset size. Embodiment-diversity is manipulated by comparing single-embodiment (AWB, AgiBot G1) vs multi-embodiment (OXE) pre-training in RDT, evaluated on ManiSkill (Franka), RoboTwin (Arx), and real Agilex (Piper) — three arms morphologically distinct from AgiBot G1.

The expert-diversity fix is where the paper’s method contribution lives. A Velocity Model predicts, from the current observation, the expected per-chunk velocity of a demonstration that starts there; the optimal chunk length $L^*$ is chosen by inverting this against the per-trajectory realized velocity, and the demonstration is temporally rescaled to fit the base chunk size $L_\text{base}$ . The velocity metric uses the L1 norm of the normalized end-effector displacement (Eq. 2), yielding a bounded scalar that decouples velocity supervision from raw trajectory geometry. Search over $L^*$ is constrained to a range around $L_\text{base}$ to keep interpolation-induced distortion small.

Results

Task diversity: episode-based sampling beats task-based sampling by +0.1 average score across 4 tasks × 3 scenarios; largest gains on high-semantic-content tasks [Fig. 4].
Pre-training scaling: power-law fit $y = A - Bx^{-C}$ with Pearson $r = 0.9998$ ; scores 0.28 (no pre-train) → 0.58 (1M demos) under fixed diversity [Fig. 5].
Cross-embodiment (ManiSkill sim): RDT-AWB matches RDT-OXE at 250 samples/task and pulls ahead with more data; power-law slope favors AWB [Fig. 6].
Cross-embodiment (RoboTwin sim, Arx): RDT-AWB reaches comparable performance to RDT-OXE with minimal fine-tuning data [Fig. 8].
Cross-embodiment (Agilex real, Piper): RDT-AWB wins on 3 of 4 tasks, average 0.45 vs 0.40 at 100 demos/task [Table I].
Expert-diversity debiasing: GO-1-Pro (GO-1 + Velocity Model) achieves +15% average performance gain, equivalent to using 2.5× pre-training data per the paper’s power-law fit [Abstract, §V].

Why it’s interesting

This paper is the sharpest filed diagnostic on the “which diversity axis matters” question that VLA Models has been implicitly debating across multiple recipes — cutting against the “clean teleop is the lever” position (Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models) by identifying velocity variation as the confounder that “clean” data implicitly controls for, and against the “scale-of-embodiment-free-UMI-data” position (Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training) by showing that fixing collection to one high-quality robot embodiment is a viable alternative to embodiment-free collection tools. The velocity-model debiasing recipe adds an action-representation-side lever to the Human-to-Robot Retargeting discussion of representation-side interventions (translation-as-bridging-action drops rotation, ACE-Ego-0 keeps everything with reliability weighting) that specifically attacks the temporal rather than kinematic dimension of demonstrator variability. The power-law $r=0.9998$ under fixed task diversity is one of the strongest scaling-law fits on file for VLA pre-training data volume, sitting alongside EgoScale’s log-linear egocentric-hours fit (EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data $R^2=0.9983$ ) and XR-1’s UMI-hour curve.