Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models

TL;DR

Qwen-RobotManip is a Vision-Language-Action foundation model from the Qwen team built on Qwen-VL, designed around the thesis that alignment must come before scale in robotic manipulation. The paper introduces a unified alignment framework spanning representation (canonical state-action with per-dimension binary masking), motion (camera-frame delta-pose parameterization), and behavior (in-context policy adaptation reading intra-episode history as an implicit embodiment identifier), and combines it with a human-to-robot synthesis pipeline that renders ~1,933 hours of egocentric human videos onto 15 bimanual robot morphologies to produce ~24,808 hours of synthetic data — bringing the total pretraining corpus to ~38,100 hours from only open-source data. The team further argues that standard in-domain benchmarks fail to distinguish genuine generalization from in-distribution memorization, and introduces two new OOD benchmarks — RoboTwin-IF (instruction-following as a real control signal) and RoboTwin-XE (zero-shot cross-embodiment transfer) — alongside existing OOD suites (LIBERO-Plus, RoboCasa365, EBench, RoboTwin-Clean2Rand). Qwen-RobotManip reports beating π0.5 and GR00T-N1.7 across all OOD axes, ranking 1st on RoboChallenge Table30-v1 with a 20% relative improvement, with real-robot validation on AgileX ALOHA, Franka, UR, and ARX.

Key claims

The unified alignment framework has three components: a canonical state-action representation with per-dimension binary masking accommodates diverse robot morphologies in one template, a camera-frame delta-pose parameterization makes visually similar motions numerically proximate across coordinate frames, and an in-context policy adaptation mechanism reads intra-episode execution history as an implicit embodiment identifier [§1, §3].
Without alignment, scaling data volume produces interference rather than synergy across embodiments — alignment is a prerequisite for data scaling itself, not an independent engineering choice [§1].
The pretraining corpus is ~38,100 hours total — 11,420h real robot demonstrations (3,808h single-arm + 6,744h dual-arm + 868h mobile/humanoid) + 1,933h egocentric human videos + 24,808h synthesized human-to-robot data, all from open-source sources only [Table 1].
The human-to-robot synthesis pipeline retargets MANO hand keypoints to parallel-jaw gripper actions (virtual finger = 0.7·index + 0.3·middle; end-effector position at the midpoint of thumb and virtual finger; gripper width as their Euclidean distance), then runs SAM3 hand segmentation + ProPainter inpainting + MuJoCo IK + depth-guided compositing to replace the human appearance with each of 15 robot models [§2.3, Fig. 1, Eqs. 1-4].
Robot-base placement for ego-video retargeting is solved as an optimization over base positions maximizing keyframe IK feasibility, independently per morphology, because egocentric trajectories are embodiment-free with no reference base [Eq. 3].
A five-stage state-action filtering pipeline plus three cross-modal consistency checks is applied to all data; the audit revealed e.g. 81% of episodes in the RoboMIND UR-type subset failed state-action trend alignment and were excluded [§2.4, Fig. 2].
A dual-stream co-training strategy jointly optimizes manipulation and vision-language objectives — including a curated VL mixture with embodied chain-of-thought and egocentric video understanding — to prevent the VLM backbone’s perceptual and reasoning capabilities from eroding under action-prediction pressure [§1].
Standard in-domain benchmarks systematically fail to distinguish genuine generalization from in-distribution pattern memorization; the paper proposes OOD-only evaluation as the correct measure of foundation-model capability for VLAs [§1].
Two new OOD benchmarks are introduced: RoboTwin-IF tests whether policies condition on language as a genuine control signal rather than exploit visual shortcuts, and RoboTwin-XE tests zero-shot transfer to morphologically distinct robots [§1].
Qwen-RobotManip outperforms π0.5 and GR00T-N1.7 across all OOD evaluation axes (LIBERO-Plus, RoboTwin-Clean2Rand, RoboCasa365, EBench, RoboTwin-IF, RoboTwin-XE), ranks 1st on the RoboChallenge Table30-v1 generalist track with a 20% relative improvement, and is validated on AgileX ALOHA, Franka, UR, and ARX real-robot platforms [§1].

Method

The model takes the Qwen-VL backbone (Qwen3-VL family per the cited Yang et al. 2025 / Bai et al. 2025 / Team 2026 references) and attaches the alignment framework around it. The canonical state-action representation packs heterogeneous embodiment information (joint positions, EE pose, gripper, dexterous-hand DoFs) into a fixed unified vector with a per-dimension binary mask indicating which slots are active for the current robot. The camera-frame delta-pose parameterization expresses the next end-effector pose as a delta in the camera’s coordinate frame rather than the world frame — so the same visual motion (e.g. “move 5cm to the right of the camera view”) receives the same numerical action target regardless of where the robot base sits or how the camera is mounted, which the paper argues is the geometric basis for cross-embodiment numerical consistency. The in-context policy adaptation mechanism feeds intra-episode action history into the context so the model can implicitly identify the current embodiment from how the robot has been moving, eliminating the need for explicit embodiment tokens. The dual-stream co-training loss mixes manipulation supervision with a curated VL stream (embodied CoT data + egocentric video understanding), the latter explicitly designed to preserve VLM-level perception under action-prediction pressure.

The data engine has two halves. The scaling engine converts ~1,933h of egocentric human video into ~24,808h of synthetic robot demos: hand keypoints → MANO + virtual-finger formula → gripper pose + width; Savitzky-Golay + Gaussian-SLERP smoothing; SAM3 + ProPainter to clean the human appearance; per-morphology IK-feasibility search to place a virtual robot base; MuJoCo render + Depth-Anything-v3 composite. The curation engine runs the five-stage signal-quality pipeline (sudden-change detection via residual + acceleration + jerk; state-action trend alignment via directional-agreement on cross-correlation lag; extreme-value filtering via per-embodiment q01/q99 bands; FK consistency via Pinocchio vs. logged EE pose; base-frame + orientation alignment) plus three cross-modal checks (VLM-judged instruction consistency with structured-reasoning prompting and multi-expert adjudication; video-state consistency via URDF reprojection + SAM3 IoU; video-quality filtering for black/corrupted/blur/static segments). Action-speed alignment downsamples each ego source to match robot teleop speed (EgoDex 60%, EgoVerse 45%, ViTRA 25%).

Results

Corpus scale: ~38,100h total pretraining data assembled only from open sources (no proprietary collection): OXE (Fractal + Bridge + BC-Z, ~600h), AgiBotWorld-Beta (~2,400h gripper subsets), RoboMIND + RoboMIND 2.0 (~1,400h, 10 embodiments combined), Galaxea Open-World (~500h), RoboCOIN (~430h across 10 embodiments), DROID (~500h, ~95k Franka trajectories), RH20T (~1,100h, 140+ tasks across 4 embodiments), RDT-1B (29h ALOHA), InternData-A1 (~3,600h sim); + EgoDex (732h used of 829h) + VITRA-Ego4D+EPIC-KITCHENS subsets (~247h) + EgoVerse industry portion (~954h); + ~24,808h human-to-robot synthesized across 15 robot configurations [Table 1].
Curation reveal: 81% of RoboMIND UR-type episodes failed state-action trend alignment (DA threshold 0.6-0.7) and were excluded — an example of how unsynchronized clocks / packet loss silently corrupt manipulation corpora [§2.4 Stage 2].
OOD benchmark performance: substantially outperforms existing VLA models including GR00T-N1.7 and π0.5 across LIBERO-Plus, RoboTwin-Clean2Rand, RoboCasa365, EBench, RoboTwin-IF, and RoboTwin-XE [§1] (exact deltas not in the body excerpt retrieved at filing).
Competitive ranking: 1st on RoboChallenge Table30-v1 generalist track with 20% relative improvement over prior submissions [§1].
Real-robot validation: deployed on AgileX ALOHA, Franka, UR, and ARX across in-domain, OOD, few-shot adaptation, and zero-shot cross-embodiment transfer settings [§1].
Emergent capabilities reported from the open-source-only corpus: zero-shot instruction following, robustness to perturbations, reactive error recovery, and cross-embodiment knowledge transfer [Abstract].

Why it’s interesting

This is the most concrete instantiation yet of the “alignment-first” thesis for VLAs — and an explicit counter to the “scale the data, the policy will sort it out” position. It directly extends VLA Models on the Qwen-VL backbone axis, joining Spirit-v1.5: A Robotic Foundation Model by Spirit AI, LingBot-VLA: A Pragmatic VLA Foundation Model, and Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models as the fourth filed Qwen-VL-backed open VLA, but at a substantially larger scale and with a unique alignment framework that none of the others have. The “alignment as prerequisite for scale” framing is in productive tension with Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models‘s “clean teleop data is the lever” thesis (Spirit-v1.5 picks small + curated; Qwen-RobotManip picks large + aligned) and with Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models‘s “action pretraining is not strictly required” claim (Qwen-RobotManip doubles down on action pretraining but argues it only works with the alignment recipe). The human-to-robot synthesis pipeline is the largest filed instance of egocentric→robot data scaling and complements EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World (EgoVerse as a curated transfer-study substrate) and LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World (3DGS-rendered teleop-free training) on the synthetic-supervision axis for Synthetic Training Data. The paper’s OOD-benchmark argument also slots cleanly into Video Generation Benchmarks‘s thesis that in-domain metrics are the wrong measurement target, and RoboTwin-IF/RoboTwin-XE join RoboCasa365 — Large-Scale Simulation of Everyday Tasks for Generalist Robots and RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies as the next-generation evaluation substrate for RL Environment Platforms.