Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

TL;DR

Qwen-VLA is a unified vision-language-action generalist policy built on the Qwen3.5-4B multimodal backbone with a 1.15B DiT flow-matching action expert, trained jointly across robotic manipulation, vision-language navigation, trajectory prediction, and egocentric human demonstrations via embodiment-aware prompt conditioning. The recipe casts all four task families into a shared action-and-trajectory prediction space (with masked, zero-padded fixed-width channels per control mode) so a single model and a single output head cover everything from delta-EE poses to dexterous hands to VLN waypoints. A four-stage training pipeline — text-to-action DiT pretraining (T2A) on frozen-VLM language-only inputs, continued multimodal pretraining, multi-task SFT, then PPO RL in SimplerEnv — produces Qwen-VLA-Instruct, which reports 97.9% on LIBERO, 73.7% on Simpler-WidowX, 86.1/87.2% on RoboTwin-Easy/Hard, 69.0% OSR on R2R, 59.6% SR on RxR, 76.9% average OOD success in real-world ALOHA, and 26.6% zero-shot success on DOMINO dynamic manipulation. The bet: heterogeneous embodied decision-making collapses to one conditional prediction problem if the action interface is unified and the data mix is broad enough.

Key claims

• A single 4B-VLM + 1.15B DiT model with one shared action head can simultaneously support tabletop, mobile, bimanual, dexterous manipulation, and VLN, without per-platform output heads or per-task fine-tuning [§2.4, §2.2].
• Embodiment-aware prompt conditioning — a templated text prefix specifying robot tag, arm configuration, control frequency, chunk size, and instruction — is sufficient to specialize a single set of DiT parameters to heterogeneous control conventions [§2.3].
• Text-to-action DiT pretraining (T2A) with the VLM frozen and images withheld installs a language-indexed action prior before any visual grounding, framed as a structured decompression of compact language/embodiment tokens into hundreds of high-dimensional joint values [§3.1, Stage I].
• The four-stage T2A → CPT → SFT → RL recipe separates action-prior compression, visual grounding, task specialization, and success-driven refinement — the authors argue naive joint training from a randomly-initialized DiT plus a pretrained VLM destabilizes the backbone and wastes compute [§3.1].
• Pretraining mix is dominated by robot manipulation trajectories (74.2%) — including >10,000 hours of public real-robot data (RobotSet, AgiBot World, RDT-1B, DROID, BridgeData V2, RH20T, RT-1, BC-Z, RoboMIND, Galaxea, RoboCOIN) plus simulation (InternData-A1, GR00T-X-Embodiment-Sim) — supplemented by navigation (7.5%), egocentric (6.0%), and synthetic simulation (3.7%) [§3.2, Tab. 1].
• RL is performed only in SimplerEnv with sparse binary success rewards starting from the SFT checkpoint, yet the authors report the gains transfer to unseen environments and embodiments [§3.1, Stage IV].
• The flow-matching action loss uses two-level averaging — per-channel MSE then mean over active channels under a per-channel-per-step validity mask — so padded channels don’t bias gradients and each control dimension contributes equally regardless of how many channels an embodiment uses [§2.5, Eq. 1–2].
• Qwen-VLA-Instruct reports 97.9% on LIBERO, 73.7% on Simpler-WidowX, 86.1/87.2% on RoboTwin-Easy/Hard, 69.0% OSR on R2R, 59.6% SR on RxR, 76.9% average OOD success in real-world ALOHA, and 26.6% zero-shot success on DOMINO dynamic manipulation [Abstract].

Method

A pretrained Qwen3.5-4B VLM (hybrid gated-linear + grouped-query softmax attention, native ViT-token-stream multimodal fusion) provides perception and reasoning. A single-stream DiT-style flow-matching action expert (~1.15B params; 16 DiT blocks of 70.8M each) concatenates VLM hidden states with a noisy action chunk and runs joint self-attention with AdaLN time conditioning and multi-section RoPE aligned to the backbone. Action targets live in a fixed (H, C) tensor per sample with a per-channel-per-step validity mask; control modes (delta-EE position, Euler/quaternion rotation, absolute joint positions, gripper aperture, dexterous-hand joint angles, VLN per-waypoint Δx/Δy/Δθ) use the leading channels and zero-pad the rest. Each sample is prefixed with the embodiment-aware prompt; navigation samples get a VLN analog.

Training proceeds in four stages from the pretrained Qwen3.5-4B VLM. T2A: VLM frozen, no images, DiT trained only on language + embodiment prompt → action — installs a “language-indexed action prior.” CPT: unfreeze both modules and co-train on the heterogeneous data mix (manipulation + sim + ego + nav + spatial grounding + AD VQA + general VL). SFT: branched into multi-task SFT (VQA + grounding + manipulation + navigation, embodiment-balanced) and in-house teleop SFT for real-world deployment. RL: PPO with sparse binary success rewards in SimplerEnv only, on top of the multi-task SFT checkpoint, producing Qwen-VLA-Instruct.

Inference: action chunks are produced via a few Euler integration steps of the flow-matching velocity field from t=0 to t=1.

Results

• LIBERO: 97.9% [Abstract].
• Simpler-WidowX: 73.7% [Abstract].
• RoboTwin-Easy / Hard: 86.1% / 87.2% [Abstract].
• R2R OSR: 69.0% [Abstract].
• RxR SR: 59.6% [Abstract].
• Real-world ALOHA: 76.9% average OOD success [Abstract].
• DOMINO dynamic manipulation: 26.6% zero-shot success [Abstract].
• These figures are reported by a single unified generalist model, while several of the per-benchmark baselines compared against are specialist models fine-tuned per benchmark [Abstract, §1].

Why it’s interesting

Qwen-VLA is the strongest filed instance of the “single Qwen-VL family backbone + flow-matching action head” recipe that VLA Models currently identifies as the dominant open-VLA pattern (sibling to Spirit-v1.5: A Robotic Foundation Model by Spirit AI and Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models) — but it pushes that recipe further by also unifying manipulation with VLN and trajectory prediction in one model, and by paying for action pretraining at scale (74% of mix is real-robot trajectories) rather than substituting for it with VLM-side pointing/grounding. The result lets the wiki compare two clean positions on the VLA Models page: Embodied-R1.5’s “strong embodied VLM + small action head” against Qwen-VLA’s “strong VLM + large action expert + 10k+ hours of action pretraining,” with LIBERO numbers within ~0.6 points of each other. The four-stage T2A → CPT → SFT → RL recipe and embodiment-aware prompt conditioning are also a direct architectural answer to π*0.6: a VLA That Learns From Experience (RECAP)‘s RECAP — both papers are wrestling with the same problem (RL on top of flow-matching policies) but Qwen-VLA falls back to PPO with sparse rewards in a single sim env, where RECAP uses CFGRL-style advantage conditioning in the prefix to sidestep the no-log-likelihood problem entirely.

See also

• VLA Models — fourth filed instance of “Qwen-VL-family backbone + action head” as the dominant open-VLA recipe, and the largest filed VLA action expert (1.15B DiT) to date
• π*0.6: a VLA That Learns From Experience (RECAP) — sibling flow-matching VLA; contrast Qwen-VLA’s PPO-in-sim recipe with π*0.6’s RECAP advantage conditioning
• Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models — comparable LIBERO number (97.3 vs 97.9) but with a much smaller action head and no large-scale action pretraining
• Spirit-v1.5: A Robotic Foundation Model by Spirit AI — Qwen3-VL + DiT sibling, contrasting data-quality bet against Qwen-VLA’s data-scale bet
• NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry — counter-structure: NVIDIA’s open VLA pairs an action model (Isaac GR00T N1.6) with a separate world-model peer (Cosmos Reason 2), where Qwen-VLA fuses both into one model
• Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models — Reuss’s WAM taxonomy gives the design-space frame in which Qwen-VLA sits as a “VLA, no explicit world model” extreme
• Robots Need More than VLA and World Models — position paper directly counter to Qwen-VLA’s “scaling VLA across embodiments is the right axis”; argues four missing interfaces (data autolabelling, embodiment retargeting, physics world-model reasoning, reward inference) are the actual bottleneck
• Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level — sharper counter-position: VLA framing under-weights tactile / contact / sensorimotor dynamics
• Action100M: A Large-scale Video Action Dataset — overlapping bet on automated VLM-pipeline labeling at scale, but for video-action recognition rather than robot trajectories
• Dual-stream diffusion transformer — DiT flow-matching action expert with AdaLN time conditioning and joint self-attention concatenating VLM hidden states with noisy action chunks
• Synthetic Training Data — 3.7% of the pretraining mix is synthetic simulation trajectories (InternData-A1, GR00T-X-Embodiment-Sim) and Stage IV RL runs entirely in SimplerEnv
• Reasoning RL — Stage IV uses PPO with sparse binary success rewards on top of an SFT checkpoint, with the (debatable) claim that single-env RL gains transfer OOD
• Open foundation-model releases — official repo at github.com/QwenLM/Qwen-VLA; positions Alibaba’s robotics push alongside the rest of the Qwen3.5 family