Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models

TL;DR

Embodied-R1.5 is an 8B open-weights “Embodied Foundation Model” (EFM) from a Tianjin University / Noah’s Ark Lab consortium that unifies embodied cognition, task planning, correction, and pixel-precise pointing in a single Qwen3-VL-8B-derived VLM, trained with a multi-task-balanced RL recipe over a 15B-token data system. The model reports SOTA on 16/24 embodied VLM benchmarks (claimed to surpass Gemini-Robotics-ER-1.5 and GPT-5.4), and — without any action pretraining — its VLA fine-tune (Embodied-R1.5-VLA) outperforms π₀.₅ across SimplerEnv (Google Robot + Bridge) and LIBERO suites. The release is full-stack: model weights, datasets, training code, and EmbodiedEvalKit (a 20+ benchmark unified eval harness).

Key claims

Embodied cognition, task planning + correction, and pointing/location can co-exist and reinforce each other in a single 8B parameter VLM, eliminating the fragmented multi-specialist pipeline that has typified prior embodied stacks [§Unified EFM].
Three automated data-construction pipelines produce a 15B-token corpus paired with a multi-task balanced RL recipe whose output-type-specific rewards resolve the interference inherent in heterogeneous multi-task training [§Data System & RL].
A Planner–Grounder–Corrector (PGC) closed-loop framework lets the same model do high-level decomposition, precise spatial grounding, and self-correction over long-horizon tasks — without handoff to a separate executor model [§PGC Framework].
Headline benchmark numbers: SOTA on 16/24 embodied VLM benchmarks, beating Gemini-Robotics-ER-1.5 and GPT-5.4 on aggregate [§SOTA Results, Per-benchmark scores tables].
Embodied-R1.5-VLA is built by attaching an action head to the VLM (no large-scale action pretraining); on SimplerEnv Google Robot it reports 92.4 overall (vs π0 71.4, GR00T N1.5 35.2), on SimplerEnv Bridge 74.0 (vs GR00T N1.5 62.0, π0 27.1), and on LIBERO overall 97.3 (matching/edging OpenVLA-OFT and π0.5) [§VLA tables].
The release ships EmbodiedEvalKit — a unified evaluation framework covering 20+ embodied benchmarks (Embodied QA, Spatial Reasoning, Pointing, Affordance/Location, Planning) with 20+ models supported across vLLM/HF/API backends, with all benchmarks normalized to HuggingFace Parquet [§EmbodiedEvalKit].

Method

The architecture is a single 8B-parameter VLM (Qwen3-VL-8B backbone) trained to emit four output types: free-form embodied reasoning text, structured plans, correction signals, and pointing/location coordinates. The training data is built by three automated pipelines that produce >15B tokens spanning embodied cognition, planning/correction supervision, and pointing/grounding traces. RL training is multi-task with output-type-specific reward functions designed to keep gradients from heterogeneous tasks from interfering — the project page calls this the central recipe contribution alongside the data system.

For embodied control, the same VLM serves as Planner, Grounder, and Corrector inside a closed-loop PGC pipeline: it decomposes a high-level instruction, grounds the resulting subgoals to pixel coordinates, executes (via an action head when running as the VLA), monitors progress, and self-corrects on failure. The VLA variant (Embodied-R1.5-VLA) attaches a small action head to the VLM and is fine-tuned without large-scale action pretraining — the design bet is that broad VLM-level embodied capability transfers to manipulation. Real-world deployment uses dual-arm setups across five tasks (tool affordance and multi-step reasoning).

Results

SimplerEnv Google Robot (variant aggregates): Embodied-R1.5-VLA 92.4 overall, vs π0 71.4, π0-FAST 46.4, SpatialVLA 55.3, OpenVLA 24.5, GR00T N1.5 35.2. Most striking gap: Open Top Drawer & Place Apple, 97.2 vs π0 46.6 (next-best) [§Google Robot table].
SimplerEnv Bridge: 74.0 overall vs GR00T N1.5 62.0, SpatialVLA 42.7, π0-FAST 32.1, π0 27.1 — Put Eggplant in Basket and Put Carrot on Plate are both ≥75 [§Bridge table].
LIBERO overall: 97.3 — matches/edges OpenVLA-OFT-with-action-pretraining (97.1), GR00T N1.6 (97.0), π0.5 (96.9) without action pretraining, and beats every other no-action-pretraining baseline by a wide margin (next-best π0.5-no-pretrain 93.6) [§LIBERO table].
16/24 SOTA on the embodied VLM benchmark sweep, with mean performance “markedly exceeding” Gemini-Robotics-ER-1.5 and GPT-5.4 [§Per-benchmark scores].
Robustness: evaluated under 7 perturbation types (camera, robot, language, light, background, noise, layout) — claimed comprehensive stability advantage over native backbones [§robustness section].
Manipulation grounding (Where2Act-style affordance benchmark): Embodied-R1.5 averages 0.66 on unseen categories vs ManipLLM 0.54, Implicit3D 0.41 — large gains on USB, Faucet, Washer; mixed on Toaster/Phone [§seen/unseen tables].

Why it’s interesting

This is the first filed paper that puts a unified pointing-capable embodied VLM and a competitive VLA on the same model, in the open-weights tier — a direct counter-datapoint to π0.5 / Spirit-v1.5 / Gemini-Robotics-ER-1.5 that did require separate stacks or large action pretraining. It strengthens VLA Models from a third angle: where Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models argued clean teleop data was the lever and π*0.6: a VLA That Learns From Experience (RECAP) argued on-policy + intervention RL (RECAP) was the lever, Embodied-R1.5 argues that a VLM with strong pointing/grounding is the lever and action pretraining is optional. The pointing-as-first-class-output design also extends the line of work on MolmoPoint: Better pointing architecture for vision-language models (MolmoPoint) into the action loop — and provides an additional rebuttal target for BOP-Ask: Object-Interaction Reasoning for Vision-Language Models (BOP-Ask), which has called out pixel-precise grounding as the actual VLM bottleneck for embodied tasks.