Gemini Robotics 1.5 brings AI agents into the physical world

TL;DR

Google DeepMind releases two paired models for physical-agent control: Gemini Robotics 1.5, a vision–language–action (VLA) model that turns visual input plus instructions into motor commands while emitting an internal reasoning trace, and Gemini Robotics-ER 1.5, a vision–language model that plans, reasons spatially, and natively calls digital tools (e.g. Google Search) plus user-defined functions. The two are wired into an agentic framework where ER 1.5 acts as the high-level planner and Robotics 1.5 executes per-step natural-language sub-instructions, with both models fine-tuned from the core Gemini family for their respective roles. Gemini Robotics-ER 1.5 is exposed today via the Gemini API in Google AI Studio; the action model is partner-gated. The headline capability claims are cross-embodiment skill transfer (ALOHA 2 ↔ Apptronik Apollo humanoid ↔ bi-arm Franka) and top scores on 15 embodied-reasoning benchmarks including ERQA and Point-Bench.

Key claims

Gemini Robotics 1.5 emits an internal reasoning trace before producing motor commands, which the post frames as enabling longer-task generalization, robustness to environment changes, and human-readable explanations of robot decisions [§ “Thinks before acting”].
Gemini Robotics-ER 1.5 is described as “the first thinking model optimized for embodied reasoning” and reports state-of-the-art performance on 15 academic benchmarks including ERQA and Point-Bench (covering pointing, image QA, video QA), with detailed numbers in the linked tech report [§ “Understands its environment”].
ER 1.5 natively calls digital tools — Google Search and arbitrary user-defined functions — inside its planning loop, instantiated in a worked example of context-grounded recycling-sorting that requires fetching local recycling rules before acting [§ “Gemini Robotics 1.5: Unlocking agentic experiences for physical tasks”].
Cross-embodiment transfer: tasks demonstrated on the ALOHA 2 bimanual platform during training “also just work on” Apptronik Apollo (humanoid) and a bi-arm Franka, and vice versa, without per-embodiment specialization of the model [§ “Learns across embodiments”].
Safety is described as layered: high-level semantic reasoning (“thinking about safety before acting”) aligned to Gemini Safety Policies, sitting above low-level on-board sub-systems (e.g. collision avoidance) [§ “Responsibly advancing AI and Robotics”].
ER 1.5 reports state-of-the-art performance on an upgraded ASIMOV semantic-safety benchmark, with the thinking mode itself credited as a contributor to better semantic-safety understanding and physical-constraint adherence [§ “Responsibly advancing AI and Robotics”].
Release posture: ER 1.5 is shipped to developers via the Gemini API in Google AI Studio; Robotics 1.5 (the VLA) is restricted to “select partners” [intro, § release].

Method

The system is a two-model agentic stack, both derived by fine-tuning the core Gemini family on different datasets to specialize behavior. ER 1.5 is the high-level brain: it ingests the task and observations, plans multi-step routines, calls tools (Search or user-defined functions) when external information is needed, estimates its own progress, and emits a natural-language sub-instruction at each step. Robotics 1.5 takes that sub-instruction plus current sensory input and produces motor commands, while also generating an internal reasoning trace that decomposes long-horizon goals into shorter executable segments (the post walks through a laundry-sorting example: pick a level for “sort by colour” → “white sweater → white bin” → motion-level “approach the sweater this way to pick it up cleanly”). Cross-embodiment generalization is asserted rather than ablated in the blog post itself; the tech report linked from the post is the canonical source for benchmark numbers and architecture details. No parameter counts, training data sizes, or architectural details are given in the blog post.

Results

The blog post is light on quantitative results. The substantive claims are: (1) ER 1.5 is SOTA on 15 embodied-reasoning benchmarks — ERQA and Point-Bench named explicitly, with the rest deferred to the tech report; (2) ER 1.5 is SOTA on the upgraded ASIMOV semantic-safety benchmark, with thinking-mode contribution attributed but not isolated in a number; (3) cross-embodiment transfer between ALOHA 2, Apollo, and bi-arm Franka is shown qualitatively (videos in the post) rather than as a percentage on a held-out task suite. Per-benchmark numbers, ablations, and per-embodiment success rates are in the tech report at storage.googleapis.com/deepmind-media/gemini-robotics/Gemini-Robotics-1-5-Tech-Report.pdf — not reproduced here at filing time.

Why it’s interesting

This is the closed-flagship counterpart to SIMA 2: A Generalist Embodied Agent for Virtual Worlds from the same lab: SIMA 2 puts Gemini Flash-Lite inside generated virtual worlds (Genie 3) as the embodiment, while Gemini Robotics 1.5 puts Gemini-derived models inside real hardware (ALOHA 2, Apollo, Franka). Both papers commit to the same architectural pattern — a Gemini-family base finetuned into separate planner and action heads — but split on the embodiment substrate, making them a natural pair for tracking how the same recipe scales across the simulation↔reality boundary. The native-tool-calling claim (“ER 1.5 calls Google Search inside its plan”) extends the Tool-Use Agents pattern out of pure text-agent settings into embodied control — a regime the current concept page does not yet cover, where the agent’s tools include both digital APIs and a downstream VLA. The cross-embodiment transfer claim contrasts with Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations, where the recipe sidesteps the embodiment problem by reading 6D pose out of generated pixels; Gemini Robotics 1.5 is the policy-trained side of the same question, claiming the policy itself transfers across embodiments without per-robot specialization. Finally, the “thinking before acting” framing makes this a clean datapoint for the Thinking with Modalities candidate cluster, generalized from “model thinks with video/images during reasoning” to “model thinks with proprioceptive + visual + linguistic state during action selection.”