Skip to content
Bud's Wiki

OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies

TL;DR

Section titled “TL;DR”

OpenRUA (Chu, Barr, Le Goues, O’Hearn, Harman, Sarro, Ye — UCL + CMU) is a zero-abstraction harness for turning an off-the-shelf coding agent into a robot visuomotor policy by exposing ROS 2 — the robot’s native software interface — as the only tool, with no bespoke abstraction layer, no agentic workflow orchestration, and no task-specific training. The agent sees only the ROS 2 documentation and a terminal; perception is reframed as file I/O over ROS topics and manipulation as writing ROS client code. Driven by Claude Code on Claude Opus 5, OpenRUA reports 99.0% on CaP-Bench and 87.0% on LIBERO-PRO. Analysis of the emergent behavior finds that under the minimalist design, the coding agent spontaneously writes raw-sensor processing programs in 96.80% of episodes, builds motion-control clients in 95.87%, and authors closed-loop control programs (adjust motion from sensor feedback) in 50.13%.

Key claims

Section titled “Key claims”
  • OpenRUA replaces the bespoke harnesses that prior coding-agent-as-robot-policy work builds with a zero-abstraction interface: off-the-shelf coding agents get only terminal access to ROS 2’s native CLI and client library, plus ROS 2 documentation, and are left to organize their own work [Abstract].
  • Under this design, perception is recast as file I/O (reading ROS topics) and manipulation as coding (writing ROS clients), so the agent’s two capabilities — sensory processing and motion control — collapse into one capability: writing programs against a documented API [Abstract].
  • With Claude Code powered by Claude Opus 5, OpenRUA reaches 99.0% success on CaP-Bench and 87.0% on LIBERO-PRO without bespoke primitives or task-specific training [Abstract].
  • The minimalist design surfaces emergent behavior in the coding agent: it spontaneously writes programs that process raw sensory inputs and derive metric measurements in 96.80% of episodes [Abstract].
  • The agent spontaneously builds motion-control clients (e.g. gripper control) in 95.87% of episodes [Abstract].
  • The agent spontaneously builds closed-loop control programs (e.g. adjusting motion based on sensor feedback) in 50.13% of episodes [Abstract].
  • The paper’s thesis is that the “harness engineering” layer that CaP-X, Waddle, GRID, Harness VLA, RoboClaw, RoboHarness, and RobotUse all layer on top of frozen agents may be unnecessary once the native robot-software interface is itself a well-typed tool surface the agent already knows [Abstract framing].

Method

Section titled “Method”

OpenRUA eliminates the orchestrator / primitive-library / typed-interface layers typical of the “coding agent drives robot” lineage. Instead, it provides the coding agent with only three things:

  1. Terminal access to the robot (sim or real), with ROS 2 as the sole runtime interface.
  2. ROS 2 documentation as the reference the agent reads to discover what it can do.
  3. Basic tools (shell, file ops) — nothing custom-authored for robotics.

There is no agentic workflow (no plan/act/verify loop baked in), no skill library seeded by the harness, no VLA under the hood, no custom perception primitives, and no action-space wrapper. The agent organizes its own work from the ROS 2 CLI + rclpy client library onward.

Under this contract, two reframings fall out of the ROS 2 primitives themselves. Perception is file I/O: sensor streams (camera topics, joint-state topics, force/torque topics) are read via standard ROS subscriptions or CLI dumps — operations the coding agent already does fluently in any software project. Manipulation is coding: motion commands become rclpy publisher programs against motion-control topics / services, which is just Python the agent writes.

The paper driver is Claude Code with Claude Opus 5. Benchmarks are CaP-Bench (from CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation) and LIBERO-PRO, both already instrumented for frontier coding-agent evaluation elsewhere on the recipe-lever board.

The arxiv landing page provides the abstract only; the companion code (github.com/terminalworld/OpenRUA, Apache-2.0) is linked by the authors but was not retrieved at ingest, and per-task numbers / baseline tables / ablation axes are not surfaced in the retrieved text.

Results

Section titled “Results”

Headline numbers [Abstract]:

  • CaP-Bench: 99.0% success with Claude Code + Claude Opus 5 driving OpenRUA.
  • LIBERO-PRO: 87.0% success on the same configuration.
  • Emergent-behavior rates: 96.80% raw-sensor processing programs, 95.87% motion-control clients, 50.13% closed-loop control programs.

For context on the recipe-lever board on VLA Models and Tool-Use Agents: CaP-X’s own CaP-Agent0 (frontier base + multi-turn + VDM + auto-synthesized skill library + parallel reasoning) matches-or-exceeds-human-expert on 4 of 7 CaP-Bench core tasks (CaP-X Fig. 8); OpenRUA at 99.0% CaP-Bench under a minimalist harness is the sharpest claim on this benchmark to date and is the first filed instance where the headline number comes from removing harness scaffolding rather than adding more. On LIBERO-PRO, prior filed VLA-side results sit roughly: OpenVLA / π0 at 0% under Pos+Task perturbations across libero-{object,goal,spatial}, π0.5 at 0.17–0.38 Pos and ≤0.01 Task, and CaP-Agent0 at 0.22 / 0.26 / 0.12 Pos and 0.18 / 0.17 / 0.14 Task (CaP-X Table 2). OpenRUA’s reported 87.0% is a step jump against those references, though the exact perturbation protocol used in the abstract isn’t resolvable at filing time.

Why it’s interesting

Section titled “Why it’s interesting”

OpenRUA stakes out the sharpest minimalist position on the LLM-as-controller row of the VLA Models recipe-lever board and the embodied-tool-use design space tracked by Tool-Use Agents. The row has been accumulating distinct interface bets:

OpenRUA rejects all of these interface bets in favor of ROS 2 itself. The implicit thesis — that the coding agent’s native software-engineering competence over a well-documented API is what has been doing the work in every “frontier LLM drives a robot” demo filed this year (Jay Chooi (Robocurve) — Astra vs Fable 5.1 on real-robot manipulation via IK, Comparing Fable 5.1 to Fable 5 on robotic manipulation, thijs — Astra paints the Golden Gate Bridge with a real robot arm, Wenli Xiao — GPT-6 Astra does physical in-context learning from a single human video via a motion-planning tool harness, LeRobot demos Claude Code operating SO-ARM101 zero-shot via Anthropic MHS — self-calibration to 4.1mm accuracy, Robot-Use Agents) — gets a concrete falsification target: 99.0% CaP-Bench. The paper’s data-point on where the gains come from is also the sharpest in the lineage: 96.80% / 95.87% / 50.13% emergent write-rates for raw-sensor processing, motion-control clients, and closed-loop control under conditions where none of these were handed to the agent as primitives. If this reproduces, it directly connects the empirical “LLM-as-controller works” thread to the Agentic Software Engineering substrate on this wiki — the capability transferring is agentic software-engineering competence over a documented API, not a robotics-specific skill. It also runs in productive tension with the Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level sensorimotor counter-position, which predicts contact-establishment failures the LIBERO-PRO and CaP-Bench tasks sample unevenly — the obvious next test is whether a 99/87 result holds up under the contact-rich / force-mediated regime Getting a Grip on Robotic Data Collection — Co-Designed Koala Gripper for Force-Mediated Manipulation and FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks identify as the structural blocker. Finally, the sibling filed in the same message — RobotUse: Allocating Computation, Context, and Decisions RobotUse — stakes the opposite position (more harness structure: Main/Subagent/Backend + persistent playbook) at exactly the same moment, making the two papers a natural head-to-head on whether harness scaffolding helps or hurts under the latest coding-agent generation.

See also

Section titled “See also”
  • RobotUse: Allocating Computation, Context, and Decisions — direct sibling filed in the same Slack message; opposite bet (three-role factorization + persistent playbook + visual target/pose selection vs OpenRUA’s zero-abstraction ROS 2 terminal). Natural head-to-head.
  • CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation — CaP-X owns the CaP-Bench benchmark OpenRUA tops at 99.0%; CaP-X’s Takeaway 2 (performance rises with primitive abstraction) is directly challenged by OpenRUA’s “zero abstraction + 99%” result.
  • Agent as Policy for Robotic Manipulation — AGP uses a typed 7-command EE-pose/joint/gripper interface + saved-procedure experience accumulation; OpenRUA drops both in favor of the native ROS 2 interface.
  • Show-Harness: Just a VLM Agent Can Play Robots — Show-Harness’s semantic-action-unit vocabulary with embodiment-specific interpreters is the opposite pole on the interface-design axis.
  • Guava: An Effective and Universal Harness for Embodied Manipulation — Guava argues iterative ReAct loops + semantic abstractions + multimodal observations are jointly load-bearing; OpenRUA’s result tests whether the semantic-abstraction leg is actually necessary when ROS 2 is the API.
  • Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents — Harness VLA wraps a frozen VLA inside an LLM orchestrator with learned operating ranges; OpenRUA removes the VLA entirely and reports comparable or better LIBERO-PRO numbers.
  • Introducing Waddle: Agents that Control Robots — Waddle productizes the three-level program (primitives → parametrized skills → task programs); OpenRUA is the “no library at all, write ROS” counterpoint.
  • Robot-Use Agents — Isola’s positional “robot-use agents” essay; OpenRUA is the sharpest quantified empirical realization of the thesis to date.
  • Previewing the Model Hardware Standard — MHS exposes devices as read/write primitives with natural-language tags, one abstraction layer above a bare driver; OpenRUA goes one further and uses the existing (ROS 2) interface directly, no new standard required.
  • Tool-Use Agents — the embodied-tool-use design surface this paper lands on; OpenRUA is the “tool surface = native robot API, no wrapping” extreme.
  • VLA Models — the LLM-as-controller row of the recipe-lever board; OpenRUA’s “no VLA” + 99/87 result is a load-bearing new datapoint.
  • RL Environment Platforms — the CaP-Bench and LIBERO-PRO numbers sit on packaged env substrates; OpenRUA is the second filed harness-level recipe (after RobotUse) that cites them as its headline eval.
  • Agentic Software Engineering — the capability this paper argues is actually transferring to robotics: coding agents’ native software-engineering competence over a documented API, applied to ROS 2.