Playful Agentic Robot Learning

TL;DR

RATs (Robotics Agent Teams) introduces Playful Agentic Robot Learning — a pre-task “play” stage where an embodied Code-as-Policy agent self-directs exploration to grow a persistent code skill library before downstream tasks arrive. A task-proposer agent picks rarely-tried-but-learnable object-skill combinations, a planner retrieves and composes relevant skills, an execution agent writes robot-control programs, goal and per-step verifiers diagnose failures, and successful routines are distilled into named, callable code skills. On held-out downstream tasks, the play-learned skill library gives +20.6 pp on LIBERO-PRO and +17.0 pp on MolmoSpaces over CaP-Agent0 with no model finetuning, and plugs into other inference-time Code-as-Policy agents for +8.9 / +8.8 pp on RoboSuite / real-world transfer [§Abstract].

Key claims

Playful Agentic Robot Learning is defined as a paradigm where an embodied coding agent uses self-directed play as a continual skill-learning stage before downstream tasks arrive — distinct from prior agentic robot systems that write code, observe feedback, and revise across attempts but remain task-driven [§Abstract].
The task proposer favors rarely-tried object-skill combinations that remain learnable — neither trivial nor impossible — explicitly framing exploration as moving along a “competence frontier” rather than random sampling [Project page §01].
During play the agent runs a loop of: propose novel-yet-learnable task → plan + retrieve relevant skills → write & execute robot-code policy → verify intermediate progress with goal and per-step verifiers → diagnose failures → retry with dense, step-level feedback → distill successful executions into a persistent code skill library [§Abstract, Project page §Method].
A failing-step diagnoser routes concrete feedback either back into retries or into isolated SubAgent practice — separating local-bottleneck refinement from outer-loop skill acquisition [Project page §03].
At test time the agent reuses relevant skills from this frozen skill library to help solve new tasks — no model finetuning is required to deploy play-learned skills on downstream evaluations [§Abstract].
On LIBERO-PRO, play-learned skills give a +20.6 percentage-point gain over CaP-Agent0; on MolmoSpaces, +17.0 pp [§Abstract].
The learned skills transfer across inference-time Code-as-Policy agents by simple retrieval into context: +8.9 pp on RoboSuite and +8.8 pp on real-world transfer without finetuning the underlying model [§Abstract].
Verified routines become persistent tools that survive beyond a single episode and become ingredients for later tasks, with the proposer empirically exploring object-skill combinations across iterations rather than repeatedly sampling the same easy behavior [Project page §Analysis].

Method

RATs is a multi-agent Code-as-Policy system composed around a persistent code skill library. The Task Proposer scores candidate object-skill combinations by novelty (under-explored in prior play) and learnability (neither trivially solvable by existing skills nor far above the current frontier), grounded in the current scene. The Planner decomposes the proposed task into ordered steps and retrieves only the skills relevant to those steps from the library, so the Policy Writer generates an executable robot-control program from a focused tool prompt rather than the full library. Goal verifiers check final success, while per-step verifiers localize where progress stalls and give the diagnoser concrete evidence for targeted retries; when a single step persistently fails, an isolated SubAgent practices that step in isolation. Successful end-to-end executions are extracted as named code skills and added back into the library; failures become compact lessons that shape future proposals.

The play stage runs autonomously over LIBERO-PRO and MolmoSpaces. Downstream evaluation freezes the skill library and the underlying coding-agent model, retrieves the relevant subset of skills into the context, and runs the standard Code-as-Policy loop on held-out tasks (CaP-Agent0 baseline) plus transfer to RoboSuite tasks and real-world manipulation (no model finetuning at any stage).

Results

LIBERO-PRO held-out tasks: +20.6 pp over CaP-Agent0 [§Abstract].
MolmoSpaces held-out tasks: +17.0 pp over CaP-Agent0 [§Abstract].
Cross-agent transfer (skill library plugged into other inference-time Code-as-Policy agents): +8.9 pp on RoboSuite, +8.8 pp on real-world transfer; no fine-tuning of the underlying model [§Abstract].
Qualitative analysis: opening / closing / picking / placement tasks retrieve distinct mixtures of learned perception, geometry, grasping, and motion helpers — evidence the library is composing into reusable primitives rather than collapsing to one mode [Project page §Analysis].
Real-world transfer demonstrated on plate wiping, localization + transport, and grasping + pulling routines using the same frozen library [Project page §Real-world transfer].

Why it’s interesting

RATs is the cleanest filed instance of “let the agent design its own training distribution” applied to embodied Code-as-Policy, and it sits directly downstream of CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation — same Berkeley / Goldberg lab, same LIBERO-PRO benchmark, same Code-as-Policy + persistent-skill-library architecture, with the same Letian Fu / Justin Yu / Ken Goldberg authors. Where CaP-Agent0’s skill library was bootstrapped by extracting recurring functions from successful S3 downstream rollouts, RATs flips the order: the skills are discovered during a dedicated play stage that the agent designs for itself before the downstream task distribution is even visible. This makes it the embodied analog of the closed data→model→data loops that Self-Play Data Generation tracks for text agents — InfTool’s role-playing trajectory synthesis (Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing) and MiroThinker’s multi-paradigm self-synthesized rollouts (MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling) — with the verification gate now being a physical / simulated robot rollout rather than a string matcher or a unit test. It also complements ENPIRE: Agentic Robot Policy Self-Improvement in the Real World on a different axis: ENPIRE’s coding agents autonomously improve a fixed downstream policy through a real-robot RL loop, while RATs autonomously acquires task-agnostic skills through a play loop and shows they transfer without finetuning. The +8.8 pp cross-agent real-world transfer datapoint is unusually strong evidence that the library — not the agent — carries the value.