RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks

TL;DR

RoboClaw (AgiBot + NUS + SJTU) unifies data collection, policy learning, and long-horizon task execution under a single VLM-driven agent that reasons over structured memory and invokes modular VLA policies through an MCP tool interface. The signature idea is Entangled Action Pairs (EAP): for every forward manipulation policy, a paired inverse recovery policy forms a self-resetting loop so the robot alternates forward/reset and collects on-policy data with minimal human intervention. Deployed on an Agibot G01 dual-arm mobile platform, RoboClaw reports a 25% success-rate improvement over baselines on long-horizon tasks and a 53.7% reduction in human time investment, while iterative EAP rollouts lift per-skill success rates dramatically (e.g., lipstick insertion 2/50 → 23/50 across five iterations).

Key claims

Unifying data collection and task execution under one VLM meta-controller with shared structured memory (role identity, task-level memory, working memory) keeps contextual semantics consistent across the lifecycle and reduces train-deploy state-distribution mismatch [§1, §3.1].
Entangled Action Pairs (EAP) — pairing every forward policy π_f with a simpler inverse reset policy π_r trained to restore the precondition region — enables continuous on-policy data collection without manual environment resets [§3.2, Eq. 6-8].
The paper deliberately makes inverse policies simpler than forward policies so reset reliability underwrites the data-collection loop; measured inverse success rates are 36/50 to 43/50 across the four tabletop tasks [§4.2, Table 2].
Iterative EAP rollout monotonically improves forward-policy success rates on all four evaluated skills: Body Lotion 21→43/50, Primer 23→40/50, Lipstick Insertion 2→23/50, Tissue Wipe 11→26/50 over five 50-sample iterations on top of a fixed human-demo seed [§4.2, Table 3].
Deployment-time skill scheduling is done by the same agent that collected the data: it monitors subtask status via Env Summary / Fetch Robot Stats, calls Change Policy on failures, and escalates to Call Human for safety-critical or unrecoverable states [§3.3].
Compared to a manual-teleop baseline collecting the same number of trajectories, RoboClaw requires substantially less human time (baseline ≈ several× more human effort) and substantially fewer human interventions during model rollouts [§4.1, Fig. 4a-b].
On long-horizon vanity-table organization, RoboClaw reports +25% absolute success over both a same-data model without the agent framework and a product-of-subtask-success-rates baseline [§4.3].

Method

RoboClaw is organized as a three-level hierarchy — Policies (VLA models producing low-level actions via flow-matching), Tools (MCP-exposed system interfaces: Start/Terminate/Change Policy, Env Summary, Fetch Robot Stats, Call Human), and Skills (reusable procedures that orchestrate tools). At every timestep the agent holds structured memory M_t = (role identity, task-level memory, working memory), and a VLM performs chain-of-thought reasoning over observations + memory to pick the next tool call. The low-level policies are flow-matching VLAs conditioned on visual observation, agent-generated language instruction, and proprioceptive state, predicting an H-step action chunk trained with a standard conditional flow-matching objective.

Data collection: for each manipulation policy the authors train a forward π_f and a reset π_r, and the agent alternates them to form self-resetting entangled trajectories τ_f, τ_r stored in D. Deployment: given a long-horizon instruction, the agent selects subtasks from a candidate set, invokes the corresponding π_f via MCP, periodically queries environment summaries to check subtask completion, retries or switches policies on failure, and escalates to human intervention when autonomous recovery fails. Trajectories generated at deployment are folded back into D under the same contextual semantics, closing the lifecycle loop.

Results

Evaluated on the Agibot G01 dual-arm mobile platform (20 DoF + AGIBOT OmniPicker grippers) across four real-world scenarios (bedroom vanity table, kitchen shelf, study desk, convenience-store shelf) and four single-skill tasks (Body Lotion placement, Primer placement, Lipstick insertion, Tissue wipe). Headline numbers: 25% absolute success-rate improvement over baselines on long-horizon tasks and 53.7% reduction in human time investment relative to manual data collection [§Abstract, §4.3]. Iterative EAP rollouts (1→5 iterations of +50 samples each) drive per-skill success rates from 21/50→43/50 (Body Lotion), 23/50→40/50 (Primer), 2/50→23/50 (Lipstick insertion), 11/50→26/50 (Tissue wipe) [Table 3], with inverse reset policies holding 36-43/50 to sustain the loop [Table 2]. Training uses bfloat16, batch 16, 10k steps, LoRA rank 16, warmup 100, 3 inference steps.

Why it’s interesting

RoboClaw stakes out a distinct lever on the VLA recipe board tracked in VLA Models: rather than a new action-pretraining recipe, tokenizer, backbone, or world-model interface, it argues the lifecycle boundary is the object of design — the same agent that collects the data also schedules the skills, so state distributions match and semantics stay consistent. This is a natural extension of the VLA-as-interruptible-tool framing that VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation introduced (VoLoAgent wraps a VLA as a tool under a VLM orchestrator), pushed one step upstream into data collection via EAP. It also complements ENPIRE: Agentic Robot Policy Self-Improvement in the Real World and Evo-RL: Open Real-World Offline RL on SO-101 and AgileX PiPER on the “agentic self-improvement of manipulation policies” axis — ENPIRE uses frontier coding agents to write policy improvements, Evo-RL uses RECAP-style RL, RoboClaw uses a VLM agent to schedule and reset around fixed VLA primitives — three distinct answers to “how does a VLA stack keep improving after deployment.”