Robotic Data Inoculation

TL;DR

Data Inoculation (DI) is a shared-control data-collection protocol in which a human teleoperates the robot while a residual RL policy — trained online on the growing dataset with the same reward objective later used to extract the downstream policy — perturbs the human’s commands within a bounded L_∞ ball. The perturbations drive the robot into states where a policy trained on the data so far would deviate from the demonstrator, and the human’s continuous corrections become recovery demonstrations against those specific failure modes. On three real-world precision tabletop tasks (T-Cube stacking, peg insertion, keycap pulling) collected on a Franka arm, DI-collected data trains downstream behavior-cloning and offline-RL policies that match or beat matched-time baselines, and the residual policy’s action magnitude decays as training progresses — eventually assisting the teleoperator rather than perturbing them.

Key claims

Coverage, not quantity, is the binding constraint on BC-trained real-world policies. Without coverage of recovery states, small learned-policy errors compound into OOD states the policy cannot recover from; naive heuristic teleop under-samples these states and DAgger-style interventions only trigger after the policy already fails [§1, §2].
A residual policy bounded by α in L_∞ is sufficient to steer data collection without taking control from the human. At each step the residual observes the robot and the live human command, then adds a bounded residual; the executed action deviates from the human command by at most α [§3].
Training the residual with the same reward objective used for downstream policy extraction is the load-bearing design choice. Because the residual’s perturbations aim to maximize the same reward, they steer data toward the deviations a policy trained on the data would make — turning the human’s corrections into supervision specifically against future-learner failure modes [§3, §4].
The residual’s action magnitude decreases as data collection progresses, and the human’s collection success rate rises to meet and then exceed the undisturbed baseline. On T-Cube, DI initially lowers collection success while the residual policy is untrained, then surpasses the no-perturbation baseline as the residual learns to steer toward productive deviations [§5, Fig. (T-Cube curves)].
DI improves downstream policy success under both behavioral cloning and offline RL across three precision real-world tasks, without the policy’s learner needing any modification [§5, Table 1].
Human corrections concentrate on the task phase that most matters. On T-Cube stacking, interventions concentrate on the place/release phase — the sub-task where final success is decided — rather than on uniform perturbation across the trajectory [Fig. (correction heatmap)].
DI does not depend on a simulator and does not freeze the residual. Prior residual-perturbation recipes rely on sim training or static noise models; DI trains the residual online as data accumulates, so the perturbation distribution tracks the current policy’s actual failure modes [§2, §4].

Method

A human teleoperates a robot through a standard closed-loop command interface (Franka arm, SpaceMouse per the LinkedIn commentary). At each control step, the residual policy π_ρ observes the current robot state and the human’s live command a_h, and emits a residual Δa bounded such that ||Δa||_∞ ≤ α (per the project page, α ≈ 3 mm translation and 0.01 rad rotation held for ~0.5 s so the operator can react). The executed action is a_h + Δa.

Asynchronously, π_ρ is trained online with RL on the growing dataset to maximize the same task reward that will later be used to extract the downstream policy. The reward-objective equivalence is the lever: π_ρ’s perturbations steer the robot toward states where a policy trained on the data so far would produce the same deviation — exactly the states where the human then corrects. Those corrections become recovery supervision concentrated on genuine failure modes rather than heuristic noise.

Downstream policy extraction runs unchanged: either plain behavior cloning or offline RL on the DI-collected dataset.

Results

Evaluated on three real-world precision tabletop tasks (T-Cube stacking, peg insertion, and keycap pulling) collected on a single Franka arm with one SpaceMouse and three operators per method, 20 trials per evaluated downstream policy.

Downstream BC and offline RL policies trained on DI data match or beat matched-time baselines across the three tasks [Table 1 per abstract].
On T-Cube specifically, DI’s data-collection success starts below the un-perturbed baseline (residual untrained) and rises above it as the residual trains, with residual action magnitude decreasing in parallel [Fig. (T-Cube curves)].
On keycap pulling (Task 3), the three methods land within noise — DI’s advantage is not uniform across all tasks [per LinkedIn / Pedro Alves summary].
Correction density concentrates on the release phase of T-Cube stacking — qualitative evidence the residual learns to perturb where the task is actually decided [Fig. (correction heatmap)].
Operator time cost: the human remains in the loop for every episode, which trades data efficiency against total human hours [acknowledged limitation, §6].

Why it’s interesting

Data Inoculation stakes out a new cell on the data-collection half of the VLA Models recipe-lever board, distinct from existing teleop-side levers: not clean-teleop scaling (Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models Spirit v1.5), not UMI-hour scaling (Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training XR-1), and not the three on-robot intervention-scaling threads (Enact launch — post-training infrastructure that generates targeted recovery data for robotics VLAs Enact, MicroFactory — 99.9% reliability via $5 human-in-the-loop DAgger retraining on Jetson (Ilir Aliu × Igor Kulakov podcast) MicroFactory, Empirical results fine-tuning π0.5 on a real manufacturing task Dream Machines) — all of which treat failure data as something to collect after a seed policy fails. DI generates the failure regime during initial collection via a residual policy trained with the downstream reward, so recovery demonstrations are captured on first contact rather than through iterated deploy-and-fix rounds. The reward-objective equivalence between residual training and downstream extraction is the sharpest version of the “coverage comes from steering collection toward future-learner failure modes” idea; sibling in spirit to OopsieData — a multi-lab dataset of real robot manipulation failures OopsieData (same first-author-adjacent lineage — Zhiyuan Zhou is an author on both) but a mechanism for producing failure data rather than a consortium for aggregating it. Contrasts with Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning Q-Planning on the same policy-extraction dependency: Q-Planning assumes off-robot failure trajectories exist and extracts value signal from them; DI generates them on-robot with a human in the loop.