Learning from Fleet Experience: Collective Experience Learning across 96 Robots

TL;DR

Collective Experience Learning (CEL) is a per-task fleet self-improvement paradigm: after SFT + (M)HG-DAgger initialization, a fleet of 96 AgiBot G2 robots runs autonomous rollouts of the same policy; humans supply only binary success/failure labels and environment resets. The method that makes this work is Failure-guided Experience Retrieval Fine-Tuning (FERT) — successful rollouts enter a shared experience buffer, failures act as queries that upweight visually- and proprioceptively-similar successful segments during online behavior cloning. Across five real-world bimanual manipulation tasks (phone packaging, phone charging, stool assembly, flower arrangement, clothes folding), FERT lifts mean success 81.0% → 97.3% without additional expert demonstrations, and on phone packaging specifically improves from 79.2% → 98.3% success (92% failure reduction) when scaling the training fleet from 16 → 96 robots.

Key claims

• Robot fleets can self-improve a shared policy from autonomous rollouts gated only by binary task-outcome labels, with no additional expert demonstrations or corrective-action interventions beyond the SFT + (M)HG-DAgger initialization [§01].
• Failure trajectories, which lack valid supervision signal under behavior cloning, can be reused as queries rather than as training targets — visually and proprioceptively similar successful segments are retrieved from the shared experience buffer and sampled more frequently during online BC on successes only [§02].
• FERT lifts mean success across 5 real-world manipulation tasks from (M)HG-DAgger’s 81.0% to 97.3% (16.3 pp gain, 86.0% mean failure reduction), outperforming all baselines and human operators on every task [§04].
• Phone charging reaches 97.5% success (from 60.0% initial), reducing failures by 93.8%; phone packaging reaches 98.3% at 96 training robots (from 79.2% initial), reducing failures by 92.0% [§04].
• Scaling the training fleet at fixed per-robot collection time yields monotone gains: phone packaging reaches 50.8% / 86.7% / 98.3% at 4 / 16 / 96 robots respectively — the 4-robot run actually underperforms the 79.2% starting policy, while 16 → 96 robots adds 11.7 pp [§04].
• Retrieval-plus-context is load-bearing: removing the 30-frame bilateral context around each retrieved segment while keeping retrieval drops phone-packaging success from 86.7% → 57.5% at 16 training robots — both relevance and temporal continuity matter for sequence learning [§04 ablation].
• Hyperparameters that worked best in the ablation: synchronize policy weights every 500 optimizer steps and retain 88% of previous sampling priorities each sync — bounds the staleness vs. stability trade-off in asynchronous training [§04 ablation].
• Fault-tolerant Asynchronous Training System (FATS) keeps collection running through local interruptions: a 96-robot run collected 9,834 episodes despite 30 fault incidents affecting 22 robots, with faults costing only 1.3% of total robot-time [§05].
• The HotSwarm hot-plugging layer gives each restarted worker a new identity so late in-flight actions from the predecessor are rejected; incomplete episodes are discarded, so training data stays consistent across robot/inference/training failures [§05].
• SwarmLink pools observations across cloud inference workers and streams execution records to a trajectory collector that releases episodes to training only when all fields and the outcome are present — interaction traffic takes priority over bulk transfers so collection and learning overlap on one network link [§05].
• Deployment-time guarantee: policies improved via FERT across the five tasks achieved 93.3–99.2% success when evaluated on two AgiBot G2 robots excluded from training data collection (60 trials × 2 robots = 120 trials per task) [§01, §06].

Method

The pipeline has three stages, with sources of actions changing at each stage: SFT uses human demonstrations, (M)HG-DAgger uses expert interventions during robot execution, and the FERT stage uses the fleet’s own successful autonomous rollouts.

In the FERT stage, every rollout has a binary success/failure label (human-provided). Successful trajectories enter a shared experience buffer. For each failure, the observation window preceding the failure point serves as a query against the buffer: visual features plus joint-state time series are matched against successful segments, and the k closest matches (expanded with ±30 frames of surrounding context) are sampled with higher probability during the next training phase. Training itself is plain online BC on the (reweighted) successful segments — no RL, no reward model, no policy-gradient term. Policy weights are periodically synchronized back to the fleet (every ~500 optimizer steps), and the sampling-priority buffer retains 88% of the previous priorities after each sync.

The infrastructure layer (FATS) decouples cloud inference workers, the trajectory collector, and training workers so that the fleet can keep producing episodes through local faults: HotSwarm restarts individual control/inference workers with fresh identities (discarding partial episodes, rejecting late actions addressed to a predecessor), and SwarmLink prioritizes action-request traffic over bulk observation transfers while pooling observations from available inference workers.

Results

• Headline parity with human teleoperators and all baselines across all 5 tasks at the fleet scales tested: FERT 97.3% mean success vs (M)HG-DAgger 81.0% starting point (sim-and-real averaged across flower arrangement, clothes folding, phone charging, stool assembly, phone packaging) [§04].
• Phone packaging scaling, matched per-robot collection time: 4 robots → 50.8%, 16 robots → 86.7%, 96 robots → 98.3%. The 4-robot result is below the 79.2% starting policy — small fleets underperform the initial checkpoint on hard tasks [§04 scaling trends].
• Phone charging: 60.0% → 97.5% (93.8% failure reduction) [§04].
• Ablation (phone packaging, 16 robots, 120 trials each): Full config 86.7%; retrieval-without-context 57.5% (−29.2 pp); the two other knobs (sync every 500 steps, 88% priority retention) each max out the task’s success-rate curve [§04 ablation].
• Deployment: policies evaluated on two robots held out from training data collection, 60 trials per robot per task, no human intervention, 93.3–99.2% success across the 5 tasks [§01, §06].
• Fault stress-test: 96-robot run collected 9,834 episodes with 30 fault incidents affecting 22 robots — 1.3% of total robot-time lost to faults [§05].

Why it’s interesting

CEL/FERT sits at an interesting corner of the recipe-lever board on VLA Models: it is a per-task self-improvement loop that keeps the model-side recipe fixed (SFT + (M)HG-DAgger + online BC) and puts the entire improvement budget into fleet scale and failure-as-query reuse of autonomous rollouts. This is a sharper distributed/operational instance of the on-robot intervention-scaling thesis seen in Enact launch — post-training infrastructure that generates targeted recovery data for robotics VLAs Enact, MicroFactory — 99.9% reliability via $5 human-in-the-loop DAgger retraining on Jetson (Ilir Aliu × Igor Kulakov podcast) MicroFactory, and Empirical results fine-tuning π0.5 on a real manufacturing task, which also use intervention-rich DAgger-style loops but at single-digit-robot scale with human corrections remaining in the loop throughout; CEL replaces human corrections after initialization with autonomous rollouts, trades them for a fleet of 96, and uses failures only as retrieval queries (not as negative training signal, cf. Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning Q-Planning’s Q-function-based reweighting on the same substrate).

The 4-robot-underperforms-initial-policy result is also a direct empirical answer to a question the fleet-self-improvement literature has been circling: at what minimum fleet size does the loop start to pay? Here, the inflection sits above 4 robots on phone packaging, with 16 being the smallest fleet that reliably improves over the (M)HG-DAgger initialization — concrete calibration for anyone budgeting fleet-RL/DAgger infrastructure.

See also

• Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning — Q-Planning also freezes a BC policy and treats failed rollouts as valid signal, but routes it through a learned Q-function used for best-of-N re-weighting; FERT puts the signal into retrieval+resampling without any value head.
• Enact launch — post-training infrastructure that generates targeted recovery data for robotics VLAs — Enact productizes the failure-recovery DAgger loop as a service with human corrections; FERT drops the corrections after initialization and scales to 96 robots.
• Empirical results fine-tuning π0.5 on a real manufacturing task — 240 DAgger-style intervention rollouts on a 1h seed policy reach 88% success on bimanual pick-and-insert; sibling result at single-robot scale with full human-in-the-loop corrections rather than CEL’s binary-label-only fleet loop.
• ENPIRE: Agentic Robot Policy Self-Improvement in the Real World — ENPIRE agentic policy-self-improvement loop over a single robot/sim with mixed PI regimes (heuristic, BC, offline RL, online RL); CEL is the fleet-scale pure-online-BC-on-successes instance of the same self-improvement framing.
• UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning — Agentic intervention for efficient real-world RL with a learned intervention agent; CEL replaces the intervention agent with autonomous rollouts and uses failures as retrieval queries instead of invocation triggers.
• VLA Models — recipe-lever board for VLA/manipulation foundation models; FERT occupies the “per-task self-improvement from autonomous fleet rollouts with binary labels only” cell.
• Synthetic Training Data — FERT’s shared experience buffer is a model-generated dataset (fleet’s own successful rollouts) curated by a verification gate (per-episode binary success label + similarity-based retrieval against failure queries).