OopsieData — a multi-lab dataset of real robot manipulation failures

TL;DR

Zhiyuan Zhou announces OopsieData, a multi-lab effort to build the first large-scale open dataset of real-robot manipulation failures and suboptimal behavior. The premise: labs produce failure clips continuously during policy evaluations, play-data collection, and online RL sessions, but almost all of it is thrown away — leaving RL, reward modeling, failure detection, and world modeling starved for the negative data they actually need. The project ships a toolkit (recording, formatting, uploading in “a few lines,” plus a Codex/Claude agent skill), has 17 labs already contributing, and accepts contributions until Oct 5; significant contributors qualify for dataset-release co-authorship and access to the pooled corpus. Project page: oopsie-data.com; a CoRL 2026 workshop is planned.

Key claims

Behavior cloning on success-only data yields a policy with no learned notion of a bad grasp and nothing to recover from once off-distribution — the failure and suboptimal data that would teach this are exactly what current corpora omit [Post].
Anything that reasons about quality of behavior — RL, reward modeling, failure detection, world models — benefits from mixed-quality real-world data, and no good open dataset of that shape currently exists [Post].
The most useful data in robotics is routinely deleted: every policy eval, play-data run, and online RL session generates suboptimal behavior that gets discarded [Post].
Contribution is near-zero-cost because the data is already being produced — the OopsieData toolkit records, formats, and uploads rollouts in a few lines, and ships an agent skill so Codex/Claude can wire it up [Post].
17 labs have already contributed; contribution deadline Oct 5; contributors get access to the pooled corpus and can qualify for co-authorship on the public release [Post].
The success/failure asymmetry shows up even in a simple ball-grasp: same policy, same conditions, a slight gripper offset flips outcome — the kind of fine-grained mode the dataset is designed to capture [Post; project page].

Method

OopsieData is a data-collection consortium rather than a paper. The toolkit standardizes recording and upload format across contributing labs so heterogeneous robots and tasks produce trajectories consumable by a single downstream pipeline. Labs sign up at oopsie-data.com, install the toolkit, and either (a) convert existing rollouts to the OopsieData format or (b) start logging new rollouts with a few lines of code — including an agent skill so Codex/Claude can autowire the integration. The submitted data includes both successes and failures from the same policy runs, so downstream users get matched-condition pairs, not just isolated failure clips. The design bet is that the pooled multi-lab corpus will be much more diverse (across robots, tasks, cameras, lighting, gripper hardware) than any single lab could collect alone, and that diversity is what unlocks the downstream RL / reward-model / failure-detection / WFM use cases.

Results

No quantitative results yet — this is a dataset-in-progress announcement. Reported alpha state: 17 labs contributing, contributions accepted until 2026-10-05, workshop planned at CoRL 2026. Example paired episodes (successful grasp / failed grasp of the same ball with the same Aloha + diffusion-policy setup) are shown on the project page to illustrate the fine-grained failure modes the dataset targets.

Why it’s interesting

Sits at a load-bearing gap in the wiki’s current robotics coverage. Two recent filed papers attack the failure-data-scarcity problem from opposite directions but neither ships a real-world multi-lab open corpus: EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstration proposes collecting failures via egocentric human capture (>10× teleop rate) and DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation proposes simulator-synthesized physical failure trajectories to sidestep human labeling. OopsieData stakes out the third position — pool the real-robot failure data that’s already being generated across labs and would otherwise be deleted — and complements both: it produces the real-hardware distribution simulator-synthesized data can’t reach, and it scales differently from single-lab egocentric-human capture because contribution cost is near zero per lab.

For VLA Models, this is the data-substrate counterpart to the RECAP-style (π*0.6: a VLA That Learns From Experience (RECAP)) recipe that requires both successes and failures to compute advantage-conditioning labels, and to the World Value Models line (World Value Models for Robotic Manipulation) that argues a separate, generalist, recipe-agnostic value model needs mixed-quality data — Suboptimal-Value-Bench (introduced in WVM) is the natural evaluation target for anything trained on OopsieData. The RW-RL Dataset: Real-World Reinforcement Learning for Robots RW-RL Dataset already takes a related “on-robot mixed-regime data at scale” position but from a single lab (SJTU MINT); OopsieData’s federated model is the multi-lab generalization of the same bet. Failure-verifier products like Instance Labs — Verifying Robot Learning Episode Success are another obvious downstream consumer.