HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone

TL;DR

HiFi-UMI (Simple AI / Simple World Lab) is a portable robot-free UMI capture rig co-designed for action fidelity rather than just scale: head-mounted offline stereo-inertial SLAM, native (not reconstructed) inter-gripper relative pose, a shared microsecond GPIO trigger across sensors, and two ~200° wide-angle cameras per hand. It reaches 3 mm workspace-local end-effector accuracy without any external tracking infrastructure. Using this corpus, the paper demonstrates zero-robot post-training — a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot and matches an in-domain teleoperation baseline across three separate VLA/WAM backbones (StarVLA-QwenPI, OpenPI-π0.5, LingBot-VA), with success-rate differences of only −2.5, +3.1, and −0.6 percentage points respectively. The team open-sources HiFi-UMI-2K, 2,000 hours of microsecond-synchronized ultra-wide-FoV demonstrations, each auto-reconstructed and simulation-replay-validated (CC-BY-4.0).

Key claims

The central thesis is stronger than prior UMI recipes: eliminate target-task robot teleoperation from post-training entirely, rather than merely reduce its fraction [§Abstract].
The rig reaches 3 mm workspace-local end-effector accuracy without external tracking infrastructure via head-mounted offline stereo-inertial SLAM [§Abstract].
Inter-gripper relative pose is captured natively (from the rig geometry + SLAM) rather than reconstructed post-hoc from independent per-hand estimates [§Abstract].
Cross-sensor synchronization is <40 μs via a shared microsecond GPIO trigger, tightening the temporal alignment budget below what post-hoc timestamp alignment can hit [§Abstract].
Two wide-angle cameras per hand cover ~200° per hand, i.e. six-view ultra-wide-FoV per bimanual demonstration [§Abstract].
Zero-robot post-training matches teleop across three backbones: post-training solely on HiFi-UMI data yields success rate deltas of −2.5, +3.1, −0.6 percentage points vs an in-domain teleop baseline on StarVLA-QwenPI, OpenPI-π0.5, and LingBot-VA respectively [§Abstract].
On a precision insertion task, the strongest policy reaches 85% success even though the teleop baseline is collected in the evaluation scene and no HiFi-UMI trajectory is — the accuracy budget of the rig is what makes precision tasks work without an in-scene teleop anchor [§Abstract].
Pre-training on 4,000 hours from the same corpus lowers action error on ten unseen tasks by 41%, and on StarVLA-QwenPI raises real-robot success by a further +18.1 percentage points on top of the zero-robot post-training result [§Abstract].
The public release is HiFi-UMI-2K: 2,000 hours of microsecond-synchronized ultra-wide-FoV demonstrations, each automatically reconstructed and validated via simulation replay [§Abstract].

Method

HiFi-UMI is a portable, robot-free bimanual UMI system whose design is driven by four fidelity axes rather than raw throughput: trajectory accuracy, inter-gripper relative pose, cross-sensor synchronization, and field of view. Trajectory accuracy comes from a head-mounted offline stereo-inertial SLAM stack (i.e. the SLAM solve is run after data collection rather than online), avoiding both external mocap infrastructure and the drift budget of online VIO. Relative pose between the two grippers is captured directly from the rig’s known geometry plus the shared SLAM frame — the paper’s contrast is that reconstructing it from two independent per-hand pose estimates loses accuracy in exactly the regime (bimanual coordination) where it matters most. Time alignment is enforced at the hardware level via a shared microsecond-GPIO trigger, dropping cross-sensor sync below 40 μs. Each hand carries two wide-angle cameras giving ~200° coverage per hand, i.e. a six-view ultra-wide-FoV bimanual observation. The processing pipeline is fully automated end-to-end: trajectory reconstruction, simulation replay, and quality validation are run per demonstration before it enters the dataset. Downstream training is a standard pre-train + post-train VLA recipe applied to three heterogeneous backbones — StarVLA-QwenPI, OpenPI-π0.5, and LingBot-VA — with the load-bearing experimental question being whether no real-robot teleop data in the post-train stage is enough.

Results

Zero-robot post-training vs teleop baseline (success-rate delta, percentage points): StarVLA-QwenPI −2.5, OpenPI-π0.5 +3.1, LingBot-VA −0.6 [§Abstract]. Deployment is direct — no fine-tuning on target robot post-HiFi-UMI-only post-training.
Precision insertion: the strongest policy reaches 85% success on a precision insertion task, where the teleop baseline is collected in the evaluation scene and the HiFi-UMI corpus contains no in-scene trajectory [§Abstract].
Pre-training scaling: 4,000 hours of HiFi-UMI pre-training lowers action error on 10 unseen tasks by 41% and adds +18.1 pp real-robot success on StarVLA-QwenPI beyond the zero-robot post-training baseline [§Abstract].
Corpus: HiFi-UMI-2K release contains 2,000 hours of microsecond-synchronized ultra-wide-FoV demonstrations, each auto-reconstructed and sim-replay-validated (CC-BY-4.0) §Abstract, dataset page.

Why it’s interesting

HiFi-UMI is the sharpest test yet of whether the UMI-as-pretraining shortcut can be pushed all the way to zero real-robot teleop in post-training, and the answer the paper puts on the table is “yes, if you pay for it in rig fidelity.” That’s a different lever than the two other UMI-scale points already filed: Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training (XR-1) scales embodiment-free UMI to 100K hours but still keeps 7,200 hours of in-house teleop data in the post-train mix, and HandUMI — Robot-free bimanual data collection retargetable to any parallel-gripper arm (Brik H. Meza launch tweet) (HandUMI) opens the collection substrate to small teams with a hardware kit but says nothing about fidelity or eliminating the real-robot anchor. HiFi-UMI is smaller (2K public hours, 4K used for pre-train scaling) but claims that the accuracy budget of the rig (3 mm, <40 μs, native relative pose, six-view) is what lets you drop the anchor entirely across three heterogeneous VLA backbones. It contrasts most directly with EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data‘s scale-with-noisy-channels position (20,854 h of egocentric video, keep the full 22-DoF and pay in scale) — HiFi-UMI’s counter-position is “raise fidelity, drop the anchor, hit precision tasks.”

For the Human-to-Robot Retargeting concept it stakes a new point on the interface-location axis: fix the retargeting problem in the collection rig (like HandUMI and Xiaomi-XR-1) but push the fidelity bar high enough that the fixed rig substitutes for in-domain teleop on precision tasks, not just tasks where the retargeting-quality vs data-scale tradeoff has slack. The three-backbone ablation (VLA/WAM families both) is unusual and load-bearing — it suggests the result is a property of the data, not of a favorable interaction with one backbone’s inductive biases.