Human-to-Robot Transfer via Bridging Representation and Action Space in VLAs

TL;DR

H2RB adapts a pretrained Vision-Language-Action (VLA) policy to a new target robot embodiment using unpaired human and robot demonstrations, bridging the human-robot embodiment gap at two interfaces simultaneously: a geometry-anchored visual alignment that puts human and robot observations in a compatible representation, and a decoupled-decoder tokenizer that shares an action latent space across the two embodiments without forcing both domains through a single action mapping. On real-world tasks, H2RB lifts object-transfer success from 8.0 to 51.8 and raises skill transfer to 62.7 in settings where co-training baselines completely fail to transfer human-demonstrated skills, with out-of-distribution object-placement transfer improving from 6.0 to 45.3.

Key claims

Pretrained VLA policies cannot cheaply absorb cheap-to-collect human demonstrations because the embodiment gap at both the observation and action interfaces breaks naive co-training on unpaired human + robot data [§Abstract].
A geometry-anchored visual alignment stage grounds human and robot observations in a compatible representation, serving as the first of two bridging interfaces that lets human demonstrations provide useful behavioral supervision without washing out the VLA’s inherited visual/semantic/control priors [§Abstract].
A decoupled-decoder tokenizer shares action latents across the human and robot domains while allowing each to keep its own action mapping, rather than collapsing both to a single action head [§Abstract].
On real-world tasks, H2RB lifts object transfer from 8.0 to 51.8 and reaches 62.7 on skill transfer in settings where co-train baselines “totally fail to transfer human-demonstrated skills” [§Abstract].
The benefit persists under object-placement shift: out-of-distribution transfer improves from 6.0 to 45.3 [§Abstract].

Method

H2RB is a human-robot co-training framework for adapting a VLA pretrained primarily on robot data to a new target robot + environment using unpaired human and robot demonstrations. The embodiment gap is attacked at two interfaces simultaneously. On the observation side, a geometry-anchored alignment stage grounds human and robot observations in a compatible representation. On the action side, a decoupled-decoder tokenizer shares an action latent space but routes human and robot through separate decoders rather than a single shared mapping — an explicit design choice to keep the pretrained VLA’s rich visual/semantic/control priors intact while still letting human demonstrations shape the behavior. Builds on the RLDX-1 backbone (Apache 2.0, github.com/RLWRLD/RLDX-1).

Results

Real-world evaluation reports object transfer rising from 8.0 to 51.8 and skill transfer reaching 62.7 — in settings where co-training baselines “totally fail to transfer human-demonstrated skills” — with out-of-distribution transfer under object-placement shift improving from 6.0 to 45.3 [§Abstract]. No sim-benchmark or per-task breakdown is retrievable from the abstract at filing time.

Why it’s interesting

Stakes out a distinct “bridge at both interfaces simultaneously” cell on the Human-to-Robot Retargeting recipe board — observation-side geometry-anchored alignment paired with action-side decoupled-decoder sharing a latent but not a mapping. On the representation axis it contrasts with LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition LUCID (task-intent as shared interface, not raw observations) and with World Action Learning via Interaction-Centric Spectral Latent Guidance WING (DCT-truncated latent-action guidance); on the action axis it contrasts with Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots Translation-as-Bridging-Action’s reduce-to-3-DoF-wrist recipe by keeping the full action channel and absorbing the gap at a decoupled decoder instead. Also complements the VLA Models “keep the pretrained backbone, learn a small adapter” line (frozen-base + residual proxy, latent-interface, foresight-token bottleneck) by moving the adaptation substrate from robot-only data to unpaired human + robot data.