Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

TL;DR

UHAS proposes a shared, embodiment-invariant action space for multi-fingered manipulation: a canonical sphere whose deformations parameterize any dexterous hand’s finger configuration, coupled with a Cascade Inverse Kinematics (CIK) algorithm that maps a deformed sphere to embodiment-specific joint angles. RL policies are trained directly in this sphere-deformation space for in-hand cube reorientation and evaluated across Allegro, LEAP, Shadow, and MANO Human hands in both sim and real. The main claim is that a single policy trained in UHAS controls kinematically dissimilar hands with different finger counts, transfers zero-shot to unseen embodiments, and finetunes quickly across hands.

Key claims

Robot hand actions can be represented as geometric deformations of a canonical sphere parameterized by four driving planes with lateral rotation (Δθ) and radial displacement (Δr), producing a low-dimensional action space that is shared across Allegro, LEAP, Shadow, and MANO Human hands [§Abstract, Fig. sphere deformation].
A Cascade Inverse Kinematics algorithm maps a deformed sphere to embodiment-specific joint configurations by first solving lateral joints (finger spread/rotation) and then encompassing joints (finger curl), producing valid per-hand joint configurations from the shared sphere representation [§CIK].
A single RL policy trained in UHAS controls multiple robotic hands simultaneously on in-hand cube reorientation, and transfers zero-shot to unseen hands with rapid finetuning across embodiments [§Abstract, experiments].
Real-world in-hand cube reorientation is demonstrated on both LEAP Hand and Allegro Hand from the same UHAS-space policy [§experiments].

Method

The method has two components. The action-space side represents any dexterous hand as a canonical sphere carrying four “driving planes”; rotating the planes controls lateral finger deformation (Δθ), and radially displacing the drivers controls encompassing deformation (Δr). The final sphere surface is reconstructed via interpolation between the driving vectors, producing a low-dimensional action vector that is the same shape for all hands.

The retargeting side is a Cascade Inverse Kinematics solver. Hand joints are classified into (a) lateral joints — those that abduct/adduct fingers — and (b) encompassing joints — those that flex/curl fingers. Given a target deformed sphere, CIK solves the lateral joints first (aligning finger direction to the sphere-plane rotation) and then the encompassing joints (matching the radial deformation), producing per-hand joint targets. RL policies (PPO on Isaac Gym, based on the project-page description) are then trained end-to-end in the sphere-deformation action space on in-hand cube reorientation, with the CIK-produced joint commands sent to whichever embodiment is being simulated. Because the policy outputs live in the shared UHAS coordinate system, one policy can drive multiple embodiments without retraining the action head.

Results

Simulation: dexterous in-hand cube reorientation succeeds on Allegro, LEAP, Shadow, and MANO Human hands from a shared UHAS policy [§experiments].
Zero-shot transfer to unseen hands and “rapid finetuning across embodiments” are reported [§Abstract]; specific transfer deltas are not in the abstract.
Real-world deployment: successful in-hand cube reorientation on both LEAP Hand and Allegro Hand [§real-world experiments].

Why it’s interesting

UHAS lands squarely on the unresolved axis on the Human-to-Robot Retargeting concept page: where should the embodiment-invariant interface live? The existing filed positions are 6-DoF joint-pose retargeting with physics-in-the-loop (ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting), kinematic hand-object reconstruction (Do as I Do: Dexterous Manipulation Data from Everyday Human Videos), a reduced 3-DoF wrist-translation channel (Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots), full 22-DoF Sharpa-hand retargeting paid for with scale (EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data), and task-intent as scene-level prediction (LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition). UHAS proposes a sixth: a geometric interface at the fingertip-surface level (sphere deformation) that is neither joint-space nor task-space nor low-DoF-wrist, and includes MANO Human Hand as one of the four target embodiments — closing the loop with human data via the same representation the robot policy uses. Contrasts most directly with EgoScale, which keeps the full 22-DoF joint channel and argues for scale, and with Translation-as-a-Bridging-Action, which drops rotation channels because they are noisy on humans; UHAS keeps rotation but reparameterizes it into a shape that is by construction shared across embodiments.