CHORD: Object-Centric Contact Wrench Guidance for Dexterous Manipulation

TL;DR

CHORD (NVIDIA Isaac) transfers human manipulation skills to dexterous robot policies through an object-centric contact-wrench representation: instead of matching where contact occurs in space, it matches the 6-D force-and-torque the contact induces on the object. This recasts the human-to-robot retargeting reward from position similarity (which loses contact intent across the kinematic gap) to wrench similarity, making contact-rich RL scalable across rigid, articulated, and bimanual tasks. The team releases a simulation benchmark of 4,739 bimanual dexterous tasks built from motion-capture and reconstructed in-house videos; on 1,831 evaluations CHORD reports 82.12% average success, 90.77% on whole-body manipulation from hand-only / third-person demos, plus open-loop and closed-loop real-world transfer.

Key claims

Position-based contact guidance matches where contact occurs in space but locations close in space induce different contact normals and moment arms, so position similarity can still produce unstable object motion [§Method, Position vs Wrench panel].
The CHORD representation packs each contact as a 6-D wrench (force + torque) computed from contact location, surface normal, and moment arm; matching wrenches reproduces the force-and-torque generation behind the demonstrated manipulation [§Method].
Demonstrated contacts and contact wrenches are converted into RL rewards for dexterous robot-hand policies in a multi-stage pipeline (contact extraction → wrench computation → reward shaping → RL) [§Method, pipeline figure].
The released benchmark contains 4,739 simulation-ready bimanual dexterous manipulation tasks paired with human demonstrations spanning rigid, articulated, and multi-object manipulation, constructed from mocap datasets and reconstructed in-house videos [§Benchmark].
Across 1,831 diverse evaluations CHORD reaches 82.12% average success rate; on the in-paper task suites it reports AUC 0.687 ± 0.358 vs DexMachina’s 0.232 ± 0.214 on DM, MT-SR 0.639 vs ManipTrans 0.428 on MT, and AUC 0.895 ± 0.052 vs DexMachina 0.211 ± 0.138 on Ours-1 [Results table].
The contact-wrench representation is embodiment-agnostic: paired with a whole-body motion inpainting module, a three-fingered humanoid learns from hand-only and third-person demos at 90.77% success rate [§Embodiment generalization].
Learned policies transfer to real dexterous hands in both open-loop and closed-loop settings [§Real-world transfer].

Method

CHORD takes a human demonstration (mocap or reconstructed from video) and converts each frame’s contact set into 6-D wrenches on the manipulated object — combining force vectors at contact points with their moment arms about the object center. An RL policy for the robot hand is trained in simulation with a reward that compares the wrench induced by the robot’s current contacts to the demonstrated wrench at the same task phase, rather than penalizing distance between human and robot contact locations. This sidesteps the kinematic gap (robot fingertips can’t reach the same surface point as a human thumb but can generate the same net force/torque elsewhere) and reframes “match the demonstration” as “reproduce the demonstration’s instantaneous effect on the object.” The benchmark side of the contribution is the 4,739-task simulation suite built from mocap (GRAB-style) and a separately reconstructed in-house human-video dataset, exposed as a uniform RL environment. For whole-body humanoid manipulation a motion-inpainting module fills in the body trajectory consistent with hand-only or third-person demos, then the same wrench-similarity reward drives policy learning.

Results

Headline: 82.12% mean success across 1,831 benchmark tasks, 90.77% on whole-body manipulation from hand-only / third-person demonstrations [Results section]. Head-to-head, on DM CHORD reports AUC 0.687 ± 0.358 vs DexMachina 0.232 ± 0.214; on MT MT-SR 0.639 vs ManipTrans 0.428; on SP SP-SR 0.359 ± 0.482 vs Spider 0.333 ± 0.488 (statistical tie on this slice); on the authors’ own Ours-1 suite AUC 0.895 ± 0.052 vs DexMachina 0.211 ± 0.138 and SP-SR 0.999 ± 0.000 vs Spider 0.133 ± 0.327; on Ours-2 SP-SR 0.982 ± 0.022 vs Spider 0.533 ± 0.503 [Results table]. The page notes scores are comparable within each row but baselines use different task suites and evaluation protocols. Real-world transfer is shown qualitatively (clip grid) in both open-loop replay and closed-loop control on dexterous hands [§Real-world transfer]. The blog does not report ablations isolating the contact-wrench reward vs the benchmark contribution; that decomposition would require the underlying paper.

Why it’s interesting

CHORD slots directly into the active sensorimotor-vs-VLA debate the wiki has been tracking. Where ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control (ConTrack, UCSD/Xiaolong Wang) treats object tracking as a hard RL constraint and lets a dual variable redistribute the rest of the reward, CHORD goes one level finer: not just match the object trajectory but match the wrench — the 6-D causal signal that generates the object trajectory. Both papers are constructive answers to Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level‘s critique that VLA work skips contact dynamics, and they’re complementary: ConTrack’s object-tracking constraint can in principle consume CHORD-style wrench shaping as its style/contact term. CHORD also rhymes with Do as I Do: Dexterous Manipulation Data from Everyday Human Videos (Do as I Do, Malik group) which is the upstream perception+retargeting half — DO AS I DO converts in-the-wild human RGB into hand-object trajectories, CHORD provides the RL recipe that makes those trajectories physically executable via wrench-matching. The 4,739-task benchmark is a concrete answer to the “where do dexterous RL environments come from at scale” gap that RL Environment Platforms tracks, sitting between 2606.18123-roboarena-distributed-real-world-evaluation‘s real-world distributed protocol and synthetic-data factories like SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation. The most novel ingredient is the embodiment-agnostic claim: if wrench-similarity actually transfers from human hands → three-fingered humanoids at 90.77%, this is the cleanest filed demonstration that the causal abstraction over contact (force + torque on object) generalizes across kinematics in a way joint poses and even contact locations don’t.