Skip to content
Bud's Wiki

DemoMimic: One Demonstration, Many Objects — Generalizing Manipulation via Local Contact Geometry

TL;DR

Section titled “TL;DR”

DemoMimic (Stanford; Sharma, Sahoo, Huang, Fei-Fei Li, Wu, Sadigh, Bohg) trains a dexterous multi-fingered manipulation policy from a single human demonstration by having the policy focus on object geometry local to the contact points rather than global object shape. A contact-centric reward design — an Alignment Reward aligning contacting hand-link normals with object normals, plus a Sustained Contact Reward that penalizes intermittent releases — narrows the sim-to-real gap so a single real-world policy transfers across objects of varying shape, scale, mass, and friction whenever local contact structure is preserved. Reports 71% average real-world success across 16 objects, four tasks, and two hand embodiments, with the smallest sim-to-real gap among baselines.

Key claims

Section titled “Key claims”
  • A single human demonstration can seed an RL policy in simulation that transfers to real hardware across many objects, provided the policy focuses on local contact geometry rather than global object shape [§Method].
  • Contact-centric rewards — Alignment Reward (surface-normal alignment between hand link and object at contact) and Sustained Contact Reward (quadratically growing bonus for contact streaks inside demonstration windows) — improve sim-to-real consistency by making contacts stable and well-conditioned, preventing intermittent releases that pass in simulation but drop objects on hardware [§Method].
  • Baselines and reward ablations (DexMachina*, HERMES*) perform well in simulation but drop sharply in the real world; DemoMimic’s contact rewards keep real-world performance close to simulated performance [§Results, comparison table].
  • The same policy generalizes to unseen objects that vary in material, weight, and size — the failure mode disclosed on the page is the Robot Hand Box task, where the lid geometry differs from what was seen in simulation [§Results, note].
  • End-to-end pipeline: single human demo → RL policy π_RL trained in sim with extensive domain randomization + contact rewards → distilled to a deployable policy [§Method].

Method

Section titled “Method”

The single human demonstration is used to train a reinforcement-learning policy π_RL in simulation. Extensive domain randomization plus the two contact-centric reward terms produce a policy the authors then distill for deployment. The Alignment Reward (AR) aligns each contacting hand link’s surface normal with the object’s surface normal at the contact point — this encourages stable, well-conditioned contacts rather than glancing or singular contacts that a physics engine tolerates but real hardware cannot. The Sustained Contact Reward (SCR) rewards maintaining continuous contact within demonstration-derived windows and grows quadratically with the length of the contact streak; this specifically targets the intermittent-release failure mode that inflates simulator success rates. The framing of “local contact geometry” is the load-bearing generalization axis: the policy is not asked to memorize object shape but to reproduce the contact configuration around the interaction site, so any object whose local geometry at the contact points matches the demonstration can be handled by the same policy.

Results

Section titled “Results”
  • 71% average success across 16 objects × 4 tasks × 2 hand embodiments on real hardware [Results].
  • Smallest sim-to-real gap among baselines evaluated (DexMachina*, HERMES* and reward ablations included) — baselines match DemoMimic in simulation but drop sharply on hardware [§Results, comparison table].
  • Failure disclosed: Robot Hand Box task has lower success because the lid geometry differs from the training simulation, i.e. the local contact geometry assumption is violated [§Results note].
  • Per-object, per-task success rates, exact baseline numbers, and hand embodiments (make/model) are not spelled out on the project page — full breakdown belongs to the paper body, which was not fetched at filing time.

Why it’s interesting

Section titled “Why it’s interesting”

DemoMimic stakes a new position on Human-to-Robot Retargeting‘s interface-location axis: the shared human–robot interface is local contact geometry at the interaction site, and the retargeting error is absorbed by sim-side RL with contact-centric rewards rather than by a fixed kinematic pipeline. This complements CHORD: Object-Centric Contact Wrench Guidance for Dexterous Manipulation CHORD, which uses object-centric contact wrenches as the retargeting interface — DemoMimic’s Alignment Reward is the geometric analog, and its Sustained Contact Reward is the temporal one. Contrasts with same-authorship DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Library DexAgent (Stanford, Fei-Fei/Wu/Huang overlap) which fixes the sim-to-real problem at the pipeline layer via an agentic tool library over one video, whereas DemoMimic fixes it at the reward layer inside a single-video RL run — orthogonal levers. Sibling in framing to Do as I Do: Dexterous Manipulation Data from Everyday Human Videos DO AS I DO (kinematic 4D reconstruction from monocular video → retarget to multi-fingered hand) but distinct in staking generalization on local rather than global geometry preservation. On the Tactile sensing for manipulation axis, DemoMimic is the “geometry-only, no tactile sensor” endpoint — the Alignment Reward is what tactile sensing would supply as measurement, replaced here by a simulation-side reward that assumes access to ground-truth contact normals during training.

See also

Section titled “See also”