Skip to content
Bud's Wiki

HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning

TL;DR

Section titled “TL;DR”

HydroShear is a tactile-sensor simulator that extends hydroelastic contact models with path-dependent force tracking — instead of computing tactile forces from instantaneous contact only, it remembers each contact point’s motion history across a soft elastomer membrane, then converts accumulated deformation into realistic 3-D force fields with friction, slip, and material properties. RL policies trained entirely in the simulator and deployed zero-shot on a real Franka arm with GelSight Mini sensors hit 93% average success across four contact-rich tasks (peg insertion, bin packing, book shelving, drawer pulling with force perturbations), against 34% for TacSL and 58–61% for FOTS. The simulator is GPU-parallelizable and calibrated to real GelSight Mini data via four physical parameters (force dissipation, tangential/normal buildup, friction coefficient).

Key claims

Section titled “Key claims”
  • Path-dependent force tracking — remembering per-contact-point motion history across the elastomer — is the key innovation, distinguishing HydroShear from instantaneous-contact tactile approximations [blog body].
  • HydroShear handles full 3-D motion (tilting, rolling, in-plane sliding), not just planar shear, which the authors argue is required for dexterous manipulation [blog body].
  • Zero-shot sim-to-real: policies trained purely in HydroShear and deployed on a real Franka + GelSight Mini with no modification or fine-tuning achieve 93% average success on four contact-rich tasks [blog body].
  • Baselines TacSL (simplified force approximations) and FOTS (learning-based) reach 34% and 58–61% respectively on the same tasks — a large gap attributed directly to tactile-shear fidelity [blog body].
  • The four evaluation tasks span distinct tactile-feedback regimes: peg insertion (unknown grasp orientation, alignment via tactile only), bin packing (multi-object contact from many directions), book shelving (broad contact patches larger than fingertip), drawer pulling (slip detection under external force perturbations) [blog body].
  • Policies observe only touch + proprioception (joint positions, gripper state), with no access to object pose [blog body].
  • Calibration isolates four physical parameters (force dissipation across the sensor, tangential and normal force buildup, object-elastomer friction) that ground the simulator to real GelSight Mini data [blog body].
  • The performance gap over TacSL/FOTS correlates with simulation fidelity: peg insertion (precise-force-critical) and drawer pulling (slip-critical) show the largest margins [blog body].

Method

Section titled “Method”

HydroShear starts from an existing hydroelastic contact model — a continuum approximation that represents soft-body contact via a scalar pressure field over a compliant surface — and augments it with per-contact-point path tracking. When a grasped object translates or rotates relative to the sensor, distinct surface points contact the elastomer at different times; HydroShear tracks each such point’s tangential and normal displacement history and integrates that history into an accumulated shear/normal deformation, from which the local force field is derived (friction and slip are enforced at the material-contact level). The simulator runs GPU-parallel, enabling large-scale RL rollouts.

Calibration uses controlled data from a real robot arm sliding known geometries across a GelSight Mini, fitting the four physical constants. RL policies are trained with tactile + proprioceptive observations only; the same policy weights are then loaded onto a Franka arm with GelSight Mini fingertips with zero adaptation.

Results

Section titled “Results”
  • Average success rate across four tasks: HydroShear 93% vs. FOTS 58–61% vs. TacSL 34% [blog body].
  • Task suite: peg insertion (cylindrical peg into tight socket, unknown grasp orientation), bin packing (cube into slot in crowded bin), book shelving (large-face lateral insertion under gravity), drawer pulling with random force perturbations (slip detection + grip modulation) [blog body].
  • Blog claims the fidelity–performance correlation: on precision-force tasks (peg insertion) and slip-reactive tasks (drawer pulling) HydroShear’s margin is largest; on broader-contact tasks the gap narrows [blog body].
  • Reported speed advantage: Tactile Genesis (a separate concept-page paper) reports 3–20× GPU-throughput vs. HydroShear among other simulators — HydroShear is fast enough for RL at scale but not the fastest option on the wiki (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks cites HydroShear as a baseline).

Why it’s interesting

Section titled “Why it’s interesting”

HydroShear directly addresses the placement / abstraction / simulator triangle that Tactile sensing for manipulation has been triangulating. Where Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks ablates what tactile abstraction and where to put sensors under a unified GPU simulator, and where FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation and PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images argue you can skip the sensor entirely and infer tactile from vision, HydroShear stakes the claim that simulator fidelity of shear is the load-bearing variable — and backs it with a large zero-shot sim-to-real number (93%) on tasks the earlier work does not run. It also names the simulator that Tactile Genesis lists as one of its faster-but-lower-fidelity baselines, which lets the wiki attribute the throughput-fidelity trade-off to a specific comparator instead of leaving it abstract. Complements Solving Dexterity: A Full-Stack Approach (mimic hand M1 + wearable U1) which pursues the hardware-side alternative (backdrivable motor-current force sensing on every joint) — HydroShear’s argument for simulator investment is orthogonal to mimic’s argument for actuator investment, and both target the same downstream: contact-rich manipulation without expensive real-world data collection.

See also

Section titled “See also”