Tactile sensing for manipulation

TL;DR

Recipes for making touch a first-class ingredient of dexterous manipulation policies — either by ingesting high-frequency contact signals directly, by inferring contact from vision, or by simulating tactile sensors at scale so hardware and policy choices can be ablated. The three filed instances split along that design axis: Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction handles the policy side (variable-rate MoT + temporal tactile VQ-VAE on real hardware), PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images handles the vision-only substitute (weakly-supervised per-pixel fingertip pressure from RGB), and Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks handles the upstream tooling question (GPU-parallel unified tactile simulator with 8 sensor abstractions + explicit placement / resolution / noise / abstraction ablations across three dexterous tasks). Directly addresses the sensorimotor-axis counter-position the VLA Models cluster keeps circling back to — that pure VLM-plus-action-head recipes under-weight contact dynamics.

Key claims

• Proprioception is insufficient for contact-rich dexterous tasks — even the cheapest binary-contact tactile abstraction beats a no-tactile baseline on all three tested tasks (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks §4.1, Fig. 7).
• Placement dominates sensor type: whole-hand coverage substantially outperforms fingertip-only, and adding taxels on the palm and proximal phalanges is a bigger marginal win than upgrading the fingertip sensor (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks §4.1). This directly contradicts the de-facto convention of commercial tactile hardware, which concentrates spatial resolution at the distal fingertip pad.
• Per-taxel force/torque is the recommended default abstraction across dexterous tasks, matching or outperforming binary contact, contact depth, elastomer displacement, and proximity in aggregate — with proximity edging it out only on tasks where the object approaches the hand rather than being already grasped (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks §4.1).
• Resolution matters far less than coverage: ~200 taxels distributed across the whole hand suffice for the three tested dexterous tasks (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks §4.1).
• Elastomer marker displacement (the GelSight-style abstraction) underperforms force/torque when local-force reading is what the task needs — displacement at one taxel is a function of indentation and shear at neighbouring taxels, blurring the local force vector (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks §4.1).
• The rate mismatch between high-frequency touch (e.g. kHz-class fingertip sensors) and slower vision/language streams can be handled by a variable-rate Mix-of-Transformer with a per-modality tactile encoder, without sacrificing the capabilities of an existing VLA (Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction project page).
• Discrete tokenisation via a temporal tactile VQ-VAE turns raw touch into a code stream the policy can attend over — contrasted on the project page with prior “static” tactile encoders — and delivers >30% absolute average success-rate improvement over the strongest baseline across 12 delicate-force / deformable-object tasks (Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction).
• Fingertip pressure can be estimated from a single RGB image by an SE-ResNeXt-50 + FPN encoder-decoder trained with weak “press finger X at force Y” contact labels plus adversarial domain alignment — reaching 41.9% contact IoU / 27.5% volumetric IoU on textured overlays and human-annotator-parity 80.5% contact accuracy on in-the-wild surfaces where no pressure sensor could be mounted (PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images Table 2).
• Weakly-supervised prompts (“press ring finger at low force”) can substitute for instrumented ground truth as pressure supervision on 100+ natural surfaces — a scaling recipe that sidesteps the “each sensor defines a new robot” bottleneck the simulator work also flags (PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images §3.1).
• Sim-to-real transfer works for tactile policies at the fingertip-aggregate level: a policy trained on the agg_bool observation type in simulation matches its simulation success rate when deployed on a real XHand1 whose only tactile channel is fingertip-aggregate force (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks §4.1).
• Temperature is a plausible tactile modality but current real-hardware temperature sensors are too insensitive to learn even a hot-ball-among-8-distractors discrimination task from touch alone — the sensitivity gap between sim and hardware is the blocker, not the algorithm (Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks §3.3, Fig. 6).

Recent contributions

• [2026-11-02] 3D World Simulator: Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation: 3D World Simulator represents tactile signals as additional 3D particles co-located with the Gaussian geometry rather than as a side-channel sensor stream — the same latent voxel grid predicts geometry and touch spatially aligned at every timestep. Complements FELT (FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation) and T-Rex (T-Rex: Tactile-Reactive Dexterous Manipulation) on the “touch as a first-class ingredient of the world model” axis, but at the level of a predictive 3D world model rather than a reactive policy or vision-to-tactile synthesis module. No quantitative evaluation of the tactile prediction fidelity is reported on the project page.
• [2026-11-02] WUJI Hand 2 autonomous pen spinning with disturbance rejection via RL + motion-reference tracking: WUJI Tech’s own LinkedIn demo of WUJI Hand 2 autonomous pen spinning adds the vendor’s internal baseline for the same hand + task family that Dex-SuperMimic — Reproducing SuperMimic on Dexterous Hand Manipulation Dex-SuperMimic ships as hand2-pen-spin, with the added qualitative claim of disturbance rejection under operator-applied perturbation (unquantified). Sibling to Wentao Zhu — GPT-6 Astra autonomously trains an Isaac Lab RL policy for pen spinning on the Sharpa hand over 1.5 days‘s Sharpa 22-DoF pen-spin on the dexterous-manipulation evaluation surface this concept tracks.
• [2026-10-29] Dex-SuperMimic — Reproducing SuperMimic on Dexterous Hand Manipulation: Dex-SuperMimic reproduces SuperMimic (SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model) on the Wuji Hand 1/2 for in-hand cube reorientation + pen spin — two canonical dexterous-manipulation tasks for the tactile cluster but trained here without explicit tactile sensing, with contact + cage reward terms in the reorient task configuration doing the work tactile-reactive policies elsewhere delegate to high-frequency force signals.
• [2026-10-28] CARD: Contact-Anchored Retargeting for Dexterous Manipulation: CARD (Dai, He, Hogan et al.; Amazon FAR internship + CMU/Berkeley) stakes a controller-side tactile integration position distinct from the policy-input lineage this page tracks (Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction T-Rex per-finger tactile VQ-VAE, FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT per-layer force modulation, ReForce — force-aware retargeting via closed-loop residual force tracker ReForce residual force tracker atop a learned reference): the retargeter emits explicit per-hand-region contact-force targets and an integral-dominant tactile admittance controller tracks them during hardware open-loop replay, with no learned policy in between. Load-bearing ablation evidence: position-only replay vs replay with the tactile tracker on 🔨 hammer lift goes 1/10 → 8/10 (author thread) — direct quantitative statement that force set-points alone are not sufficient, the controller side carries the gain. Vid-HACO, the paper’s video-to-contact model, adds a vision-only contact inference leg beating image-based HACO (F1 63.3→67.2, AP 54.3→72.2 on HRDexDB) that sits in the same lineage as FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation FELT and PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images PressureVision++ but targets per-vertex hand contact for retargeting rather than per-pixel pressure for policy input. Answers the concept’s “observation-space contract between tactile front-end and policy” open question with a sharply different proposal: per-hand-region force set-points consumed by a controller rather than by a policy — the tactile channel’s job is tracking a planner-emitted reference, not conditioning action generation.
• [2026-10-22] CW-WAM: A Structured Contact-Wrench World Action Model for Force-Sensitive Manipulation: CW-WAM (anonymous; cw-wam-project.github.io) stakes a 6-D contact wrench position on the sensor-abstraction axis — not per-taxel force/torque, not elastomer displacement, not binary contact, but a 3-force + 3-moment interaction wrench between the end-effector and the environment — and makes it a first-class predicted modality inside the WAM alongside motion and video. Load-bearing structural choices: pair-causal attention binds each future wrench to its paired action and earlier pairs; a long fixed-length wrench memory keeps sustained interaction information (payload tension, continuing resistance) in context; the predicted wrench drives a 20 Hz admittance controller instead of being consumed as an auxiliary loss. Direct quantitative demonstration that the wrench channel resolves vision-ambiguous tasks: lifting-height accuracy improves across variable-mass loads where visually-identical 30 g and 830 g boxes cannot be distinguished from RGB alone. Peg insertion 25% → 70% (1 mm clearance per side) and adapter seating 40% → 95% vs the no-wrench WAM baseline; force-reference RMSE 4.8 N → 2.7 N (peg) and 2.0 N → 1.3 N (wiping) with smaller force fluctuations during successful execution. Sibling to CHORD: Object-Centric Contact Wrench Guidance for Dexterous Manipulation CHORD on the “contact wrench as the useful abstraction” axis but staking the WAM-side position (predict wrench + admittance controller) rather than the retargeting-side position (object-centric contact-wrench targets for dexterous imitation).
• [2026-10-19] FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity: FACT (Stanford; Cao, Hong, Yang, Choi, Fei-Fei Li, Wu, Huang Huang) stakes out a foundation-model-scale, force-in-human-pretraining-corpus position on the force-integration axis — the first filed instance of recording contact forces across a large-scale egocentric human manipulation corpus (EgoForce, released with the paper) and binding future-force prediction to coarse action refinement in a two-stage planner+refiner architecture. Distinct lever from FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks (per-layer force modulation inside the VLA — same acronym different team), ReForce — force-aware retargeting via closed-loop residual force tracker ReForce (sim-trained residual force tracker atop a reference), and Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction T-Rex (variable-rate MoT + tactile VQ-VAE on robot-side kHz touch): FACT moves the force channel one layer upstream into the human pretraining data via EgoForce and uses coupled future-force + refined-action prediction as the integration pattern. On ten contact-rich tasks (tool use, musical instruments, precision lab procedures), fine-tuning with just 10 robot + 90 human trajectories beats T-Rex by up to 60 pp. Ablations isolate force supervision in the human pretraining data as load-bearing — removing it collapses the downstream advantage while removing track or future-video prediction has smaller effects. Human:robot ratio sweep shows FACT learns from human demonstrations nearly as effectively as from robot demonstrations — the first filed quantitative statement of the exchange rate under a force-aware recipe. Sibling to TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning TouchScale on the “scale a non-visual contact channel in human data” thesis but with 6-DoF recorded force rather than 880-taxel tactile gloves.
• [2026-10-19] Blind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts: Bronars, Park, Kedia & Agrawal (MIT) stake out a proprioception-is-already-a-high-resolution-contact-sensor counter-position for a specific regime (sub-mm-clearance peg insertion, gear meshing, nut threading): blind policies with only joint position + velocity + recent-action history — no F/T, no tactile, no vision — reach 73% zero-shot real-robot peg insertion across 8 pegs (incl. unseen tighter 0.5 mm clearance), with matched visual-policy baselines generalizing worse to unseen part shapes/sizes. Direct narrow challenge to the Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level Malik sensorimotor framing and to the tactile-rich recipes (Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction T-Rex, VTAP Gripper: Synergizing Fingertip Sensing and a Visuo-Tactile Active Palm for Dexterous In-Hand Manipulation VTAP, FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT, ReForce — force-aware retargeting via closed-loop residual force tracker ReForce) — the claim is not that tactile is unnecessary in general, but that for this task class joint sensing is the sensorimotor channel, legged-robot-locomotion-style. VTAP’s 7/10 at 14 mm peg / 1 mm tolerance with vision + tactile + active palm vs this paper’s 73% across 8 pegs (incl. 0.5 mm) with nothing is the sharpest filed head-to-head on the “is proprioception enough for tight-clearance assembly” question. Also ships mjfit, an MIT-licensed MuJoCo asset generator that keeps tight clearances intact — complementary tooling to Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks Tactile Genesis’s 8-sensor-abstraction simulator but on the clearance-geometry rather than tactile-sensor axis.
• [2026-10-13] InternW0: A Foundational Physical World Model for Efficient Real-World Interactions: InternW0’s contact-aware post-training stage augments the action prediction target from a_t to [a_t; f_t; τ_t] — action chunks plus 6-D interaction wrenches (3 forces + 3 moments) plus tactile representations — denoised jointly by the action expert. Only pre-chunk contact history is used as conditioning; future wrench/tactile channels are noised and denoised alongside action channels, so the model learns to predict interaction-force consequences of its own actions rather than conditioning on measured force at the moment of acting. Distinct from the policy-architecture side of this concept (Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction T-Rex variable-rate MoT, FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT per-layer force modulation, ReForce — force-aware retargeting via closed-loop residual force tracker ReForce residual force tracker): InternW0 integrates force at the WAM post-training layer, keeping the heterogeneous-pretraining stage force-free and only augmenting the action-side interface for contact-rich downstream tasks (e.g., the 5-stage dexterous quantitative pipetting workflow). Does not disclose sensor abstraction, hand hardware, or quantitative force-tracking numbers in the fetched excerpt.
• [2026-10-13] The Will to Act — Natura-1: Natura-1 (Nature Will) is a vision-and-language-only dexterous-hand foundation model at million-hour scale — the blog explicitly lists only language and vision as model inputs, with no mention of tactile or force sensing in the dexterous-hand deployment. Yet it reports emergent contact-sensitive behaviors (compliant calligraphy brushstrokes, deformable fabric folding, multi-stage tool switches, mid-air wrist rotation to steady toppling stacks) typically used as evidence that tactile integration is required (cf. Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction T-Rex, T-Rex: Tactile-Reactive Dexterous Manipulation, FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT). Counter-datapoint to the sensorimotor counter-position (Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level) in the specific narrow sense that vision-only pretraining at 1M-hour scale reaches at least qualitative contact-rich manipulation; strength of the evidence is limited by (a) six of the long-horizon demos use additional task-specific tuning and (b) no quantitative force-tracking / contact-establishment numbers are reported, so the direct force-aware comparison against TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning TouchScale’s 22.5% → 57.5% lift isn’t possible.
• [2026-10-12] TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning: TouchScale (Li et al., 500 h bimanual human-tactile corpus; MIT; code + HF dataset) directly fills the “no large-scale tactile dataset at consistent sensing” gap this concept page has been naming. Single unified wearable rig (head-mounted RGB-D + two wrist RGB at 30 Hz + bimanual tactile gloves with 880 taxels/hand at sub-2 mm) across ~2K task descriptions. Holding sensor and protocol fixed and varying only training data, reports zero-shot contact IoU on an unseen sensor rising 0.134 → 0.383, highest action recognition on three benchmarks after visual-encoder pretraining, and real-robot contact-rich manipulation (xArm6 + BrainCo Revo 2, 50 demos/task, 20 trials) lifting from 22.5% → 57.5% average under tactile mid-training with a future-tactile-change prediction objective. Hardware operationalization of Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks Tactile Genesis’s whole-hand-coverage directive on the dataset side, and the at-scale corpus answer to Neocambrian AI launches Mitts — tactile gloves for high-precision robot touch data Mitts (hardware-only) and sibling to ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine ACE-Data-0 (smaller scale, mocap + exo + ego + tactile). Head-to-head vs Being-H0.8: A Latent Tactile World-Action Model at Scale Being-H0.8’s infer-touch-from-vision-at-500k h bet is the obvious next experiment.
• [2026-10-10] Neocambrian AI launches Mitts — tactile gloves for high-precision robot touch data: Neocambrian AI launches Mitts — wearable tactile gloves with 450+ sensing points across the palm and fingers, 5 g minimum detectable force, 30 Hz sampling, synced with iPhone and RoboCap capture. Hardware operationalization of Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks Tactile Genesis’s placement-over-fingertip-only directive on the data-collection substrate side (vs the simulator or policy side); sibling to ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine ACE-Data-0’s full-palm tactile grids and to Sharpa releases the Origami Dataset for the IROS 2026 Robotic Origami Challenge Sharpa Origami Dataset’s 30-FPS cross-modal sync, but worn by a human demonstrator in uncontrolled industrial environments rather than a robot. Announcement surface only — no sensor abstraction (per-taxel F/T vs binary contact vs pressure heatmap vs elastomer displacement) disclosed, no corpus release, no policy or benchmark numbers.
• [2026-10-03] FINGR: Learning Dexterous Hand Control for Real-World Rubik's Cube Solving: FINGR (Liang, Peng, Kim, Wang) stakes out a no-tactile-sensor position on the sensor-abstraction axis for a sustained contact-rich task: solves 2×2×2 Rubik’s cubes single-handed via finger-relative point-cloud geometry plus auxiliary prediction of future contact-force changes, layer-turn progress, and finger-joint displacement at multiple time scales. 99.0% success on 300 real turn attempts vs 79.7% base flow policy; all ten scrambled cubes solved in ~137 s mean. Sibling to Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction T-Rex and T-Rex: Tactile-Reactive Dexterous Manipulation on per-finger observation decomposition, but gets the contact-aware shaping from predicting future force rather than ingesting current tactile signals — the first filed geometry-only + future-force-supervision instance on the concept.
• [2026-10-02] New Hands for Atlas | Boston Dynamics: Boston Dynamics unveils a redesigned 13-DoF four-finger hand for the electric Atlas humanoid (up from 7 DoF, pinky removed after self-ablation), explicitly engineered for high-fidelity simulation and sim-to-real RL; direct-drive backdrivable actuators plus dense tactile sensors on both fingertips and palm make it the first filed industrial-humanoid hardware instance of Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks Tactile Genesis’s “placement dominates type / whole-hand coverage beats fingertip-only” design directive. Companion position to ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning ADEPT on the sim-first-tactile-RL recipe, with a distinct hardware trade from the five-finger backdrivable set (Unitree Dex5-S — 22-DoF 1:1 anthropomorphic dexterous hand with backdrivable joints, from $6.5K Dex5-S, Solving Dexterity: A Full-Stack Approach (mimic hand M1 + wearable U1) mimic M1, ORCA: An Open-Source, Reliable, Cost-Effective, Anthropomorphic Robotic Hand for Uninterrupted Dexterous Task Learning ORCA) — lowest DoF count, only direct-drive-in-hand design, with pinky dropped to free actuator budget. Video has no transcript; design rationale sourced from BD’s accompanying technical post.
• [2026-09-30] DemoMimic: One Demonstration, Many Objects — Generalizing Manipulation via Local Contact Geometry: DemoMimic (Stanford; Sharma, Sahoo, Huang, Fei-Fei Li, Wu, Sadigh, Bohg) is the “geometry-only, no tactile sensor” endpoint of this concept — instead of ingesting or synthesizing tactile signals at runtime, it substitutes ground-truth sim-side contact normals into an Alignment Reward that aligns each contacting hand-link normal with the object normal at the contact point, and adds a Sustained Contact Reward that quadratically bonuses maintained contact streaks. The load-bearing claim is that the sim-to-real gap on dexterous manipulation comes largely from contact being poorly-conditioned and intermittent under standard RL rewards — fixing contact quality inside sim closes the gap even without any tactile sensor on the real robot. 71% average real success across 16 objects × 4 tasks × 2 hand embodiments; the smallest sim-to-real gap among the compared baselines and reward ablations. Complementary to FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation FELT (which synthesizes tactile from vision) and to CHORD: Object-Centric Contact Wrench Guidance for Dexterous Manipulation CHORD (contact-wrench interface) — DemoMimic’s contribution is that a purely geometric contact-quality reward is sufficient when local contact geometry is preserved between demonstration and deployment objects.
• [2026-09-28] Chris Paxton — Sharpa robot at IROS catches randomly falling foam cylinders with faster reaction time than the tweeter: Chris Paxton posts an IROS 2026 exhibit-floor video of a new Sharpa robot catching randomly-dropped foam cylinders in real time — a public-demo bookmark for a reactive-catching regime not covered by the paper-folding (Sharpa releases the Origami Dataset for the IROS 2026 Robotic Origami Challenge) or in-hand-rotation (ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning) Sharpa instances already on the wiki. No technical disclosure yet; file exists to hook the eventual write-up.
• [2026-09-23] TacBPM: A Tactile-conditioned Behavior Prior Model for Dexterous Reorientation: TacBPM (arxiv, authors unknown from abstract) proposes a new integration pattern on the concept’s “how should touch enter the policy” axis: distill multi-scale sphere specialists into a tactile-conditioned latent behavior prior, then let downstream task policies produce residual latent actions on top of the frozen prior rather than exploring in raw joint space. Distinct from FACT (per-layer force modulation), ReForce (residual joint commands atop a reference), and T-Rex (variable-rate multi-expert fusion) — the tactile channel enters through a distilled prior and adaptation happens in the prior’s latent space, architecturally closer to Proxy Policy Steering’s frozen-base recipe. Reports accelerated training and stable policies where matched raw-action PPO fails across arbitrary-pose transfer on anisotropic objects, commanded-axis rotation, and an arm-hand Grasp-to-AnyPose task, with sim-to-real transfer in both in-hand and arm-hand settings. Abstract does not disclose the tactile-sensor abstraction, simulator, hand hardware, or absolute success rates — full PDF needed to place this next to ADEPT / T-Rex / Tactile Genesis quantitatively.
• [2026-09-18] ORCA: An Open-Source, Reliable, Cost-Effective, Anthropomorphic Robotic Hand for Uninterrupted Dexterous Task Learning: ORCA (Christoph et al., ETH Zurich; IROS 2025) integrates binary FSR (RP-C7.6-ST) tactile sensors on all five fingertips of a 17-DoF anthropomorphic hand — sensitive to ~0.7 N (below the 1 N rated minimum) via silicone-skin preloading, but degrades non-uniformly: silicone wear raises AT to 1.7 N by 2,000–4,000 cycles on some fingers, and the 0.2 mm copper signal wires snap at the MCP/ABD boundary by 4,500–7,000 cycles across three fingers. Real-hardware corroboration of the Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks Tactile Genesis finding that fingertip-only placement is the sparsest workable position, and a concrete failure-mode datapoint for the “wire routing through actuated joints” problem that no simulator on the wiki currently models.
• [2026-09-06] How we actually test a robot hand: RLWRLD’s How we actually test a robot hand + the All Hands Up! launch is the empirical-hardware companion to Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks‘s simulator-driven placement-over-type finding: 10+ commercial dexterous hands physically procured, disassembled, and run through the internal DexBench simulator, with per-hand URDFs + specs + 11-level Kapandji thumb-opposition results published in an in-browser archive. Argues from the vendor-audit side that “hardware options don’t account for actual actions” — datasheet specs don’t predict which hands survive real-workplace skills. No head-to-head DexBench success rates disclosed publicly; the public artifact is the URDF+spec+Kapandji comparison and the framing that “no perfect robotic hand exists yet”.
• [2026-09-05] Universal Manipulation Exoskeleton: Learning Compliant Whole-body Policies with Real-time Torque Feedback: UME (Liang et al., Inclusion AI + Stanford BDML) attacks the force-integration problem at the data-collection layer rather than the policy layer: a low-cost 7-DoF upper-limb exoskeleton with real-time transparent haptic torque feedback that also records whole-arm joint torques during teleop, so downstream ACT policies can learn active compliance from data that includes the torque modality. Sibling to FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT (training-side per-layer force modulation), ReForce — force-aware retargeting via closed-loop residual force tracker ReForce (sim-trained residual force tracker), and Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction Tactile-Rex (policy-side variable-rate MoT) on the same force-in-the-loop axis but distinct in staking the position on collection-substrate design + transparent haptic loop rather than on model architecture or auxiliary loss. Ablations on four tasks (20 trials each) sharpen the “torque channel is load-bearing” claim: box pushing 0.90 UME vs 0.50 no-torque vs 0.40 UMI-style end-effector-only; box flipping 0.85 vs 0.00 vs 0.00 (No-torque and UMI cannot distinguish “insufficient pressing force” from “sufficient friction to flip” — visually identical, only distinguishable via joint torque); GPU picking in a 5 mm-clearance corridor 0.95 vs — vs 0.00 (UMI systematically bends and collides); fridge drink retrieval 0.95 vs 0.90 vs 0. Torque feedback also delivers 3.3× throughput improvement on force-mediated demonstrations, reaching 71% of unassisted-human speed. Coaxial ball-joint actuator arrangement + quasi-direct-drive Damiao/Unitree motors + per-joint Coriolis/friction/gravity compensation is what makes the transparent haptic loop feasible at low cost — the mechanical counter to the ALOHA/GELLO high-ratio-planetary-gearbox approach that cannot backdrive.
• [2026-09-04] ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine: ACE-Data-0 provides full-palm tactile-glove pressure grids on human hands, synchronized to millisecond residuals with 8-view exo video, egocentric fisheye, 41-joint mocap, and articulated hand pose — a unified-with-vision-and-motion tactile stream at a single lab, complementary to Open-X-Tactile — community effort to build the world's largest heterogeneous tactile manipulation dataset Open-X-Tactile’s heterogeneous pooled corpus across many labs.
• [2026-09-03] Getting a Grip on Robotic Data Collection — Co-Designed Koala Gripper for Force-Mediated Manipulation: RAI Koala pushes the “force is a distinct modality from vision” argument upstream into the collection device rather than the policy or simulator: fingertip force sensors mirror 1:1 between the handheld and robot variants of a co-designed 3-DoF gripper, and the post makes the modality-not-resolution case explicitly (a hand resting on vs squeezing a bottle cap produces near-identical images regardless of camera resolution). Complements Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks‘s per-taxel-F/T default and FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT / ReForce — force-aware retargeting via closed-loop residual force tracker ReForce’s force-integration recipes with a collection-side position: put force in the training data via the device itself. No sensor abstraction, taxel counts, or head-to-head reported.
• [2026-09-02] TwinDEX: A Scalable Path from Robot-Free Data Collection to Real-World Dexterous Manipulation: TwinDEX (X Square Robot) mirrors integrated fingertip sensors 1:1 across the wearable and robot three-finger nine-DoF end-effectors, so demonstrator tactile observations match the robot’s tactile stream by construction — hardware-side answer to the “observation-space contract between tactile front-end and policy” open question, in the same lineage as mimic (Solving Dexterity: A Full-Stack Approach (mimic hand M1 + wearable U1)) but scoped to a three-finger dexterous configuration. Blog does not disclose the sensor abstraction (F/T vs per-taxel vs contact-only), the tactile encoder, or the fusion recipe; announcement surface only.
• [2026-08-27] Project SuperDex — A unified simulation platform for dexterous manipulation: Project SuperDex (Meta) advertises tactile sensors as a first-class feature of its unified multi-physics engine (alongside soft/rigid articulations, non-convex collision, rods/tendons, shells/cloth) — a Meta-scale counter-position to Tactile Genesis’s eight-abstraction interface and Tacmap’s single-abstraction commitment. No sensor abstractions, taxel counts, or sim-to-real numbers disclosed on the landing page.
• [2026-08-21] T-Rex: Tactile-Reactive Dexterous Manipulation: T-Rex (Berkeley + NVIDIA GEAR, full arxiv version of Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction) delivers the quantitative counter-example the concept needed: naive tactile addition regresses a strong VLA (π0.5 + tactile 6% vs π0.5 17% across 12 tasks) — tactile integration must be architecturally deliberate, not appended. The recipe is a variable-rate Mixture-of-Transformer-Experts with an async cascaded flow-matching split at τ=0.4 (slow visuomotor expert + fast tactile expert firing 4× per vision tick), a per-finger temporal VQ-VAE (K=64 codebook over 16-frame force windows) with EMA + magnitude-weighted loss to prevent no-contact collapse, and the three-stage EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data pretraining recipe extended with a 100 h motor-primitive-organized mid-training dataset. Reports 65% avg vs 35% EgoScale on 12 real-world contact-rich tasks (30 pt gap comparable to EgoScale’s own 54 pt pretraining gap), with training-stage ablation showing from-scratch 18 → +human-pretrain 34 → +tactile-mid-train 45 → +full 65. Answers the concept’s “observation-space contract between tactile front-end and VLA backbone” open question with per-finger VQ-VAE codes + instantaneous force projection + ResNet deformation-map features fused via a separate high-rate expert.
• [2026-08-20] ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning: ADEPT (NVIDIA + U. Michigan) is the first filed instance on the wiki of a fully sim-trained RL policy consuming five vision-based tactile sensors on a 29-DoF Flexiv-Sharpa hand and zero-shot transferring to real hardware — a concrete existence proof for the sim-first tactile-integration route the Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks / Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map tooling papers have been arguing for. Same recipe deploys separately on a tactile-free 23-DoF Kuka-Allegro (two RGB cameras only), so the tactile channel is architecturally optional rather than load-bearing to the recipe.
• [2026-08-17] ReForce — force-aware retargeting via closed-loop residual force tracker: ReForce (@wu876768 tweet) is a policy-composition counterpart to FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks FACT on the force-integration axis: rather than modulating a VLA per-layer by current force, ReForce trains a closed-loop residual force tracker in simulation whose input is a motion+force reference plus live tactile state, and adds its residual joint command atop the reference at deployment. Reports 30%→70% force-safe success and 7/10→0/10 severe missing-contact failures on paper-cup grasping with a learned ACT reference. Answers the concept’s “how should the force channel enter the policy” question with residual-atop-reference rather than input-concatenation or per-layer modulation. No sensor abstraction, simulator, or robot spec disclosed in the thread.
• [2026-08-04] FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks: FACT (Parés-Morlans et al., tweet thread) argues force integration in current VLAs fails not because force sensing is absent but because force is appended as an input rather than modulating the policy — proposes per-layer modulation by current force, 2 s of force history as tokens, and contact-gated learning; reports 66% vs 41% baseline across 5 contact-rich tasks and ~2,500 real rollouts. Sits between taxel-level tactile (VTAP Gripper: Synergizing Fingertip Sensing and a Visuo-Tactile Active Palm for Dexterous In-Hand Manipulation) and F/T-only recipes on the sensor-abstraction axis this page tracks, on the F/T end with per-layer modulation rather than input-concatenation as the integration pattern.
• [2026-08-03] Aero Hand Open: low-cost, open-source, lightweight, anthropomorphic dexterous hand: Aero Hand Open takes the sparsest position on the sensor-placement axis: no fingertip tactile arrays at all, using inline tendon-force sensing as the contact/load proxy — a hardware data point complementary to Tactile Genesis’s placement-dominates-type finding.
• [2026-08-01] Open-X-Tactile — community effort to build the world's largest heterogeneous tactile manipulation dataset: Open-X-Tactile (Michael Yuan, Tsinghua IIIS with Yang Gao) is the first filed community-coordination attempt to fill the “large-scale tactile dataset” gap that Data Pyramid for Embodied Manipulation named as open challenge #1 — 50+ institutions and companies (UC Berkeley, MIT, CMU, Tsinghua, ETH Zürich, NUS, Sharpa, Wuji, Paxini, Xense) signed on, with a unified data format and benchmark ecosystem accepting contributions of any size / sensor / embodiment / task, explicitly framed as the tactile analog of Open X-Embodiment; downstream question is which sensor-abstraction bet the format spec picks (single-abstraction à la Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map vs 8-abstraction superset à la Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks).
• [2026-07-30] Sharpa releases the Origami Dataset for the IROS 2026 Robotic Origami Challenge: Sharpa’s Origami Dataset release fills the “large-scale tactile dataset” gap named as open challenge #1 in Data Pyramid for Embodied Manipulation with 682 episodes / 4.76M frames of bimanual paper folding on two 22-DoF SharpaWave hands, 10 fingertips × 6-axis F/T synchronized frame-by-frame with 6 camera views at 30 FPS, exported in both raw and deform-map tactile formats (matching Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map‘s abstraction) with full LeRobot 3.0 coverage; the paper-folding task also stress-tests the low-force / deformable-object claims of Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction and stakes a fingertip-only sensor placement counter to Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks‘s coverage-over-fingertips finding.
• [2026-07-29] Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map: Tacmap (Su, Peng, Ren, Mao, Du, Zhang, Zhu; Sharpa + HKUST + NVIDIA) is the sim-to-real alignment half of the tactile stack — commits to a single tactile abstraction, geometry-consistent penetration-depth (“deform map”), computed by GPU ray-casting in Isaac Lab / MuJoCo and recovered from real tactile images by a ResNet inversion trained against ground-truth depth measured by an automated indenter rig. Reports sub-mm contact position error, ~0.5 N net-force error, >85% deform-map IoU on square+cylinder indenters, and a zero-shot sim-to-real PPO in-hand rotation policy on the SharpaWave hand. Complements Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks (unifies 8 abstractions, argues per-taxel force/torque as default) by taking the opposite bet — commit to one abstraction and monopolize the alignment story around it. Answers the concept’s “observation-space contract between tactile front-end and policy” open question with a specific proposal: normal-projection penetration-depth pixel field. Explicitly punts on tangential shear (§VI limitations) — the HydroShear: Non-Holonomic Hydroelastic Tactile Simulator / HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning axis.
• [2026-07-29] HydroShear: Non-Holonomic Hydroelastic Tactile Simulator: HydroShear (project page) is the upstream reference for the shear-simulation baselines Tactile Genesis benchmarks against and whose elastomer model Tactile Genesis extends. Non-holonomic hydroelastic simulator that models stick-slip, path-dependent force/shear build-up, and full SE(3) object-sensor interactions via SDF-tracked on-surface indenter displacements. Reports 93% avg zero-shot sim-to-real RL success across peg insertion / bin packing / book shelving / drawer pulling — vs 34% for tactile-image-trained baselines and 58-61% for alternative shear simulators. Concrete quantitative case that shear-fidelity, not tactile-image rendering, is the load-bearing piece for contact-rich RL transfer. Closes a citation gap in the concept page since Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks‘s §3.1 elastomer model builds on it directly.
• [2026-07-28] Data Pyramid for Embodied Manipulation: Data Pyramid names large-scale tactile datasets as the first of six open challenges for embodied-manipulation data recipes — external corroboration that the concept-page thesis (tactile is the load-bearing missing modality for dexterous manipulation) is now the consensus survey position, not a fringe critique. Complements the three existing filed instances (Tactile Genesis simulator, Tactile-Reactive Dexterous Hand policy, PressureVision++ vision-only substitute) with the recognition that the dataset layer is what still doesn’t exist at pretraining scale.
• [2026-07-28] Being-H0.8: A Latent Tactile World-Action Model at Scale: Being-H0.8 (BeingBeyond) is the first filed foundation-model-scale instance of “infer touch from vision” as the primary tactile channel — TactoHand converts 500,000+ hours of egocentric human video into dense pseudo-tactile supervision (binary contact + continuous proximity on canonical MANO) via geometry-supervised prediction, and a universal tactile encoder (query-based Perceiver over a coarse-to-fine pyramid) unifies these pseudo-labels with measured taxel/pressure signals under one interface. Extends the FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation / PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images vision-only-tactile recipe from per-task policy training to embodied pretraining, and answers the “observation-space contract between tactile front-end and VLA backbone” open question with a Perceiver-resampled fixed-token interface plus a learned missing-touch representation for tactile-absent samples.
• [2026-07-25] FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation: FELT (Li, Ling, Gu, Huang, Liang, Islam, Bedri, Chirikjian, Li, Nikolaidis, Seita) extends the vision-only-tactile idea from PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images to robot grippers and — critically — validates it inside a manipulation policy: a frozen vision encoder plus a lightweight dual-branch query decoder (one branch per finger, respecting sensor topology) synthesizes per-finger pressure tactile images from RGB in a feed-forward pass. Both generated tactile images and latent tactile features improve success over vision-only baselines across four contact-rich tasks, with the latent-feature variant needing no real tactile sensor at either training or deployment — a purely software tactile channel. Shared authors (Binghao Huang, Yunzhu Li) with VTAP Gripper: Synergizing Fingertip Sensing and a Visuo-Tactile Active Palm for Dexterous In-Hand Manipulation make this the third leg of the same group’s tactile-manipulation triangulation (hardware, sim, vision-to-tactile).
• [2026-07-23] VTAP Gripper: Synergizing Fingertip Sensing and a Visuo-Tactile Active Palm for Dexterous In-Hand Manipulation: VTAP Gripper (Zhou, Athar, Hu, Huang, Li, Wachs, She; IROS 2026) is the first filed hardware instance operationalizing Tactile Genesis’s placement-over-type finding — an actuated visuo-tactile active palm that switches between long-range camera view (pre-contact) and contact-rich tactile sensing (post-occlusion), paired with FlexiTac tactile-array fingers. Demonstrates 93.3% tactile-reactive grasping on YCB+fragile, autonomous vision-then-tactile peg-in-hole (14 mm peg, 15 mm hole, 1 mm tolerance, 7/10), and 3-mm-object in-hand singulation on a three-finger (not anthropomorphic) gripper. Shares author Yunzhu Li with Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction and stakes a low-DoF-gripper position against the anthropomorphic route.
• [2026-07-16] Solving Dexterity: A Full-Stack Approach (mimic hand M1 + wearable U1): mimic hand M1 pursues a hardware recipe orthogonal to Tactile Genesis’s placement-dominates-type finding: a dual motor+joint encoder scheme derives ≤0.1 N unidirectional contact-force estimates from motor current on every joint (effectively whole-hand implicit force sensing), then adds dedicated fingertip tactile sensors for tangential shear and higher spatial resolution only at the distal contact points. The argument is that <0.05 Nm joint backdrivability lets every actuator double as a force sensor, sidestepping the taxel-placement problem — but no head-to-head with dedicated palm / proximal-phalanx taxels is offered. Complements Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks‘s simulator-driven placement study by testing the alternative hypothesis that backdrivable actuation obviates whole-hand taxel coverage.
• [2026-07-14] Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks: GPU-parallel tactile simulator integrated into Genesis World with 8 sensor abstractions (contact / depth / kinematic force-torque / elastomer / proximity / temperature / audio) under a common interface, 20,000+ parallel envs and 1,000+ taxels on a single GPU (3-20× throughput over TacSL / Tacmap / HydroShear / FOTS). Ablations across three dexterous tasks × three placements × three resolutions × clean/noisy noise conditions yield the concrete design directives: placement > type, per-taxel force/torque is the default, 200 whole-hand taxels suffice. Sim-to-real validation on the real XHand1 confirms the fingertip-aggregate abstraction transfers.
• [2026-06-20] Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction: Tactile-Reactive Dexterous Hand — first policy-side answer on the wiki. Variable-rate Mix-of-Transformer + temporal tactile VQ-VAE fuses kHz-class touch with slower vision/language, on top of an existing VLA. 100 h motor-primitive-prioritized teleop dataset; >30% absolute average success rate over the strongest baseline across 12 delicate-force / deformable-object tasks. Only a project page at filing time — no paper or code yet.
• [2026-06-18] PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images: PressureVision++ — the vision-only substitute. Weak “press finger X at force Y” contact labels + adversarial domain alignment supervise a per-pixel pressure predictor from a single RGB image; 41.9% contact IoU / 27.5% volumetric IoU on textured overlays, 80.5% in-the-wild contact accuracy matching human annotators. 2.9M-frame ContactLabelDB with 51 participants × 106 surfaces released.

Open questions

• Does the policy-side answer converge on variable-rate multimodal fusion (Tactile-Rex’s MoT) or on a unified token stream with per-modality position encodings? No filed paper compares the two.
• Which downstream tasks actually benefit from per-taxel force/torque vs. per-link aggregate force? Tactile Genesis shows near-continuous-contact tasks (in_hand_repose) reward per-taxel, but the real-XHand1 hardware only exposes aggregate — so the win is real-world-bounded by what hardware exists.
• Is vision-only contact inference (PressureVision++) sufficient as the VLA’s tactile channel, or is the latency and occlusion sensitivity a hard blocker that keeps first-class touch sensors on the critical path? No filed head-to-head.
• Does the “coverage over resolution” finding from Tactile Genesis generalize to bimanual manipulation and to tasks that involve tool-mediated contact (e.g. screwdriver, chopsticks) where the informative contact point sits outside the hand? Screwdriver-task results in the paper hint the missing channel there may be temporal integration, not spatial density.
• Elastomer marker displacement is a first-class output of many vision-based tactile sensors (GelSight, DIGIT). If it underperforms force/torque per-taxel on the dexterous tasks Tactile Genesis tested, does that mean the current elastomer-simulator development effort is targeting the wrong downstream metric, or that the tasks need to change?
• What is the right observation-space contract between a tactile front-end and a VLA backbone? Tactile-Rex’s temporal VQ-VAE yields discrete codes; PressureVision++ yields per-pixel dense pressure maps; Tactile Genesis exposes per-taxel force vectors. None of these have been unified across policies.
• Does simulator-first tactile hardware design (i.e., using Tactile Genesis’s ablations to spec whole-hand coverage before building the hardware) actually shorten the hardware-iteration loop, or does the sim-to-real gap on non-fingertip surfaces re-open the loop anyway?

Papers

2026-06

• Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks — Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks (published 2026-06-21)
• Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction — Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction (published 2026-06)

2023-01

• PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images — PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images (published 2023-01-05)