CW-WAM: A Structured Contact-Wrench World Action Model for Force-Sensitive Manipulation

TL;DR

CW-WAM is a world-action model that jointly predicts video, motion, and contact wrench, aimed squarely at force-sensitive manipulation (peg insertion, adapter seating, wiping, variable-payload transfer). A unified interaction module treats motion and contact wrench as co-equal modeling targets and emits time-aligned motion–wrench chunks as control references, with pair-causal attention structuring the dependency and a long fixed-length wrench memory preserving sustained interaction history. The predicted wrench chunk acts as a force reference for a 20 Hz outer-loop admittance controller inside a slow–fast control system, which maintains force feedback and smooth motion between model queries. On four contact-rich tasks the system raises success rates substantially (e.g. 25% → 70% for 1 mm-clearance square peg insertion, 40% → 95% for power-adapter seating) and halves force-reference RMSE on peg insertion and wiping.

Key claims

Vision-only observations do not expose the contact and load information needed for force-sensitive manipulation, motivating a WAM that jointly predicts video, motion, and contact wrench [§Abstract].
The unified interaction module treats both motion and contact wrench as modeling targets and predicts time-aligned motion–wrench chunks as control references [§Abstract].
Pair-causal attention links each future wrench to its paired action and earlier pairs, pairing past motion commands with the measured wrenches from the same interval [§Interaction module].
A long fixed-length wrench memory retains past interaction information, specifically preceding payload tension or sustained resistance [§Interaction module].
The predicted wrench chunk is consumed as a force reference by a 20 Hz outer-loop admittance controller that computes pose corrections, which are then interpolated into smooth trajectories for a 1 kHz robot controller [§Control system].
Square peg insertion (38 mm peg, 40 mm hole, 1 mm clearance per side, ~50 mm depth): success rises from 25% to 70% vs a WAM without wrench input [§Results].
Power-adapter insertion: success rises from 40% to 95% vs the no-wrench baseline [§Results].
Lifting-height accuracy improves across different loads under the wrench-conditioned model [§Results].
The framework with the admittance controller reduces force-reference RMSE from 4.8 N → 2.7 N on peg insertion and from 2.0 N → 1.3 N on wiping, with smaller force fluctuations during successful execution [§Results].

Method

CW-WAM is a WAM structured around a contact wrench channel in addition to the video and motion channels that WAM-family models already predict. Inside the unified interaction module, motion commands and measured 6-D wrenches (3 forces + 3 moments) from the same time interval are paired at the token level, and pair-causal attention wires each predicted future wrench to its paired action and to earlier pairs so wrench predictions are consistent with the action chunk that would produce them. A long fixed-length wrench memory keeps a rolling window of past wrench observations so phenomena like sustained payload tension or ongoing contact resistance remain accessible even when they fall outside the visual attention window.

At deployment the model emits a joint motion–wrench chunk. The wrench chunk becomes the reference for a 20 Hz admittance controller that computes compliant pose corrections; the corrected waypoints are then interpolated down to a 1 kHz low-level robot controller. This slow–fast factorization — WAM at chunk rate, admittance at 20 Hz, controller at 1 kHz — maintains force feedback and smooth motion between model queries, so wrench tracking does not stall when the WAM is between inferences.

Results

Four task families are evaluated: square peg insertion (38 mm peg, 40 mm hole, 1 mm clearance per side, ~50 mm depth), power-adapter seating (grasp loose adapter, insert into socket), wiping (eraser removing marker from a surface under sustained contact), and spring-scale box transfer (identical-looking 30 g vs 830 g boxes, picked and transferred between bins). Reported headline numbers [§Results]:

Square peg insertion: 25% → 70% success vs WAM without wrench input (1 mm clearance per side).
Power adapter insertion: 40% → 95% success vs WAM without wrench input.
Lifting-height accuracy improved across different loads.
Peg insertion force-reference RMSE: 4.8 N → 2.7 N.
Wiping force-reference RMSE: 2.0 N → 1.3 N.
Smaller force fluctuations reported during successful execution.

Why it’s interesting

CW-WAM is the sharpest WAM-side filed instance to date of “put force in as a first-class joint prediction target plus a slow–fast controller”. It stakes a different cell on the force-integration axis than FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity FACT, which moves force upstream into the human pretraining corpus (EgoForce), and than FACT: Force-Aware Contact-Rich Manipulation via Timestep Modulation — demystifying when and why VLAs fail on contact-rich tasks which modulates a VLA per-layer by current force; CW-WAM instead keeps the WAM factorization and uses the predicted wrench to drive an outer-loop admittance controller — a direct realization of the Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level / Yunzhu Li sensorimotor counter-position on the WAM side rather than the VLA side. The sub-mm-clearance peg-insertion regime also puts the result on a shared evaluation axis with Blind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts‘s proprioception-only sim-to-real counter-recipe (73% at 0.5–1 mm clearance), inviting the obvious head-to-head: does joint wrench prediction + admittance control match or beat pure proprioception + heavy sim randomization on the same tight-clearance task class?