TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation

TL;DR

TAM inserts a learned residual-torque module between the low-level controller and the robot’s torque interface, below both the policy and any BC/RL/MPC controller. A slow (~5 Hz) history encoder infers robot-side dynamics mismatch from 12 s of proprioceptive + applied-torque history, and a fast (1 kHz) torque adaptor emits per-tick residual torques that make the target robot match the ideal-model motion under the same nominal torque. Because TAM depends only on nominal torque + proprioceptive history — not on policy observations, rewards, action space, or reward — one set of weights reuses zero-shot across an RL push policy, a BC flip policy, and an MPC ball-on-plate controller on a real Franka Panda. Training is 100% in randomized simulation (multi-robot pretrain + per-robot fine-tune) with analytic inverse-dynamics labels; no real-robot data is required.

Key claims

Moving adaptation to the torque interface (below the policy AND the low-level controller) decouples adaptation from policy architecture — the same TAM weights work under joint-target, end-effector-target, and direct-torque action spaces [§2, §3.2].
Multi-robot pretraining (Panda, Piper, RBY-1, iiwa14, Flexiv Rizon4) followed by per-robot fine-tuning gives a single shared TAM initialization, no real-robot data required at either stage [§4].
On real Franka Panda vision-based box pushing (RL policy), TAM reaches 76.2% success vs Direct 42.9%, Online SysID 57.1%, RMA 52.4%; also lifts the RMA policy from 52.4% → 71.4% when composed [Table 1].
On a behavior-cloned flip policy (ACT), TAM reaches 72% success vs Direct 43% and Online SysID 34% — Online SysID performs worse than Direct because contact-rich short episodes give poor identification data [Table 1].
On MPC ball-on-plate, TAM completes 6.20 goals per run vs Direct 3.60 and Online SysID 4.80 (ideal-simulation baseline 6.75) [Table 1].
On tracking under an unknown 1 kg payload and a mid-trajectory switch from joint-impedance to OSC, carried-state TAM cuts joint tracking error more than Online SysID or a learned physical-parameter predictor; history collected under joint impedance further improves the later OSC segment [Table 2].
Simulation cross-robot fine-tuning gives the lowest mean tracking error (2.35°) vs multi-robot pretrain (3.14°) and Online SysID (5.29°), with the multi-robot weights already reducing error on two held-out test robots (Google Robot, Unitree Z1) [Table 3].

Method

The setup assumes an ideal robot (known deterministic dynamics used to train/design the policy) and a target robot whose dynamics differ due to sim-to-real gap, unknown payload, or instance-to-instance variation. A policy π emits an action, a low-level controller converts it to a nominal torque τ, and TAM emits a residual Δτ so that the corrected torque τ + Δτ makes the target robot’s local acceleration match the ideal robot’s under τ.

TAM has two runtime components: a history encoder that ingests a long (~12 s) rolling buffer of (measured position, measured velocity, applied torque), augmented with a physics residual feature — the mismatch between applied torque and the torque the ideal model would have needed for the observed motion — and processes it via patch-wise time-series tokenization + a causal Transformer to produce a compact latent state z; and a torque adaptor that at every control tick reads a short local window plus the current nominal torque and the per-joint z (injected via adaptive layer norm) and emits Δτ.

Training uses analytic labels: for each perturbed simulator rollout, the target corrected torque is what the perturbed model would need to reproduce the ideal model’s acceleration under the queried nominal torque. Loss is a per-joint Huber penalty. Task-agnostic randomized joint-motion rollouts (no task demos) supply the training data; a two-stage pipeline pretrains on 5 robot models then fine-tunes per target robot, all in simulation.

Results

Across three real-Franka tasks with the same TAM weights: box pushing 76.2% (Direct 42.9, SysID 57.1, RMA 52.4), flip 72% (Direct 43, SysID 34), MPC ball-on-plate 6.20 goals/run (Direct 3.60, SysID 4.80, ideal 6.75) [Table 1]. On real tracking with an unknown 1 kg payload after switching from joint impedance to OSC, TAM (carried) gives the lowest error across all four conditions [Table 2]. Simulation cross-robot: multi-robot TAM lowers tracking error on every robot including two held-out (Google Robot, Unitree Z1); target-robot sim fine-tuning gives mean 2.35° across robots vs SysID 5.29° [Table 3]. The paper explicitly notes RMA is policy-side — every new RL/BC/MPC policy would need retraining or distillation with an RMA branch — while TAM is drop-in below any policy.

Why it’s interesting

TAM stakes out a structurally distinct row on the sim-to-real recipe board from the ones tracked on VLA Models: rather than adapting the policy (RECAP-style RL, clean-teleop scaling, unified-VLM pointing) or wrapping it in an orchestrator (Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents Harness VLA, RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning RoboHarness), TAM leaves the policy untouched and adapts the torque interface below the low-level controller. It’s a sibling in framing to ReForce — force-aware retargeting via closed-loop residual force tracker ReForce (residual force tracker below ACT) but with a much sharper policy-agnosticism claim — the same weights drive RL, BC, and MPC controllers with no policy observations required — and matches Going Beyond World Models & VLAs Florence’s implicit position that adaptation and control layers should be factorable rather than baked into the policy.