ModPack: An Extensible Teleoperation Interface for Bimanual Mobile Manipulation

TL;DR

ModPack is a modular, wearable teleoperation interface for bimanual mobile robots from Stanford’s REALab (Shuran Song group). A 3D-printed backpack houses an onboard PC + batteries + storage; robot-specific 3D-printed leader arms (GELLO-style joint-kinematic equivalents for ARX5-based and RB-Y1m platforms) plug into the shared spine, and capability modules — joint-level teleop with haptic feedback, mobile-manipulation control, active head-camera perception — attach through a standardized software interface. The hardware and software are fully open-sourced with a bill of materials. Beyond the collection substrate the paper demonstrates DiT policies trained on ModPack data reaching 22/25 (cloth placement, active perception) and 12/20 (box transfer, haptic feedback) on two robots, plus untrained open-ended tasks (cooking, recycling, elevator calling).

Key claims

A shared 3D-printed backpack (compute + power + storage + comms) plus embodiment-specific leader arms lets one teleop interface span multiple bimanual mobile robots, with capabilities added by plug-and-play modules rather than per-robot re-engineering [§Abstract, §Inside the Backpack].
Leader arms are constructed as kinematic equivalents of the follower per GELLO, enabling direct joint-space teleop via Dynamixel XM540 / XM430 motors sized to the minimum torque needed for gravity compensation [§Leader Arms].
On the cloth-placement-with-active-perception task, the head-camera-only DiT policy succeeds 22/25 rollouts vs 20/25 for all-camera and 3/25 for wrist-camera-only, indicating that broader egocentric view + depth dominates close-up wrist observations when the object admits many viable grasps [§Capability Experiments (a)].
On box-transfer-with-haptic-feedback, the all-cameras + joint-torque policy reaches 12/20 vs 11/20 for head-camera-only and 6/20 for all-cameras-without-torque; the paper attributes the gain to projected joint torques providing a coarse alignment signal for grasp timing and depth [§Capability Experiments (b)].
The attention-pattern analysis shows the torque-conditioned DiT attends more strongly to task-relevant image regions (box edges) than the vision-only policy, and per-arm torque-token attention rises during the contact-critical grasping phase — direct visualization of the policy leaning on torque information at contact [§Attention Pattern Analysis].
ModPack supports open-ended teleop in everyday environments: cooking in a kitchen, can recycling, and elevator calling are demonstrated across both robots as untrained tasks [§Teleoperation].

Method

The backpack shell is two 3D-printable panels (PLA, 15% gyroid infill, 0.2 mm layers, Bambu H2D / X1 Carbon), latched together with toggle clasps around five removable shelves that house a mini-PC, portable batteries, and optional accessories arranged along the operator’s spine for ergonomic load distribution. Leader arms are 3D-printed and kinematically equivalent to each target follower — 6-DoF for a customized ARX5-based bimanual platform, 7-DoF for the RB-Y1m — enabling direct joint-space mapping in the GELLO tradition. Capability modules pair dedicated hardware with a standardized software interface: joint-level teleop augmented with haptic feedback (motor torque commands to the leader arm mirror the follower’s contact forces), active perception (motorized head-camera control), and mobile manipulation (bimanual arm + wheeled base coordination). The Diffusion Transformer (DiT) policies trained on ModPack-collected demonstrations conditionally consume subsets of the available inputs (wrist / head / all cameras, optionally with joint torques) to isolate which sensing modality drives which task.

Results

Cloth placement with active perception (ARX5-based bimanual): head-camera-only 22/25, all-cameras 20/25, wrist-camera-only 3/25 [§Capability Experiments (a)]. Wrist-only fails mostly on basket-placement errors from lack of scene context.
Box transfer with haptic feedback (bimanual + rack, top or bottom shelf): all-cameras + torque 12/20 (60%), head-camera-only 11/20 (55%), all-cameras 6/20 (30%) [§Capability Experiments (b)]. Common all-cameras failure mode is bad grasp pose; the paper’s hypothesis is that torque provides an alignment signal absent from vision alone.
Real-time attention analysis: torque-conditioned policies attend to box edges (grasp-informative); torque-token attention weights spike as the robot enters contact-critical phases [§Attention Pattern Analysis].
Open-ended teleop (untrained tasks): cooking, recycling, elevator calling — demonstrated on both robots without policy training [§Teleoperation].

Why it’s interesting

ModPack stakes out a distinct position on the Human-to-Robot Retargeting concept page’s “where does the embodiment-invariant interface live” axis: not at the model (interleaved action tokens, reduced-DoF bridging), not at collection-tool geometry (mimic U1’s kinematic lock, XR-1’s UMI grippers, HandUMI — Robot-free bimanual data collection retargetable to any parallel-gripper arm (Brik H. Meza launch tweet)‘s HandUMI), but at a shared operator substrate — one backpack + swappable leader arms per follower + swappable capability modules. This is the classical GELLO / teleop-rig approach modernized as an open, extensible platform and validated on two distinct bimanual mobile robots, complementing RAIDEN: A Toolkit for Policy Learning with YAM Bimanual Robot Arms (TRI’s RAIDEN, which is a software-toolkit take on the same open-teleop-substrate need) with a hardware-first Apache-style release. The DiT policy results are also a datapoint for VLA Models‘s tactile-integration open question: joint-torque as a lightweight proxy for tactile / contact signal delivered measurable gains (60% vs 30% at matched vision) and interpretable attention structure — cheaper than a dedicated tactile stack (FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation FELT, Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction Tactile-Rex) but at coarser resolution.