World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation

TL;DR

W²-VLA is a VLA architecture for fine-grained robot manipulation that separates the roles of main-view and wrist-view observations: the VLM (Qwen-VL) contextualizes a small set of latent modeling tokens from multi-view input plus the language instruction, and a wrist predictor uses those tokens plus frozen V-JEPA wrist-history features to forecast future wrist latents, which then condition flow-matching action generation. A companion synthesis pipeline (W²-CoT) produces structured chain-of-thought supervision covering task progress, physical transition cues, and wrist-local evidence, used only at training time to shape the latent interface. The model reports gains on LIBERO, RoboTwin 2.0, and real single-arm/bimanual tasks while running above 80 Hz.

Key claims

Wrist-view and main-view observations play asymmetric roles: main-view carries global task context while wrist-view carries contact-proximal dynamics, and treating them as parallel VLM inputs underuses the wrist signal [§Abstract].
A fixed set of 16 task-conditioned latent modeling tokens inside Qwen-VL serves as a compact interface to a downstream wrist predictor, avoiding explicit natural-language chain-of-thought at deployment [Project page: Method].
The wrist predictor conditions on those latent tokens plus frozen V-JEPA features of the observed wrist history to forecast future wrist latents, which are converted into future-aware context for a flow-matching action head [Project page: Method].
The W²-CoT synthesis pipeline produces structured annotations describing manipulation progress, physical transition cues, and wrist-local evidence, used as auxiliary supervision on the latent interface [§Abstract].
W²-VLA improves fine-grained and contact-sensitive manipulation on LIBERO, RoboTwin 2.0, and real single-arm and bimanual tasks while sustaining >80 Hz action-generation [§Abstract].

Method

W²-VLA builds on a Qwen-VL backbone. Current main-view and wrist-view observations plus the language instruction are consumed by the VLM to contextualize 16 fixed-length latent modeling tokens — a compact task-conditioned interface. A wrist predictor takes these tokens and cross-attends over frozen V-JEPA features of the recent wrist-camera history to forecast a set of future wrist latents at a horizon relevant to the task. A lightweight adapter fuses the predicted future wrist context with the VLM hidden states, and a flow-matching action head produces the continuous action chunk. During training, the W²-CoT synthesis pipeline produces structured chain-of-thought annotations — task progress, physical transition cues, wrist-local evidence — that supervise the latent tokens via auxiliary losses; at deployment no natural-language reasoning is emitted, so the latency budget stays inside the action loop (>80 Hz).

Results

The paper reports improvements on LIBERO, RoboTwin 2.0, and real-world manipulation across both single-arm and bimanual settings, with the improvements concentrated on fine-grained and contact-sensitive tasks where wrist-local prediction is expected to help most; action generation runs above 80 Hz [§Abstract]. Precise per-benchmark numbers are not surfaced in the abstract or project page and require the PDF body.

Why it’s interesting

Two ideas converge here that the wiki has been tracking separately. First, W²-VLA is another instance of the world-action model family that predicts a compact latent future to condition action generation — but instead of a full pixel-space video world model (as in 1XWM: From Video to Action — 1X's video-pretrained world model as a NEO robot policy or Direct Video-Action Models — Causal Video Models Are Data-Efficient Robot Policy Learners), it predicts only wrist latents in a frozen V-JEPA space, arguing that action-proximal, wrist-local foresight is what actually pays off for contact-rich manipulation. Second, it uses distilled CoT annotations as training-time supervision on a fixed-length latent interface rather than as inference-time reasoning tokens — the same “compress CoT into latents” move that Robotic Control via Embodied Chain-of-Thought Reasoning and, more recently, HDR: Hierarchical Denoising for Multi-Step Visual Reasoning explore, but restricted to what the wrist needs to see next.