Skip to content
Bud's Wiki

Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control

TL;DR

Section titled “TL;DR”

Steerable Policies train a VLA (OpenVLA / Prismatic-7B backbone) to accept a spectrum of steering-command abstractions rather than just task-level language — task labels, subtasks, atomic motions, gripper traces (pixel sequences), points (object/target positions), and hybrids — so a high-level VLM can choose the abstraction that best fits the current situation. The authors extend Bridge from 38k task-level labels to ~2M synthetic steering commands via a Molmo + SAM2 + DETR + Gemini pipeline, then evaluate two hierarchical controllers: (a) a fine-tuned high-level embodied-reasoning VLM that emits CoT + steering command, and (b) an off-the-shelf API VLM that uses in-context learning over past frames and commands to pick the right abstraction on the fly. Both outperform ECoT / ECoT-Lite / plain OpenVLA on generalization axes, with the in-context-learning route unlocking a novel capability — reasoning about how to steer, not just what to do.

Key claims

Section titled “Key claims”
  • Steering commands span six styles — task-level, subtask, atomic motion, gripper trace (pixel list), points (object positions), and combinations — and are all expressed as text tokens so the VLA can interface with generative VLMs [§IV-A].
  • A synthetic-labeling pipeline (Molmo for object masks, SAM2 for temporal propagation, DETR for gripper traces, Gemini 2.0 for subtask decomposition and command paraphrase) expands Bridge from 38k task-level labels to 206k subtasks and ~2M total steering commands, replacing standard BC language uniformly at random during training [§IV-B].
  • No single command style dominates: a human-oracle constrained to one style at a time outperforms task-only prompting on every axis, but each style has complementary strengths — trace/point commands dominate in semantic generalization (novel objects), atomic motions dominate in spatial-relation tasks, task/subtask are most reliable on in-distribution portions [§VI-A, Fig. 4].
  • Given free choice of steering commands, a human oracle solves all evaluation tasks at ~100% success — an achievable upper bound showing the low-level policy is expressive enough given the right prompt [§VI-A].
  • Fine-tuning a VLM into a high-level embodied reasoner that emits CoT + steering commands outperforms ECoT and both ECoT-Lite variants on Bridge generalization tasks, while querying the reasoner less frequently than a per-step embodied-CoT VLA (faster inference without compile tricks) [§V-A, §VI-B].
  • Off-the-shelf VLMs can drive Steerable Policies zero-shot by using in-context learning over past observations + issued commands to pick the right abstraction — leveraging a functionality that non-steerable VLAs cannot expose because they accept only one prompt modality [§V-B, §VI-C].
  • The paper positions itself against MolmoAct as the closest prior work: MolmoAct allows only task-level language + gripper traces (two modalities, and traces cannot be issued without accompanying text), whereas Steerable Policies accept the full spectrum of abstractions independently [§II].

Method

Section titled “Method”

Steerable Policies extend a standard VLA (Prismatic-7B / OpenVLA architecture) along the prompt side rather than the action side. The training loss is unchanged next-token prediction on action tokens, but each frame’s text conditioning is drawn uniformly from a much larger set of synthetic commands rather than from the dataset’s single human-written task label.

The synthetic-command pipeline runs per trajectory: (1) programmatic motion extraction following Zawalski et al.; (2) Molmo → object names + segmentation masks for task-relevant objects; (3) SAM2 propagates masks across the trajectory to give temporally-consistent open-vocabulary bounding boxes; (4) DETR extracts gripper-pixel traces; (5) Gemini 2.0 uses task + motions + object list to decompose the episode into semantic subtasks; (6) Gemini restates each subtask in every command style — task, subtask, atomic motion, trace, point, combination — with the grounded features (traces, motions, centroids) supplied in-prompt. Additionally, for each subtask’s starting frame, Gemini writes a post-hoc rationale explaining why that subtask is needed for progress; these rationales become CoT supervision for the high-level embodied reasoner.

Two hierarchical control methods are evaluated on top of the same Steerable Policy:

  • Trained embodied reasoner (§V-A). A VLM is fine-tuned on (task, observation) → (reasoning, steering command) pairs from the pipeline. At inference it autoregressively predicts reasoning then a steering command, which the Steerable Policy executes for K environment steps before the reasoner is re-queried.
  • In-context-learning API VLM (§V-B). An off-the-shelf VLM (Gemini-class) receives the task, examples of every command style with strengths/weaknesses, and a rolling history of past observations + emitted commands. It parses the scene, decides what to do, then reasons about the abstraction level before emitting the next steering command. Because the in-context examples are the VLM’s own past commands rather than hand-crafted demonstrations, this reduces to standard vision-language in-context learning that off-the-shelf VLMs already excel at.

All experiments target the Bridge WidowX real-world setup, with generalization axes (in-distribution, motion, spatial, semantic) matching ECoT-Lite for direct comparability.

Results

Section titled “Results”
  • Human oracle with unrestricted steering commands: ~100% success across all tasks, upper-bound demonstration that the Steerable Policy is expressive enough [§VI-A].
  • Single-style human-oracle prompting still beats task-only prompting on every generalization axis; atomic motions are the best single style overall, trace/point wins semantic generalization, task/subtask wins in-distribution segments [§VI-A].
  • Fine-tuned high-level reasoner + Steerable Policy outperforms ECoT, both ECoT-Lite variants, and non-reasoning OpenVLA on the same Bridge generalization suite; a non-reasoning ablation of the high-level (VLM emits commands directly with no CoT) is included as a control [§VI-B].
  • Off-the-shelf VLM in-context-learning route yields significant performance gains over standard baselines, including on unseen longer-horizon tasks, without any robot-specific fine-tuning of the high-level VLM [§VI-C].
  • Inference speed: because the reasoner is queried less frequently than the low-level policy, wall-clock is faster than per-step embodied-CoT VLAs like ECoT without any compilation techniques [§V-A].

Why it’s interesting

Section titled “Why it’s interesting”

Steerable Policies sit squarely in the “wrap frozen/small VLA under a smarter VLM” thread on VLA Models and adds a distinct lever no filed paper has isolated: the low-level policy’s prompt interface is the variable, not the policy or the orchestrator. Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents (Harness VLA) and VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation (VoLo) keep the VLA as a narrow contact-rich primitive and let the orchestrator learn its operating range; Learning What to Say to Your VLA: Mostly Harmless VLA Steering (LFP) steers a frozen VLA via test-time language rewrites of task instructions; VIA: Visual Interface Agent for Robot Control (VIA) removes the VLA entirely. Steerable Policies flip the direction — retrain the VLA to accept many more prompt shapes so the orchestrator has more knobs — and the human-oracle experiment quantifies exactly how much of the “VLM-supervises-VLA” thread’s ceiling is bottlenecked by the VLA’s prompt vocabulary. The in-context-learning route (§V-B) is also a sharper instance of the pattern Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering hinted at: because the VLM’s job is now to choose which abstraction to command in rather than to write correct low-level actions, it reduces to standard VLM ICL that off-the-shelf models handle well.

The paper also strengthens the Thinking with Modalities cluster by using grounded pixel coordinates and gripper traces as first-class command tokens — a text-only projection of the “point / trace / mask” outputs that VLMs like Molmo produce natively, so the VLM’s pretrained grounding transfers directly. And on the Synthetic Training Data side, the Molmo + SAM2 + DETR + Gemini pipeline that turns 38k human labels into ~2M grounded steering commands is a concrete recipe for the “densify text supervision with grounded features extracted from the trajectory itself” pattern.

See also

Section titled “See also”