Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control

TL;DR

Steerable Policies (Chen, Bhatia, Glossop et al., UC Berkeley / Physical Intelligence / Google DeepMind) are VLAs trained not only on task-level commands but on a diverse mixture of steering commands spanning multiple abstraction levels — subtasks, low-level motions, and grounded pixel coordinates — all auto-generated from existing robot demonstrations via a foundation-model pipeline that culminates in Gemini synthesis. The paper argues that widening the low-level policy’s steering interface is the actual bottleneck for putting pretrained VLM knowledge to work in manipulation: two hierarchical controllers — a fine-tuned high-level embodied reasoner and an off-the-shelf VLM prompted with in-context learning over steering styles — outperform standard task-instruction VLAs, prior embodied-reasoning VLAs, and hierarchical baselines on real-world tabletop and long-horizon tasks. Instantiated on both OpenVLA and π0.5 (OpenPi) backbones.

Key claims

Standard task-level natural-language commands are too formulaic/vague to induce the range of physical skills novel manipulation tasks require; training on richer steering-command styles (subtasks, motions, grounded pixel coordinates) is proposed as the fix [§ “Our Steerable Policies are vision-language-action models trained on diverse and detailed steering commands”].
Steering-command labels are produced automatically by parsing robot trajectories with foundation models for embodied features, then compiling them into commands of all styles via an API-based VLM (Gemini) — no additional teleop or human annotation is required [§ “We use foundation models to automatically parse robot demonstrations into segments…”].
The recipe is backbone-agnostic: two Steerable Policies are trained by adapting the OpenVLA and π0.5 OpenPi codebases [§ “Our two hierarchical control methods using Steerable Policies”].
A fine-tuned high-level embodied reasoner VLM that autoregressively produces a grounded rationale before selecting a steering command outperforms standard VLAs, past embodied-reasoning methods, and a non-reasoning hierarchical ablation, on both the OpenVLA and π0.5 Steerable Policy [§ “Controlling Steerable Policies with high-level embodied reasoning VLMs…”].
An off-the-shelf VLM can steer the same policies via in-context learning — selecting a command style, observing the resulting behavior, and iteratively refining commands — with reported gains over a SayCan-like baseline that only exposes subtask-level commands [§ “Steerable Policies allow high-level VLMs…”].
Casting robot in-context learning as standard vision-language in-context learning removes the need for the structured scene / action representations prior robot-ICL methods depended on, because the steering interface is already the VLM’s native modality (text + pixel coordinates) [§ “Steerable Policies also allow VLMs to leverage in-context learning…”].

Method

Two ingredients. (1) A steering-command dataset. Existing robot trajectories are segmented and re-labeled by a multi-stage pipeline: foundation models extract embodied features (segments, motions, objects, keypoints), then Gemini compiles those features into a mixture of command styles — task-level descriptions, subtask instructions, low-level motions, and grounded pixel coordinates. The resulting labels are attached back to the same trajectories, so each demonstration is trained under many linguistic views. (2) A backbone-agnostic training recipe. Both an OpenVLA-7B and a π0.5-family Steerable Policy are trained on this mixture using the original codebases largely unchanged; the checkpoint used in the released HuggingFace artifact is steerable-policy-openvla-7b-bridge. At deployment the policy is driven by one of two hierarchical controllers: a fine-tuned high-level embodied reasoner VLM that emits a chain-of-thought rationale and then picks a steering command for the low-level VLA to execute; or an off-the-shelf VLM prompted to reason over the available command styles via in-context learning, observing the low-level VLA’s behavior and iteratively refining commands until success or timeout.

Results

Evaluated on real-world manipulation tasks including tabletop (“put the carrot in the pot”, “put the watermelon on the towel”) and long-horizon compositional / multi-step tasks (“make the blue block the only object on the plate”, “stack the pots on the towel”). Both hierarchical control methods on top of the Steerable Policy outperform matched standard VLAs, prior embodied-reasoning VLAs, and a hierarchical non-reasoning ablation, on both OpenVLA and π0.5 Steerable Policy backbones. The in-context-learning controller beats a SayCan-like baseline restricted to subtask-level commands, supporting the paper’s specific claim that gains come from choosing across command abstractions rather than from any single one. No per-task success percentages are given on the project page beyond the qualitative rollouts.

Why it’s interesting

This is a steering-interface lever for VLAs orthogonal to most others already filed under VLA Models. Where Learning What to Say to Your VLA: Mostly Harmless VLA Steering (LFP) leaves the VLA frozen and searches over natural-language rewrites of the task instruction at test time, Steerable Policies bake multi-abstraction commands into the training distribution so an off-the-shelf VLM can drive the policy in-context — a training-time complement to LFP’s test-time interface. It also contrasts with VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation (VoLo) and Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents (Harness VLA), which wrap frozen VLAs in orchestrators that route among primitives or interrupt mid-rollout: Steerable Policies changes the policy’s own vocabulary rather than adding routing logic on top. The Gemini-driven auto-labeling of trajectories with multi-abstraction commands is the same pattern surfaced by Segmenting Robot Video into Actionable Subtasks (WGO-Bench) and Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Models, pushed further into a policy-training recipe. Its use of grounded pixel-coordinate commands connects to the pointing-centric position of Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models.