Learning What to Say to Your VLA: Mostly Harmless VLA Steering

TL;DR

A CMU (Jeong / Swamy / Bajcsy) CoRL-submission project page proposing language steering as a test-time interface to frozen VLAs: instead of fine-tuning the action policy, an interactive search finds language sequences that improve closed-loop success, distills them into a Language Feedback Policy (LFP), and trains an improvement head that predicts when steering will help. The improvement head is conformalized with a class-conditional false-positive-rate guarantee, so the LFP refuses to intervene out-of-distribution and falls back to the base instruction. Reported gains over the base π0.5 VLA: +24.7% in simulation (LIBERO-OOD) and +65.0% on Franka hardware, while matching direct VLA-SFT performance with ~1/5 the on-policy rollouts.

Key claims

Closed-loop, trajectory-level language steering beats open-loop static prompt rewrites: 75.0% vs 71.3% success in simulation at matched refusal rate [Results, “Open-loop vs closed-loop” table].
Conformal calibration of the improvement head cuts harmful steering’s false-positive rate from 38.92% to 9.31% (at α=0.10 target) while raising success from 70.93% → 75.96% in simulation [Results, “Refusal improves success” + calibration table].
On hardware, conformal calibration drives the LFP’s harmful-steering FPR from 61.11% (uncalibrated π_RFT) to 2.22% (π_CP), with accuracy 84.72% → 99.17% across CubeSort / CubeMug / MarkerBlock / Microwave + two novel tasks [Results, hardware calibration table].
Language steering matches direct VLA fine-tuning at ~1/5 the data: π_RFT trained on 10 successful rollouts matches/exceeds π_VLA-SFT trained on 50, and continues to improve while VLA-SFT plateaus [Results, “On-Policy Rollout Usage”].
Direct VLA fine-tuning (π_VLA-SFT) degrades deployment success vs. the base VLA on held-out environments, while the LFP recovers it — narration alone (π_SFT) is insufficient [Results, “Baselines — training vs. deployment”].
The LFP transfers zero-shot to two unseen hardware tasks, beating both the base VLA and the narrated SFT baseline [Results, hardware “Novel tasks” rollouts].

Method

Three phases. (1) Interactive search: an outer loop perturbs the language input to a frozen VLA and rolls out in closed loop, scoring trajectory-level success; this produces (state, language-action, advantage) triples for any VLA the team can run, without access to its training data or weights. (2) Distillation into an LFP: a separate language feedback policy π_RFT is supervised to emit the language action ℓ_t given the current observation and the task instruction ℓ_task — re-eliciting frozen VLA skills at the right moment rather than rewriting the instruction once up front. (3) Conformalized improvement head: a classifier is trained alongside the LFP to predict whether steering will improve over the base instruction, then class-conditional split conformal prediction calibrates a refusal threshold so the rate of harmful steering is bounded by a chosen α (the page reports α=0.10). When the head refuses, the system falls back to ℓ_task; otherwise it emits the LFP’s language action and lets the frozen VLA act on it. The closed-loop variant is what enables recovery from mid-task adversarial perturbations (e.g. a human moving a placed cube back into the scene) — open-loop prompt edits cannot react to this.

Results

On LIBERO-OOD with a π0.5 VLA fine-tuned on LIBERO, evaluated across 200 visual×semantic perturbation combinations: base VLA ≈ 51%, narrated SFT (π_SFT) degrades deployment success vs. base, direct VLA-SFT helps only in training-like conditions, π_RFT and π_CP outperform every baseline. Reported headline: +24.7% sim, +65.0% hardware vs. the base VLA. Calibration tracks the diagonal FPR=α both in sim and on Franka hardware. Hardware setup spans four tabletop tasks (CubeSort, CubeMug, MarkerBlock, Microwave) plus two novel-composition tasks under visual (VOOD) and semantic (SOOD) perturbations; π_CP refuses harmful interventions on the less-steerable Microwave task and still beats the base VLA on the steerable ones. The “Compose” novel-task experiment is the clearest separation from direct fine-tuning: π_RFT solves unseen behavior compositions on the frozen VLA, while π_VLA-SFT degrades relative to the base VLA on the same tasks. Calibration table excerpts: simulation π_RFT (Acc 66.22%, FPR 38.92%, refusal 43.42%) → π_CP (Acc 77.74%, FPR 9.31%, refusal 57.66%); hardware π_RFT (Acc 84.72%, FPR 61.11%) → π_CP (Acc 99.17%, FPR 2.22%, refusal 24.72%).

Why it’s interesting

This is a structural counter-recipe to most other entries in VLA Models — which all modify the VLA itself (action pretraining in π*0.6: a VLA That Learns From Experience (RECAP), unified-VLM grounding in Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models, or clean teleop data in Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models) — by treating language as the only lever and the VLA as a black box. The closest sibling in spirit is VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation, which also wraps a VLA inside a closed VLM-orchestrator loop, but VoLo intervenes by interrupting and re-dispatching whereas the LFP intervenes by rewriting the prompt mid-rollout; the two could in principle compose. The conformalized refusal head is the unusual move — the wiki has no other filed VLA paper that buys an explicit OOD-harmlessness guarantee at the steering layer, and the +24.7%/+65.0% sim/hardware delta on a frozen π0.5 is the kind of cheap retrofit a deployed VLA stack would care about.