Skip to content
Bud's Wiki

R&B-EnCoRe: Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning

TL;DR

Section titled “TL;DR”

R&B-EnCoRe (Refine and Bootstrap Embodiment-specific Chain-of-Thought Reasoning) treats embodied CoT reasoning as a latent variable in an importance-weighted variational inference framework, so that VLAs can discover which reasoning primitives are actually informative for their specific embodiment and task, without external rewards or verifiers. The training pipeline generates synthetic candidate reasoning traces via “reasoning dropout” over a set of primitives (objects, plans, subtasks, moves, affordances, ego state), jointly trains prior and posterior VLA models on the diverse traces, then importance-samples a refined dataset for a second bootstrap-training pass. Applied to 1B/4B/7B/30B VLAs across LIBERO-90, real-world WidowX, four NaviTrace legged embodiments, and NuScenes autonomous driving, R&B-EnCoRe delivers +28% manipulation success, +101% legged navigation score, and −21% AV collision rate over baselines that reason over all primitives — while halving generated reasoning tokens at inference.

Key claims

Section titled “Key claims”
  • Effective reasoning primitives for a specific embodiment are unknown a priori, and indiscriminately reasoning over all primitives can hurt action prediction by feeding VLAs distracting or irrelevant signals [§I].
  • Formulating embodied reasoning as a latent variable in an IWAE framework yields an importance weight w(z) = p(a | z, c) / q(z | a, c) whose expected log gives an evidence lower bound on log p(a | c); sampling proportional to w refines training data toward action-predictive traces [§III, §IV-C, Proposition 1].
  • Warmstart data is built via reasoning dropout: for each demonstration, each reasoning primitive (Plan, Visible Objects, Subtask, Subtask Explain, Move, Move Explain, Gripper Position) is independently dropped with probability p_dropout, exposing the model to strategies ranging from concise to verbose [§IV-A, Alg. 1].
  • Prior model p(z, a | c) and posterior model q(z | a, c) share the VLA architecture and are jointly trained on the warmstart data; the posterior conditions on ground-truth action to propose action-explaining reasoning candidates [§IV-B].
  • Refinement (Alg. 2) draws K candidate traces from the posterior, computes prior-vs-posterior importance weights, and resamples one trace per demonstration via a categorical distribution over weights — yielding a compressed, high-signal reasoning training set for a second-stage bootstrap [§IV-C].
  • LIBERO-90 (1B MiniVLA): R&B-EnCoRe achieves 79.5% success vs 78.6% for reasoning-on-all-primitives, at half the average generated token count (129.3 vs 256.8) [Table II].
  • LIBERO-90 Visible-Objects ablation: R&B-EnCoRe raises Object Criticality Rate (fraction of listed objects that appear in an independent Plan/Move validation set) from 0.03% (all objects) or 3.43% (random objects) to 25.02% [Table I].
  • Bridge WidowX (7B OpenVLA): under “Action Forcing” (test-time reasoning suppressed for speed), R&B-EnCoRe matches all-primitives on in-distribution tasks and outperforms on OOD-target and OOD-scene tasks, because rigid all-primitive templates overfit to specific scene features [Fig. 6].
  • Bridge WidowX with test-time reasoning enabled: R&B-EnCoRe cuts inference time from ~4.5s to ~1.6s per step and outperforms all-primitives by ~40% on OOD-with-distractions tasks [Fig. 7].
  • NaviTrace legged navigation (30B Qwen3-VL-MoE): the reasoning distribution refined by R&B-EnCoRe concentrates on Affordances and Move primitives while suppressing counterfactual reasoning (except at specific decision moments), consistent with recent counterfactual-reasoning findings [Fig. 4b, Fig. 9, §V-C].
  • Autonomous driving (NuScenes): R&B-EnCoRe reduces collision-rate metric by 21% versus reason-on-all-primitives baseline [§V-D].

Method

Section titled “Method”

Given a robot demonstration and a set of reasoning primitives (Plan, Objects, Subtask, Move, Affordances, etc.) generated by prompting a foundation VLM as VQA tasks, R&B-EnCoRe unfolds in three stages.

Warmstart (Alg. 1). For each demonstration (context c, action a), sample k reasoning traces by independently including each primitive with probability 1 − p_dropout, producing training tuples (c, z, a) covering the full power-set of primitive combinations.

Joint prior/posterior training. The same VLA architecture is trained twice on the warmstart data: (i) as a prior p(z, a | c) that observes only context and must generate reasoning followed by action, and (ii) as a posterior q(z | c, a) that additionally conditions on the ground-truth action, learning to propose reasoning traces that explain the correct action. This mirrors a VAE encoder-decoder but adapted so both roles run on the same backbone.

Refine and bootstrap (Alg. 2). For each demonstration, draw K posterior samples z_i ~ q(z | c, a), compute importance weights w_i = p(z_i, a | c) / q(z_i | c, a), and resample one trace via categorical distribution Cat(w_1, …, w_K). The refined dataset {(c, z*, a)} is then used to retrain a fresh VLA — this second-stage model is the deployed policy. Reasoning primitives whose expected p/q ratio exceeds 1 are effectively amplified; those with ratio < 1 are pruned. The proposition in §IV-C shows this ratio estimates an information benefit quantity: the reduction in KL divergence between the model’s action distribution and the expert’s when the primitive is included.

Evaluated across four VLA scales (1B MiniVLA, 4B, 7B OpenVLA, 30B Qwen3-VL-30B-A3B Instruct MoE) and four embodiment classes (Franka in LIBERO-90, WidowX in Bridge v2 hardware, legged navigation on NaviTrace’s bipedal/wheeled/bicycle/quadruped robots, autonomous driving on NuScenes). Reasoning content is sourced from Gemini 1.0 for manipulation, and from the base VLM itself via VQA prompting for legged navigation (fully self-supervised, no external teacher).

Results

Section titled “Results”
  • LIBERO-90 (1B MiniVLA): 79.5% success (R&B-EnCoRe) vs 78.6% (all primitives) vs 76.5% (random primitives) vs 75.9% (no reasoning), with average generated tokens dropping from 256.8 to 129.3 [Table II].
  • LIBERO-90 objects-only ablation: Object Criticality Rate 25.02% (R&B-EnCoRe) vs 3.43% (random) vs 0.03% (all-objects listing) [Table I]; task success 80.3% vs 77.0% vs 76.1% vs 75.9% (no reasoning).
  • Bridge WidowX (7B OpenVLA, Action Forcing / no test-time reasoning): matches all-primitives on in-distribution; outperforms all-primitives on OOD-target and OOD-scene splits [Fig. 6, across 468 trials on 9 tasks].
  • Bridge WidowX with test-time reasoning: 4.6s → 1.6s per step; ~40% relative gain on OOD-with-distractions [Fig. 7].
  • NaviTrace legged (30B): +101% average score across bipedal/wheeled/bicycle/quadruped over all-primitives baseline; refined distribution concentrates on Affordances + Move, suppresses counterfactuals except at decision moments [Fig. 4b, Fig. 9].
  • NuScenes AV: −21% collision-rate metric [§V-D].
  • Ablation of the sampling-importance-resampling: without weight-proportional resampling, refined performance regresses toward the warmstart baseline [§V-E].

Why it’s interesting

Section titled “Why it’s interesting”

R&B-EnCoRe reframes the “which reasoning primitive helps?” question — currently answered by manual annotation guidelines and month-long labeling engineering — as a self-supervised inference problem, and the answer that falls out (Move + Gripper Position + Subtask beat verbose Plan/Explain variants for manipulation; Affordances + Move beat counterfactuals for legged navigation) is a genuinely new empirical result. This is directly complementary to Robotic Control via Embodied Chain-of-Thought Reasoning (ECoT), which fixed a rigid template of embodied reasoning primitives — R&B-EnCoRe learns which parts of that template the specific embodiment actually needs. The variational-inference framing puts it in the same family as recent latent-CoT work for pure LLMs but replaces the external verifier / reward model with the VLA’s own action-prediction likelihood, making it especially attractive for robotics where verifiable rewards are expensive or unavailable. The scaling across 1B–30B parameters and four embodiment classes is unusually broad for an embodied-reasoning paper, and the emergent “Action Forcing” result (train with reasoning, deploy without) resolves a live tension between reasoning quality and inference latency that has kept many teams away from Embodied-CoT in production.

See also

Section titled “See also”