Skip to content
Bud's Wiki

VLA Models

TL;DR

Section titled “TL;DR”

Vision-Language-Action (VLA) models are end-to-end policies that map perception plus a language goal to robot or vehicle actions, typically built on top of a VLM backbone with an attached action head (FAST-tokenized discrete actions, a flow-matching action expert, or a diffusion-transformer chunk predictor). The filed papers disagree on which capability is the actual lever for generalist manipulation — action-pretraining-at-scale (π*0.6: a VLA That Learns From Experience (RECAP)), clean teleop data (Spirit-v1.5: Clean Data Is the Enemy of Great Robot Foundation Models), or pointing/grounding in a unified VLM (Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models) — and at least one filed pointer (Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level) argues the VLA framing itself under-weights sensorimotor / tactile / contact dynamics.

Key claims

Section titled “Key claims”

Recent contributions

Section titled “Recent contributions”
  • [2026-07-20] Danfei Xu — two root-level paradigm shifts in robot learning: Sim2Real for locomotion, Behavior Cloning for manipulation: Danfei Xu frames Behavior Cloning as the root-level deep-learning-era paradigm shift for manipulation (paired with Sim2Real for locomotion), a taxonomy that puts the entire VLA lineage under one banner.
  • [2026-07-20] Introducing Cosmos 3 Edge: Cosmos 3 Edge Policy (DROID) is a 4B on-device WAM post-trained from the shared Cosmos 3 dual-tower backbone for pick-and-place, adding an edge-hardware WFM-as-VLA-backbone row to the recipe board — competitive at 15 Hz control on Jetson Thor with 32 actions/inference at 640×360 observations, released with post-training scripts so a small H100 cluster / DGX Station can specialize it. Sibling to the flagship Cosmos 3 policy at 4B scale, and a direct contrast with clean-teleop scaling (Spirit v1.5), UMI-hour scaling (XR-1), and RECAP-style RL (π*0.6) as the lever picked for edge deployment.
  • [2026-07-20] Pareto (Hebbian Robotics) indexes AllenAI's MolmoAct2 Bimanual YAM dataset — clusters + prompt normalization (Brandon Ong): Hebbian Robotics’ Pareto tool now indexes AllenAI’s MolmoAct2 Bimanual YAM dataset with cluster-discovery + prompt-normalization surfaces — a first filed instance of intra-dataset analysis tooling for VLA training data, complementing the inter-dataset catalogs (datasets.bot, TrueLabel) already on file.
  • [2026-07-20] Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline): Learning to Fold (Larchenko) is the tightest single-researcher independent reproduction yet of the π*0.6: a VLA That Learns From Experience (RECAP) RECAP recipe — a flow-matching VLA (BEHAVIOR-1K 2025 winner’s SigLIP + Gemma-2B prefix + Gemma-300M action expert) fine-tuned with AWR + RECAP-style advantage conditioning that placed 1st of 62 teams in the LeHome Challenge 2026 simulation round (79.63%, 6.1 points ahead of 2nd) and 2nd in the ICRA real-world garment-folding final. Two structural moves worth pulling out on top of RECAP: (1) policy is its own value function, Q-function, and cheap world-model substitute — one learned query token feeds cheap linear heads reading only image tokens for success, task completion, garment type, current keypoint distances, 30-frame-future keypoint distances, and an action-conditional success residual; the Q-head drives advantage estimation, live failure detection, and best-of-N candidate selection with no separate value model (a counter-recipe to the World Value Models for Robotic Manipulation “separate value model” axis and to π*0.6’s binarized text-token value); (2) AWR applied through the sampler rather than through the loss weight — high-|advantage| frames are simply loaded more often, batches stay unweighted MSE, and effective batch utilization stays at 100% of the weight mass, with auxiliary-head losses importance-weighted back to unbiased. Also introduces Thompson-sampling bandits over per-garment-type inference hyperparameters (execution length, playback speed, inpainting onset, CFG scale, noise temperature, best-of-N), asynchronous training/rollout/DAgger loop over HuggingFace Hub, and a checkpoint-rollback trick that reliably kicked the policy out of local optima. Sibling reproduction to Evo-RL: Open Real-World Offline RL on SO-101 and AgileX PiPER with different plumbing (Hub-as-message-bus vs LeRobot CLI) and a richer head stack.
  • [2026-07-18] Sunday Robotics data engine — from one Memory Developer off Craigslist to 1,000+ (Perry Jia thread): Perry Jia (Data Ops, Sunday Robotics; ex-Tesla Autopilot/Optimus data-engine lead) opens a thread claiming Sunday’s ACT-2 data engine is backed by 1,000+ paid Memory Developers grown from a single Craigslist hire over two years — puts an operational-labor number on Sunday’s real-home glove-capture recipe row on the recipe-lever board, complementing the model-side Chinchilla claim from Sunday Robotics ran the Chinchilla scaling law with their VLA model (Tony Zhao follow-up) and the “embracing chaos” thesis from Low-data inductive-bias lessons don't translate to the high-data regime — embracing chaos produced emergent policy capabilities (Nishant Desai / Sunday Robotics) with the workforce-substrate that generates the chaos. No technical detail beyond the workforce-count claim; underlying thread posts (2/N…) not retrievable at filing.
  • [2026-07-17] WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time: WAM-TTT (Feng, Lyu, Wang et al.) is the first filed paper form of test-time-training as a WAM steering interface — the RoboTTT tweet’s (RoboTTT — 8k-timestep robot policy via test-time training (Jim Fan / NVIDIA GEAR)) recipe made concrete. Stakes out a sixth structural lever: keep the WAM backbone frozen and push each demonstration into a small key–value memory updated by self-supervised video prediction on raw human videos. The load-bearing piece the tweet did not describe is a meta-training stage on paired human-robot data with a KV memory reconstruction objective that aligns human-video-derived memory content with robot-behavior-relevant features; this is what lets the human-video TTT update translate into control gains. Reports consistent wins over in-context human-video conditioning across manipulation tasks and generalization settings (no per-task numbers in the abstract). Sibling in framing to Learning What to Say to Your VLA: Mostly Harmless VLA Steering‘s LFP (frozen VLA + test-time steering via language rewrites) and to World Pilot: Steering Vision-Language-Action Models with World-Action Priors (WAM-as-policy-peer) — but the only filed instance that steers via TTT on the WAM’s own memory.
  • [2026-07-17] Worlds in One Demo: A Synthetic Data Engine for Learning Open-World Mobile Manipulation: WANDA (CMU; Guo, Li, Shi) is evaluated with π0.5 as the downstream consumer of its one-demo-to-many-trajectories synthetic data engine — reports π0.5 fine-tuned only on WANDA-generated data reaching 54.8% real-world task progress across five long-horizon Agibot G1 tasks and outperforming a π0.5 trained on 50 teleop demonstrations. Cross-embodiment story is zero-shot Linearbot deployment from an Agibot G1 source demo via world-substrate reuse + whole-body re-planning, not policy retargeting; sibling to Video2Robo and LEGS on the “3DGS-backed synthetic data as a VLA training substrate” axis, with the mobile-manipulation + contact-rich variant.
  • [2026-07-17] Mixture of Frames Policy: Multi-Frame Action Denoising for Bimanual Mobile Manipulation: Mixture of Frames Policy (Dian Wang, Shuran Song, Jeannette Bohg et al.) adds a representation-side lever orthogonal to the recipe axes already on the page: instead of picking one action-coordinate frame (base / left-hand / right-hand / rel-traj), denoise the action chunk in all of them in parallel and let an MoE router per-subtask combine the experts. Reports MoF-MoE 66.8 vs vanilla Diffusion Policy 50.3 and — critically — beats the oracle-per-task frame choice (63.8) by 3 points, evidence that gains exist from within-episode frame switching that no fixed per-task frame can recover. Ablations show best-frame supervision (−6.2), auxiliary loss (−8.0), and canonical rel-traj frame (−2.5) are all load-bearing. Sibling to B-spline Policy’s structural-action-head lever, but changes the frame the action space is defined in rather than its output type.
  • [2026-07-17] Sunday Robotics ran the Chinchilla scaling law with their VLA model (Tony Zhao follow-up): In-thread follow-up to Sunday Robotics’ ACT-2 Preview launch: Tony Zhao asserts they “actually ran the exact chinchilla-scaling-law” on their VLA, deferring the writeup to @nadeesha99 — no numbers or methodology in the tweet. If it lands, would be the first filed instance of Chinchilla-style isoFLOP (N,D)(N,D) scaling on a production home-robot VLA, sharpening Sunday’s real-home glove-capture recipe’s data-substrate-as-lever position vs teleop-scaling and UMI-scaling competitors.
  • [2026-07-16] Solving Dexterity: A Full-Stack Approach (mimic hand M1 + wearable U1): mimic robotics announces the M1 dexterous hand + U1 kinematically-locked wearable as the hardware substrate for their upcoming Video Action Model line (successor to mimic-video), taking an infrastructure-first position on VLA data recipes: teleoperation is structurally-capped by latency, retargeting friction, and operator discomfort, and only wearable-collected data at wearable-fidelity escapes those caps. The “data pyramid” (human video base → wearable middle → teleop/deployment top) is proposed as the scaling strategy, with hand morphology held constant across all three tiers to eliminate the cross-embodiment gap between pretraining substrate and deployment target — sibling to but hardware-driven rather than dataset-driven versus Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training‘s 100K-hour UMI recipe.
  • [2026-07-16] RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark: RoboMemArena (HKUST-GZ + Zhejiang + Westlake + Tsinghua) adds a memory-augmented dual-system recipe distinct from the existing recipes on this page: PrediMem pairs a Qwen3-VL-8B S2 planner with a π0.5-shaped S1 actor, an explicit keyframe bank + recent-frame buffer, and a training-only predictive-coding auxiliary head that trains S2 to anticipate next-frame ViT features (α=0.1 latent MSE + cosine) — reshaping the hidden space so keyframe-write decisions come from the standard LM head, no separate retrieval module. Reports 38.5% TSR / 55.2% CSR on their 26-task long-horizon memory benchmark vs π0.5 21.5/38.7 and MemER 27.3/49.1; ablations show removing the predictive-coding head drops TSR to 32.3% and removing the keyframe bank drops it to 17.7%. Sharpest datapoint yet against zero-shot use of frontier VLMs as memory managers — GPT-5.4 used as S2 without robotic-domain training reaches only 8.7% TSR, arguing memory-management VLMs need robotic-specific supervision rather than raw multimodal scale. Backbone-scaling curve is clean: Qwen3-1.7B 19.9% → Qwen3-4B 31.9% → Qwen3-VL-8B 38.5% TSR.
  • [2026-07-16] RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination: RxBrain (Tencent) — extends a 6.2B unified embodied-cognition model to continuous robot action generation, reporting real-robot performance without large-scale action-data pretraining.
  • [2026-07-16] Xiaomi-Robotics-1 (XR-1) — Scaling VLA Foundation Models with 100K Hours of Embodiment-Free UMI Pre-training: Xiaomi-Robotics-1 (XR-1) stakes out a seventh lever — scale-of-embodiment-free-UMI-data: 100,000 hours of hand-held UMI wrist-camera pre-training across 1,700+ scenarios (VLM-auto-labeled with per-clip state-transition captions), then cross-embodiment post-training. Reports clean pre-train scaling (validation action error monotone-decreasing in data + model) that transfers to real-robot success in unseen environments after post-training. Sharpest π0.5 head-to-head yet: at <10 h/task of demonstrations XR-1 reaches 75% overall real-robot success vs π0.5’s 40% (Phone Packing, Printer Refilling, Laundry Loading, Box Packing), and <40 h/task lifts to 85%. Also SOTA on four sim benchmarks with +58.3% relative gain on RoboDojo and +23.2% on RoboCasa365. Sibling to Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model from the same Xiaomi Robotics lab — U0 augments robot data via a WFM, XR-1 replaces it with UMI scale.
  • [2026-07-15] B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations: B-spline Policy (Han, Xiong, Chen, Liu, Torralba, Zhu, Du) moves orthogonally to the recipe axes catalogued here — it changes the action head’s output space from discrete action chunks to continuous B-spline curves (control points + knots as one fixed-size vector), a drop-in replacement in Diffusion Policy and ACT that composes with any backbone / pretraining / RL recipe. Because the curve is time-continuous and differentiable, the low-level controller can sample at arbitrary rate, temporally rescale to run 2–4× faster, and read velocity/acceleration off derivatives for feedforward PD; three real-world tasks show 2–4× time reduction at matched-or-better success (Cube Picking 6.48 s → 2.45 s at 20/20; Table Cleaning 41.4 s → 18.05 s at 11–15/20; DemoSpeedUp baseline collapses to 3/20 or 0/20 at the same speeds), and inference-time segment alignment is flagged as load-bearing.
  • [2026-07-15] RoboTTT — 8k-timestep robot policy via test-time training (Jim Fan / NVIDIA GEAR): RoboTTT (Jim Fan / NVIDIA GEAR launch tweet) stakes out a sixth structural lever alongside action-pretraining / clean-teleop / unified-VLM-pointing / frozen-WFM+action-expert / native-video+action pretraining: memory-into-weights via test-time training. A small TTT core inside the policy takes one gradient step per observation, giving 8,000 timesteps of context (~5 min) at constant per-step inference cost — 3 orders of magnitude beyond prior robot-policy SOTA. Reports one-shot in-context learning from a single human demonstration video (unseen circuit-board assembly), mid-episode error recovery from the same TTT loop, and a Context Scaling Curve where 8K-context pretraining beats 1K by 62% with no saturation. Launch-tweet only; deep-dive report deferred.
  • [2026-07-14] Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model: Xiaomi-Robotics-U0 is evaluated as an upstream data engine (not a VLA itself) for π₀.₅ — adding its style-transfer augmentation to real demos lifts π₀.₅ OOD success 36.9% → 63.2% on real-world manipulation, sharpening the case that WFM pretraining prior compounds into policy generalization.
  • [2026-07-14] How Claude Performs on Robotics Tasks: Anthropic Frontier Red Team’s “Claude Plays Robotics” evaluation stress-tests the LLM-as-VLA-supervisor interface across Claude Opus 4→4.7 + Mythos Preview, GPT-5.4, and Gemini 3.1 on LIBERO manipulation and Go2/G1 locomotion — with four control interfaces (direct torques, programmatic Python, pretrained policy / MolmoAct VLA supervision, RL training). Two findings sharpen the concept: (1) every LLM-guided MolmoAct configuration on LIBERO underperforms MolmoAct running alone, but newer Claude generations (Opus 4.5/4.6/4.7) defer to the VLA less on novel LIBERO-like tasks where MolmoAct fails, and Opus 4.5/4.6 + Gemini 3.1 outperform MolmoAct alone in that regime — sharpest quantitative evidence yet for the VoLo / LFP “orchestrator over interruptible VLA” thesis, and a concrete miscalibration axis (Mythos Preview overrides more than warranted on tasks the VLA handles well); (2) direct low-level control fails on high-DoF platforms — no model stands up the 29-DoF G1 humanoid — reinforcing that VLA supervision, not direct joint control, is the realistic deployment surface for frontier LLMs.
  • [2026-07-14] GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks: GaP (Berkeley/NVIDIA/CMU; Goldberg, Fan, Zhu) introduces Variational Automation as a new problem regime between fixed automation and generalist robotics — bounded workcell, non-trivial pose/geometry variation — and answers it with a Graph-as-Policy counter-recipe: a multi-agent coding harness composes a directed computation graph of typed skill nodes from a 51-skill MORSL library, then a parallel-simulation rehearsal loop autonomously localizes failures to specific nodes and edits topology + parameters until success plateaus. Across 8 new VA benchmarks, GaP holds 0.93–0.99 where π₀.₅ / MolmoAct2 / CaP-X collapse to 0.05–0.24 under pose variation, matches an expert hand-engineered bimanual graph (0.953 vs 0.987 on crate washing), and — critically for the recipe debate — stages VLAs as a wrist-camera preamble for >2× success (π₀.₅ 0.17 → 0.67 on Pack fixed). Ablations collapse to zero when the graph is removed or the multi-agent decomposition is collapsed. Same Goldberg lab as CaP-X and RATs; the sharpest structural move yet in the Code-as-Policy line — the graph is the policy, not just a way of sampling training data. Composes with VLAs rather than replacing them.
  • [2026-07-14] RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks: RoboClaw (AgiBot + NUS + SJTU, Yao Mu group) stakes out a lifecycle-unification lever distinct from the recipe/backbone/data axes: a single VLM agent with structured memory drives both data collection and deployment through an MCP tool interface, keeping contextual semantics consistent across the pipeline. The signature contribution is Entangled Action Pairs (EAP) — pair every forward manipulation policy with a simpler inverse reset policy trained to restore the precondition region, so the agent alternates forward/reset and generates on-policy data without manual environment resets. Iterative EAP rollouts monotonically lift per-skill success rates on an Agibot G01 (Lipstick Insertion 2/50 → 23/50 over five 50-sample iterations, Body Lotion 21→43/50) on top of a fixed human-demo seed; the paper reports 25% absolute success improvement on long-horizon vanity-organization vs a same-data non-agent baseline and 53.7% reduction in human time investment vs manual data collection. Sibling in framing to VoLo’s “VLA-as-interruptible-tool under a VLM orchestrator” (VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation) but pushed upstream into data collection, and complementary to the agentic self-improvement axis of ENPIRE (coding-agent policy authorship) and Evo-RL (RECAP-style RL) with a distinct “agent schedules + resets fixed VLA primitives” answer.
  • [2026-07-13] InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization: InternVLA-A1.5 (Shanghai AI Lab / InternRobotics) stakes out a sixth structural lever alongside action-RL / clean-teleop / unified-VLM-pointing / frozen-WFM+action-expert / native-from-scratch video+action pretraining: foresight-token bottleneck supervised by a frozen video-generation WFM. A native Qwen3.5-2B backbone keeps training on VQA + subtask prediction; a lightweight unified action expert attaches via shared full-attention while modality-specific Gated DeltaNet processing is preserved; learnable foresight tokens query the shared context for task-relevant future dynamics and are distilled against a frozen WAN 2.2-5B video model; the video branch is discarded at inference so control latency stays practical. Pretrained on 1.2M robot episodes + 3M multimodal samples, reports SOTA overall on six sim benchmarks (RoboTwin 2.0 93.2, LIBERO 98.9, LIBERO-Plus 84.8, SimplerEnv 80.8, DOMINO 27.7, EBench 35.2) and claims strongest compositional generalization on held-out instruction bindings in real-world settings — sibling in spirit to µ₀’s frozen-trace-WM + action expert and World Pilot’s dual-path WFM steering, but with the WFM interface pushed into a compact learnable-token bottleneck rather than a separate peer or trajectory abstraction.
  • [2026-07-12] 3D-Object Perception Transformer (3PT): 3PT (Intrinsic) supplies the perception front-end that VLA industrial-manipulation systems need: zero-shot 6-DoF pose with sub-millimeter accuracy from RGB alone (validated on 0.57 mm-clearance DIMM insertion and 100+ cluttered sheet-metal bin-picks), no depth sensor, no per-object fine-tuning — CAD model in, pose out. Complements the world-model / policy-learning side of the VLA stack with a solved perception layer for CAD-known objects.
  • [2026-07-11] LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition: LUCID (CMU) stakes out an action-data-free, VLM-free counter-recipe: no VLM backbone, no robot demonstrations, no action pretraining. Intent (short-horizon object flow + palm pose) is learned from unstructured human video with a CoTracker3-derived point-token transformer conditioned on DINOv3 patches; a sim-only PPO+distillation sensorimotor policy realizes that intent on a specific embodiment. Reports 73% average vs a Veo-3.1 open-loop-planner’s 28% on three web-supervised tasks and matching 19/30 success on push-T + cable routing across a LEAP hand and parallel-jaw gripper from 1 hr of smartphone video. Directly counters both the action-pretraining-at-scale position of π*0.6 and the unified-VLM-pointing position of Embodied-R1.5 by showing that a hand-designed intent interface plus a generalist sim policy suffices to span embodiments without any action-labeled data.
  • [2026-07-10] Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio: Temporal Ratio (Mishra, Chen, Xu, Liu, Chen, Mao) names the video-action generalization gap and diagnoses it as a head–backbone coupling failure: flow-matching action heads on top of Cosmos-Predict 2.5-2B silently ignore the video prediction in both directions (misleading predictions on ID → still succeed, correct compositional predictions on OOD → still fail), so raw sweeps over adaptation strategy × video noise level × rollout horizon show no clean pattern. Introduces TR = action-attention(future) / action-attention(current) as a runtime diagnostic that collapses the noisy design-space manifold to a single future-reliance axis and varies with task phase (rises during planning, falls during precise manipulation); TR-Adaptive Guidance uses it at inference to amplify compositional video conditioning when planning and back off during manipulation, improving compositional OOD success on LIBERO spatial/goal/object and on 22 real-world bimanual YAM tasks (trained on ABC-130k) while preserving ID precision — an interpretability-first alternative to the “unify the world model and action decoder” recipe of LingBot-VA and to the “add tactile/contact modality” counter-position of the Malik / Yunzhu Li thread.
  • [2026-07-10] LingBot-VA 2.0: Native Video-Action Pretraining for Generalizable Robot Control: LingBot-VA 2.0 (Robbyant / Ant Group, direct v1 successor) adds a fifth structural recipe alongside action-RL / clean-teleop / unified-VLM / frozen-WFM+action-expert: native from-scratch pretraining of the full video-action stack for causality and control, rather than adapting an off-the-shelf bidirectional video generator (v1’s approach). Instantiates it with a semantic visual-action tokenizer (ViT-AE + frozen Perception Encoder alignment + self-supervised latent-action tokenizer via IDM+FDM bottleneck), a Mixture-of-Transformers with a sparse MoE-13B-A1.9B video stream (DeepSeek-V3-style 128 experts / top-8 / auxfree balancing) and dense action expert, Multi-Chunk Prediction from Next Forcing as an auxiliary head, and Foresight Reasoning (async prediction + learned-forward-dynamics re-grounding on each freshest observation) for real-time closed-loop control at claimed 225 Hz peak. Sibling to From Foundation to Application: Improving VLA Models in Practice (LingBot-VLA 2.0) (same lab’s parallel pragmatic π-style bet on a Qwen-VL backbone) — the Robbyant lab is running both experiments in parallel; reports outperforming π0.5 and LingBot-VA v1 “by a substantial margin”.
  • [2026-07-09] From Foundation to Application: Improving VLA Models in Practice (LingBot-VLA 2.0): LingBot-VLA 2.0 (Robbyant/Ant Group) triples the v1 pretraining corpus to ~60K hours (50K robot trajectories across 20 configs + 10K egocentric human video), extends the action space beyond dual-arm to include head/waist/mobile-base/dexterous-hand DoFs, and adds a two-headed future-prediction proxy task (video representation model for semantic priors + depth estimator for geometric cues). Reports GM-100 gains in a generalist setting and strong cross-embodiment long-horizon mobile manipulation across two platforms — the deployment-focused sibling to Causal World Modeling for Robot Control (LingBot-VA)‘s architectural-unification bet in the same LingBot family, and the largest robot-side open-VLA pretraining corpus filed so far.
  • [2026-07-09] LIME: Learning Intent-aware Camera Motion from Egocentric Video: LIME (ETH Zürich, Sun et al.) adds a decoupled viewpoint-policy interface upstream of the VLA action space: given an RGB view + free-form intent, predict a relative SE(3) target pose via an auto-regressive observation-gain text head + continuous flow-matching pose head, trained on ~3M hindsight-VLM-labelled pairs mined from RoomTour3D + Nymeria egocentric video. Reports a single LIME viewpoint step before an unchanged VLA policy lifts LIBERO-Goal success from 26% → 74% under randomized initial pose across all 10 tasks — sharper isolation of the viewpoint-choice lever than ActiveMimic’s joint-with-manipulation pretraining, and evidence the interface composes with any downstream VLA.
  • [2026-07-08] RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation: RynnWorld-4D-Policy is a world-model-conditioned VLA whose inverse-dynamics head reads the tri-modal (RGB + depth + optical flow) diffusion model’s internal representations in one forward pass, avoiding denoising-time policy latency and reporting SOTA on real-world dexterous bimanual tasks.
  • [2026-07-08] Robostral Navigate: single-camera AI navigation: Robostral Navigate (Mistral) extends the “grounding-VLM-as-policy” recipe from manipulation (Embodied-R1.5) to navigation: 8B model initialized from an in-house pointing/grounding VLM, single-RGB-camera input, pointing-based action space (image-space waypoint + arrival orientation), 400K sim trajectories, prefix-cached tree-attention SFT (22× token reduction), then CISPO online RL (+3.2%); reports 76.6% success on R2R-CE validation-unseen — +9.7 over best prior single-camera baseline and +4.5 over best depth/multi-camera baseline, the first filed VLN system with published numbers behind the unified single-camera language-first framing Qwen-RobotNav announced without a report.
  • [2026-07-08] Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models: VLA-MBPO (Zhang et al., Nanjing / Mila / Yang Yu group) stakes out a UMM-as-joint-world-model recipe: finetune Bagel (a pretrained AR+Diffusion unified multimodal model) as a chunk-level world model that jointly predicts multi-view next-frames and reward tokens, then run PPO/Flow-Noise entirely inside it. Reports +9.1 avg on LIBERO (+12.2 on LIBERO-Long) over SFT with a single hyperparameter set across all suites, beats matched online RL by +3.3, and validates on 5 real-world Arx-X5 / Galaxy-R1 tasks including 3 mm cable plugging and mobile wipe-board control. Ablations: removing Interleaved View Decoding tanks wrist-view LPIPS from 0.254 to 0.454; full-horizon rollout drops LIBERO-Long from 66.8 → 52.8, empirically confirming the compounding-error blow-up their theorem predicts.
  • [2026-07-06] Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots: Embodied.cpp (Xu et al., SEU-PAISys) is the first filed llama.cpp-shaped inference runtime specialized for VLAs and WAMs — portable C++ + GGUF weights + one backend abstraction across CPU/CUDA/NPU, factored around a five-layer runtime contract (input adapters, sequence builders, backbone execution, head plugins, deployment adapters) derived from an architectural survey of the VLA/WAM zoo. Ships with production servers for pi0.5, HY-VLA, and LingBot-VA today (Apache-2.0), reports 100.0%/91.0% closed-loop task success on HY-VLA/π0.5, and pairs cleanly with DimOS on the deployment surface — filling the model-plane gap between “trained VLA” and “policy running on a robot”.
  • [2026-07-04] RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation: RoboWorld evaluates 8 open-source VLA policies (π₀.₅, PaliGemma-Fast, and others) inside a DROID-trained neural simulator, reproducing per-policy successes and failures from a shared initial frame and matching real RoboArena rankings at r = 0.989.
  • [2026-07-02] Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement: SILR (Liu et al., Fan Shi group) stakes out a sim-in-the-loop counter-recipe: keep a lightweight offline-distilled base policy, then refine every action step via prior-guided MPPI over K parallel FLASH physics rollouts seeded by an RGB→cloth-mesh real-to-sim module. Real hardware jumps from 1/10–3/10 base-policy folding success to 8/10–9/10, with component ablations showing that both the physics backend choice (FLASH beats Isaac Sim / Newton on KP error at matched K) and the RGB real-to-sim (beats point-cloud diffusion) are load-bearing — an operational answer to the sensorimotor counter-position for the deformable-object regime.
  • [2026-07-01] Scale Robot Policy Evaluation with Ray (Distributed Sim-Eval on Anyscale): Anyscale’s Ray + Anyscale reference architecture for VLA policy evaluation puts concrete numbers on the serving-side constraints downstream of the recipe recipes: GR00T-N1.7-3B has 26.79 s cold-start (Ray Serve’s 30 s default health-check window is too short, must be raised to 300 s) and ~1.6 s per forward pass returning a 40-step action chunk; disaggregating the policy fleet from the Isaac Sim workers onto separate GPUs is required because both are GPU-bound, and action chunking with action_horizon = 8 lets one policy replica feed ~8× as many concurrent simulators as query-every-step would.
  • [2026-07-01] DimOS — The Agentive Operating System for Physical Space (v0.0.13): DimOS v0.0.13 is the first filed deployment runtime for VLA policies — Python SDK / ROS-free stack for Unitree Go2/G1/B1 quadrupeds and humanoids, xArm/AgileX Piper arms, and MAVLink drones, organized around typed Modules (In[]/Out[] streams) + autoconnect() Blueprints + swappable transports (LCM, SHM, DDS, ROS 2), with agents refactored as first-class Modules with direct stream access; complements the recipe-side entries by owning the plane between “trained VLA” and “policy running on a robot”.
  • [2026-06-30] TrueLabel — Physical AI Dataset Directory: Robotics, Humanoid & Egocentric Data: TrueLabel Physical AI Dataset Directory — third-party catalog of 658 indexed robotics / humanoid / egocentric / teleoperation / simulation dataset pages with license, modality, robot platform, and popularity metadata; sibling to datasets.bot at roughly an order of magnitude more entries, useful as a scoping dashboard when picking a VLA training substrate.
  • [2026-06-30] EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data: EgoScale (NVIDIA GEAR) adds the scale-of-human-data lever to the recipe board: pretrains a flow-matching VLA on 20,854 h of action-labeled egocentric video (>20× prior work), reports a log-linear loss-vs-hours scaling law (R²=0.9983) that predicts real-robot success, and reaches +54% absolute average success over no-pretraining on five 22-DoF dexterous tasks after a small (~50h human/~4h robot) aligned-mid-training stage; one robot demo + 100 aligned human demos gives 88% on Fold Shirt, and the same prior transfers to a Unitree G1 tri-finger hand (+30%+ absolute).
  • [2026-06-30] datasets.bot — Curated catalog of robotics and embodied-AI training datasets: datasets.bot — external curated index of robotics/embodied-AI training datasets (AgiBot World 2026, HIW-500, ABC-130k, GR00T X-Embodiment Sim, …) with license + modality + popularity-proxy metadata, useful as a scoping dashboard before picking a VLA training substrate.
  • [2026-06-30] ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining: ACE-Ego-0 (ACE Robotics + CUHK MMLab) adds a fifth lever to the egocentric-vs-teleop pretraining axis: reliability-aware joint training across heterogeneous robot, sim, and pseudo-action-labeled egocentric streams (4.53K + 1.48K hours), with a unified action representation (camera-space actions + morphology conditioning + time-aligned chunking) and a human auxiliary loss that concentrates supervision on reliable samples — explicitly modeling per-sample pseudo-label noise rather than filtering it (HumanScale) or splitting network capacity (Li et al.); reports SOTA on RoboCasa GR1 TableTop and RoboTwin 2.0 with strong real-world bimanual transfer.
  • [2026-06-30] World Pilot: Steering Vision-Language-Action Models with World-Action Priors: World Pilot (Lin et al., CASIA) adds a dual-path WFM-as-policy-peer lever to the VLA recipe board: a Latent Steering path injects a scene-evolution latent into the VLA’s perception layer, and an Action Steering path supplies an anticipated trajectory as a motion prior to the action generator — both pulled from a separately-trained World-Action Model. Reports 84.7% Total on LIBERO-Plus zero-shot OOD (SOTA at submission) with largest margins on viewpoint / geometry / deformable / pose shifts; critically, the perception-side latent prior remains effective when the supplying WAM is video-pretrained-only and never action-post-trained, decoupling the WFM training data from the policy’s action data. Sibling to µ₀’s frozen-trace WAM + action expert and ImageWAM’s KV-cache injection, but the only filed instance that steers on both perception and action sides simultaneously.
  • [2026-06-30] GeneralVLA-2: Geometry-Aware Reconstruction and Governed Memory for Robot Planning: GeneralVLA-2 (AIGeeksGroup) ships two targeted upgrades to the v1 hierarchical VLA stack — GeoFuse-MV3D, a multi-view geometry-prior-guided SAM3D replacement that addresses pose hallucination on calibrated RGB-D, and a governed-memory KnowledgeBank with explicit quality/confidence/lifecycle/verifier/conflict metadata; perception evaluated on GSO-30 (small lifts vs MV-SAM3D), memory evaluated against ReasoningBank on Terminal-Bench 2.0 (+4.53% SR) and SWE-Bench Verified (+3.73% resolve), positioning governed experiential memory as a portable agent layer rather than a robot-only component.
  • [2026-06-30] Segmenting Robot Video into Actionable Subtasks (WGO-Bench): Macrodata Labs ships an open subtask-annotation pipeline (WGO-Bench + Refiner) that produces the kind of segment-level labels RT-H, π0.5, and SARM consume — Gemini-3.5-Flash on contact sheets with visual-timestamp cues reaches 0.306 segmentation F1 and runs at $2.64/hr (~19× cheaper than humans) across egocentric and robot-camera videos; directly addresses the “how do we mine subtask labels at scale” question several filed VLA papers leave open.
  • [2026-06-29] Flexion Reflect v1.0: The Path Towards Long-Horizon Autonomous Humanoid Work: Flexion’s Reflect v1.0 wraps a teleop-trained dexterous-manipulation VLA inside a separately-fine-tuned mission VLM (RL on top of SFT lifts 16-step mission completion 38% → 90%) and a library of RL-trained motion skills — concrete deployment evidence for the VLA-as-tool framing VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation proposed, and Flexion explicitly flags the dexterous-VLA layer as the brittlest, with plans to move it to RL.
  • [2026-06-29] CHORD: Object-Centric Contact Wrench Guidance for Dexterous Manipulation: CHORD (NVIDIA Isaac) takes the sensorimotor counter-position one level finer than ConTrack: not just track the object trajectory, match the 6-D contact wrench (force + torque on object) the human demonstration induced. Reports 82.12% across 1,831 dexterous tasks and the cleanest filed evidence that this causal abstraction over contact transfers across kinematics — 90.77% on whole-body humanoid manipulation learned from hand-only / third-person human demos.
  • [2026-06-29] SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation: SimFoundry evaluates 5 VLA / generalist manipulation policies (DreamZero, Gr00t N1.7, π0.5, …) on the DROID platform with sim-trained-only deployment, and reports zero-shot transfer to multi-step, articulated, and bimanual real tasks — concrete sim-to-real data point for the VLA evaluation surface beyond RoboArena’s real-world A/B comparisons.
  • [2026-06-29] World Value Models for Robotic Manipulation: WVM (ByteDance Seed) is the world-model-backbone counterpart to RECAP’s VLM-backbone distributional value head (π*0.6: a VLA That Learns From Experience (RECAP)): a separate, generalist, recipe-agnostic value model designed to plug into multiple policy-extraction approaches across embodiments and mixed-quality data. Argues VLM-backbone value models structurally underfit temporal grounding + future planning that value estimation requires, and introduces hesitation RMSE on the new Suboptimal-Value-Bench as the first filed metric quantifying how well a value/reward model localizes suboptimal segments — a measurement gap previous VLA papers (Spirit-v1.5, Embodied-R1.5, LingBot-VLA) have implicitly punted on.
  • [2026-06-29] In-Context World Modeling for Robotic Control: ICWM (Fudan NLP) — the arxiv paper behind the HuggingPapers tweet 2070794187063099893 — In-Context World Modeling for Robotic Control (ICWM) stub, now resolvable. Frames camera/morphology generalization as in-context system identification: short self-generated, task-agnostic interaction prefix is fed into the policy context window so the model implicitly infers world dynamics before task execution, no parameter updates required. Structurally distinct from the four existing levers on this page (action-pretraining, clean-teleop, unified-VLM pointing, frozen-WFM+action-expert) — puts the generalization lever in the test-time context window rather than in pretraining; reports significant gains over standard VLA baselines on novel camera viewpoints in sim and on real robots (no numbers in the abstract).
  • [2026-06-29] Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack: Tencent Hunyuan’s HyVLA-0.5 technical report — third major industrial VLA stack (after Qwen-Robot Suite and π*0.6) spanning data collection, model design, continued pre-training + SFT, RL post-training, and real-world deployment, joining the action-pretraining-plus-RL recipe family; only abstract retrievable at filing time.
  • [2026-06-29] PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation: PhysisForcing shows that physically-aligned video world models yield stronger representations for downstream policies, raising closed-loop success on the WorldArena action-planner protocol from 16% → 24% and improving downstream VLA policy success.
  • [2026-06-28] HuggingPapers tweet 2070794187063099893 — In-Context World Modeling for Robotic Control (ICWM): HuggingPapers tweet stub for “In-Context World Modeling for Robotic Control” (ICWM) — claims zero-shot adaptation to unseen cameras and new morphologies via in-context conditioning rather than pretraining scale or embodiment-invariant interface; underlying paper not linked from the tweet and not locatable at filing.
  • [2026-06-28] UMI-Bench 1.0: An Open and Reproducible Real-World Benchmark for Tabletop Robotic Manipulation with UMI Data: UMI-Bench 1.0 — first real-world benchmark dedicated to UMI-style wrist-camera manipulation policies, with a local-first auditable protocol and seen/unseen splits across 9 tabletop tasks; complements RoboArena’s distributed crowd-sourced approach.
  • [2026-06-26] Survey notes on World Action Models / VLA for robotics (index): Luma-internal survey-notes index (Dong Guo + co-author) that organizes the WAM/VLA space by directions (e.g. video-as-policy and policy-first / data-flywheel) with per-direction capability-boundary commentary — the strongest internal synthesis filed so far, complementing the public Reuss WAM taxonomy (Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models) and the World Labs functional taxonomy (A Functional Taxonomy of World Models).
  • [2026-06-26] Evo-RL: Open Real-World Offline RL on SO-101 and AgileX PiPER: Evo-RL (SJTU + Evo-Tech) is the first filed open-source community implementation of the π*0.6 RECAP recipe on commodity hardware — SO-101 and AgileX PiPER — with --value.type=pistar06 as the default value function and an advantage-conditioned indicator injected as text into the policy task prefix; ports the closed-loop iteration directly into the LeRobot CLI and pairs it with the open RW-RL dataset.
  • [2026-06-26] ACT in the browser — three.js SO-101 + leader-arm teleop + trained model running locally on the web: @pham_blnh demos the full lerobot ACT loop running in a browser — three.js SO-101 scene + real leader-arm teleop + ACT training + in-browser inference at lerobot.binhph.am — extending the mjswan: real-time interactive MuJoCo + ONNX policy simulation in the browser “policy in the browser” pattern to cover data collection and training, not just deployment.
  • [2026-06-25] ZeroDex: Zero-Shot Long-Horizon Dexterous Manipulation via Multi-View 3D-Grounded VLM Reasoning: ZeroDex (SNU) stakes out a fifth lever beyond action-pretraining, clean-teleop, unified-VLM-pointing, and frozen-trace-WM+action-expert: a zero-shot training-free recipe where a frozen VLM produces reference-view 2D keypoints, multi-view fusion (triangulation + reference-view ray voting) lifts them to 3D, and an atomic-action library handles execution. Reports beating fine-tuned VLA baselines on novel scenes for both pick-and-place and tool-use, with closed-loop replan for long-horizon tasks.
  • [2026-06-25] RAIDEN: A Toolkit for Policy Learning with YAM Bimanual Robot Arms: RAIDEN (TRI) is the first standalone MIT-licensed end-to-end VLA data-collection toolkit on file — calibration, leader-follower / SpaceMouse teleop, heterogeneous-camera mixing (ZED + RealSense), four depth backends incl. Fast Foundation Stereo, manipulability-aware IK via PyRoki/J-Parse, and WebDataset shard output; sits upstream of every VLA training recipe tracked here and is the standalone version of the data-pipeline plumbing ABC released as part of its own stack.
  • [2026-06-24] VLA에게 부족한 결정적 감각 | 로봇에게 Force·Tactile이 반드시 필요한 이유 (Why VLAs Need Force and Tactile Sensing for Robot Manipulation): First filed survey of the VLA + force/tactile design space — a Korean robotics-engineering channel explainer that groups recent attempts into three architectural patterns (force injected into the model, tactile deeply fused with vision/language, VLA wrapped with tactile feedback around planning/control) plus a touch-as-language framing where touch conveys softness/hardness/slip/pressure/misalignment that aren’t visible in RGB. Useful taxonomy crosscut for the open question on tactile/contact integration that the page already lists.
  • [2026-06-24] Next Forcing: Causal World Modeling with Multi-Chunk Prediction: Next Forcing trains a causal-AR visual-dynamics core for VLAs using multi-chunk prediction heads (MTP transferred from LLMs); benchmarks directly against LingBot-VA on RoboTwin and reports +93.1% relative gain at 5k steps with 2.3× faster convergence at 50 fps.
  • [2026-06-23] ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?: ImageWAM (SJTU + EIT) argues video generation is the wrong prior for WAMs and replaces it with pretrained image editing consumed via KV cache; flow-matching action expert reads the editing denoiser’s caches rather than a decoded target frame, claiming to beat VLA baselines and competitive WAMs at ~1/6 FLOPs and ~1/4 latency without policy pretraining — a fifth WAM/VLA recipe alongside video-rollout-as-policy (DVA), frozen trace-WM + small action expert (µ₀), unified causal flow-matching (LingBot-VA), and clean-teleop scaling (Spirit v1.5).
  • [2026-06-23] GPS: Geometric Primary Structure for Articulated Parts Perception in Robot Manipulation: GPS (SJTU, CVPR 2026 Findings) is the strongest evidence yet for a perception-rich + heuristic-policy counter-recipe to end-to-end VLAs: trains a single PointNet++ predictor of a 3-keypoint articulated-axis abstraction on 41K VR-annotated frames, then uses it to re-score AnyGrasp proposals and roll axis-constrained waypoints — 73% real-robot success across 270 trials on 9 unseen objects with no in-domain fine-tuning, vs 33% for pose-based CAPNet and 35% for flow-based GFlow.
  • [2026-06-23] Causal World Modeling for Robot Control (LingBot-VA): LingBot-VA (Robbyant/Ant Group, third LingBot paper) unifies the world-model and action-decoder of the prior LingBot-VLA and LingBot-World releases into a single causal autoregressive flow-matching model over interleaved video+action tokens, processed by a dual-stream MoT (Wan2.2-5B video stream + asymmetric narrower action stream). Two transferable contributions: Noisy History Augmentation (training the action decoder on partially noisy video latents lets inference skip half the denoising steps) and an FDM-grounded asynchronous pipeline (refreshes cached forecasts against the latest real observation before predicting the next action chunk, fixing the open-loop drift that naïve async exhibits). Counter-recipe to π*0.6’s action-pretraining + Embodied-R1.5’s pointing-centric VLM by leaning on unified world+action modeling rather than separate components.
  • [2026-06-22] Inside NVIDIA Halos for Robotics: A Full-Stack Functional Safety System for Physical AI: NVIDIA Halos for Robotics — the first filed wiki artifact addressing the deployment-time substrate (IGX Thor + IEC 61508 SIL 3 Functional Safety Island + Halos OS) that VLAs would have to integrate with for industrial certification; Outside-In Safety Blueprint’s OOD-detecting Safety AI Monitor with mandatory fall-back to onboard safety is a concrete production answer to VLM/VLA perception-failure modes that purely capability-centric papers leave open
  • [2026-06-21] HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos: HumanEgo (UMD) proposes a fourth lever in the egocentric-vs-teleop debate alongside HumanScale (filtering), ActiveMimic (viewpoint action), and Li et al. (per-embodiment specialization): entity-relative Interaction-Centric Tokens plus three free auxiliary forecasting heads (3D object motion, 2D image-plane trace, latent consistency) that act as an in-policy world model — and is the only filed recipe to claim outperforming matched-time teleop (+41% absolute) at 15–30 minutes of human video per task.
  • [2026-06-20] Tactile-Reactive Dexterous Hand: High-Frequency Physical Interaction: Tactile-Reactive Dexterous Hand directly addresses the open “how to incorporate tactile/contact dynamics into the VLA stack” question — proposes variable-rate Mix-of-Transformer + temporal tactile VQ-VAE encoder to fuse high-frequency touch with vision/language without losing existing VLA capabilities, with 100h of motor-primitive-prioritized teleop data; >30% absolute success-rate gain on 12 delicate-force / deformable-object tasks. Most concrete answer yet to the Malik / Yunzhu-Li sensorimotor counter-position thread.
  • [2026-06-19] EgoInfinity: A Web-Scale 4D Hand-Object Interaction Data Engine for Any-View Robot Retargeting and Video-to-Action Robot Learning: EgoInfinity (Hang group, Rice) — third paper in a single week (with DO AS I DO and ConTrack) arguing the human-video-to-robot pipeline is a credible alternative to teleop-data scaling; distinct framing as a modular engine whose value compounds as upstream perception models improve, with any-view retargeting that tolerates partial body visibility.
  • [2026-06-19] What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?: Li et al. (MIT/UIUC/Improbable AI) stake out a third lever in the egocentric-vs-teleop debate alongside HumanScale (filtering) and ActiveMimic (viewpoint-action): per-embodiment specialization in the vision encoder and policy network, plus 3D hand-pose label quality from multi-view triangulation. Releases TriHands (532 cooking videos / 28h, triangulated 3D hands) and reports +29.7% absolute success in the low-robot-data cotraining regime — and finds that accurate hands alone are insufficient if the motion gap isn’t absorbed by per-embodiment capacity.
  • [2026-06-19] Learning What to Say to Your VLA: Mostly Harmless VLA Steering: Language Feedback Policy (Jeong/Swamy/Bajcsy, CMU CoRL submission) — leaves the VLA frozen and steers it through test-time language rewrites discovered by interactive search, then distilled into an LFP with a conformalized improvement head that refuses OOD interventions; reports +24.7% sim / +65.0% Franka hardware over base π0.5 and matches VLA-SFT at ~1/5 the data, a black-box-VLA counter-recipe to the action-pretraining / unified-VLM / clean-teleop positions already on the page.
  • [2026-06-19] ConTrack: Constrained Hand Motion Tracking with Adaptive Trade-off Control: ConTrack (UCSD; Xiaolong Wang group, ECCV 2026) attacks the kinematic-gap problem at the sensorimotor level the Malik counter-position calls out: treats object tracking as a hard RL constraint and uses a dual-variable update to reallocate authority between object fidelity and hand-style fidelity online, eliminating per-clip reward tuning. Complements Do as I Do: Dexterous Manipulation Data from Everyday Human Videos (kinematic reconstruction + retargeting) as the closed-loop-control half of the same dexterous human-video → robot pipeline.
  • [2026-06-19] HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining: HumanScale supplies the sharpest quantitative datapoint to date in the egocentric-vs-teleop pretraining debate: at matched data volume, egocentric pretraining beats teleop by 24% on validation loss, 52.5% on in-distribution real-robot success, and 90% on OOD success — the OOD>ID gap directly supports the “diversity is the lever” interpretation that HumanNet and EgoVerse argued qualitatively.
  • [2026-06-19] MolmoMotion: Language-guided 3D motion forecasting: MolmoMotion proposes a third pretraining target for VLA backbones — language-conditioned 3D motion forecasting rather than action prediction (π*0.6) or pointing/grounding (Embodied-R1.5) — and reports ~20-point gains in MolmoBot pick-and-place success (76.3 vs 56.0) and 5–6× sample efficiency when the same downstream policy is initialized from MolmoMotion vs. the Molmo 2 backbone it was built on.
  • [2026-06-18] PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images: PressureVision++ — vision-only fingertip pressure estimation supervised by prompted contact labels rather than instrumented surfaces; a concrete (if constrained) attempt at the tactile/contact perception axis that the Malik counter-position argues VLAs under-weight, surfaced in a Slack thread questioning whether force/torque/contact inputs are scalable enough to plug into a VLA.
  • [2026-06-18] ActiveMimic: Egocentric Video Pretraining with Active Perception: ActiveMimic argues the human-video-to-robot gap is a signal-design problem rather than a scale problem: recovering synchronized camera + wrist trajectories from a single body-worn RGB stream and treating camera motion as a learned viewpoint action during egocentric pretraining is reported to match robot-data-pretrained baselines on real-world tasks — a sharper outcome than concurrent egocentric data papers’ qualitative “more data helps” findings.
  • [2026-06-17] ABC: Scalable Behavior Cloning with Open Data, Training, and Evaluation: ABC ships matched architectural ablations across the two dominant VLA design points — a 2B DiT with tiny vision encoder (1.93B head / 85.7M DINOv3) and a 4.3B Gemma-3 VLM with a 45M action head — and reports that DiT-head scaling wins per-FLOP, while the lopsided VLM/head split lets one VLM forward be amortized across k diffusion draws. Counter-recipe to π*0.6’s RL-on-flow-matching and Embodied-R1.5’s pointing-centric VLM, and a direct sibling to Spirit v1.5’s Qwen3-VL+DiT recipe with cleaner scaling evidence.
  • [2026-06-15] μ₀: A Scalable 3D Interaction-Trace World Model: µ₀ argues VLAs should be factored as a frozen action-free 3D-trace world model + a small embodiment-specific action expert, not an end-to-end VLM-with-action-head. The headline claim is that trace-conditioned policies built this way are competitive with π₀ on downstream manipulation despite µ₀ never seeing an action label — a structural counter-recipe to both the action-pretraining position (π*0.6) and the unified-VLM position (Embodied-R1.5).
  • [2026-06-15] VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation: VoLo reframes the VLA-design debate by wrapping a VLA/WAM as an interruptible tool under a VLM orchestrator (VoLoAgent) that monitors and steers mid-rollout, rather than treating the VLA as a frozen end-to-end policy — and reports substantial gains over single-VLA and single-VLM systems on a new long-horizon open-vocabulary benchmark (RoboVoLo), with real-robot validation.
  • [2026-06-14] LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World: LEGS demonstrates that VLAs (π0, π0.5, GR00T N1.6) fine-tuned purely on 3DGS-backed synthetic data match or exceed human teleoperation across 1,110 real Unitree G1 trials — a data-side counter to Spirit-v1.5’s “clean teleop is the lever” position on the same backbone family.
  • [2026-06-11] Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models: Unified 8B Embodied Foundation Model + VLA — argues that strong VLM-level pointing/grounding (Planner-Grounder-Corrector loop on a single model) substitutes for action pretraining; reports beating π0/π0.5/SpatialVLA/OpenVLA across SimplerEnv suites and matching the best-action-pretrained baselines on LIBERO.
  • [2026-06-06] Jitendra Malik: don't let CV researchers in robotics skip the sensorimotor level: Counter-position arguing the VLA framing under-weights tactile / contact / sensorimotor dynamics and that CV researchers entering robotics shouldn’t skip the sensorimotor level.
  • [2026-05-25] π*0.6: a VLA That Learns From Experience (RECAP): π*0.6 / RECAP — first filed flow-matching VLA trained end-to-end with offline RL on real-robot data via CFGRL-style advantage conditioning.

Open questions

Section titled “Open questions”
  • Does the unified-VLM PGC recipe (Embodied-R1.5) actually scale to the long-horizon real-world deployment regimes π*0.6 targets (13-hour espresso sessions, factory packaging) or does it remain a benchmark phenomenon?
  • Is the SimplerEnv→LIBERO performance of action-pretraining-free VLAs a data-distribution artifact (both suites being heavily sampled in the embodied-VLM training corpus) or a robust generalization signal?
  • How should tactile / contact dynamics be incorporated into the VLA stack — as an extra modality conditioning the VLM, as a separate low-level controller below the VLA, or via simulation-augmented training?
  • For open VLAs built on Qwen-VL: how much of the gap to closed flagships (Gemini Robotics 1.5) is from the backbone choice vs the policy-training recipe vs the data?
  • Does advantage conditioning generalize beyond flow-matching action heads (e.g. to FAST-tokenized discrete-action VLAs), or is it specifically a workaround for the no-log-likelihood property of flow models?

Papers

Section titled “Papers”

2026-06

Section titled “2026-06”

2026-05

Section titled “2026-05”