Arm-wise Compositional Generalization in Dual-Arm Vision-Language-Action Models

TL;DR

ACG-Bench reframes dual-arm manipulation generalization as arm-wise compositional generalization — recombining per-arm atomic skills across arms under unseen reorder / synchronization / cross-task coordination constraints — and ships a controlled 23-task-condition benchmark (6 in-domain + 17 held-out compositions) with every method receiving the same per-arm atomic prompts from a common rule-based phase scheduler. AE-VLA builds on a shared π₀.₅ backbone with three composable architectural choices (arm-token grouping, SkillLoRA bank-of-10 skill-specific LoRA adapters with per-arm routers, and arm-wise attention that blocks direct cross-arm action attention while preserving shared global context) and reports 21.53% simulation generalization success vs 2.94% for Single π₀.₅ / 3.06% for MA-VLA / 5.53% for two independently controlled π₀.₅ policies, and 39.00% real-SO101 mean success across five unseen conditions vs 10.00% for the strongest baseline. The three design choices work much better together than individually (grouping-only 3.82%, grouping+SkillLoRA 6.00%, grouping+AWA 5.82%, all three 21.53%). Code is Apache-2.0.

Key claims

Arm-wise compositional generalization is a distinct benchmark axis from spatial / object / visual generalization, decomposed along four coordination axes: reorder (4 conditions, which arm acts first), sync (6 conditions, designated events within time tolerance), sync+reorder (4), and cross-task (3, bring together skills learned in different source tasks); success requires satisfying both physical milestones and specified order/timing constraints, with arm collisions counted as failures [§ACG-Bench, Project page].
Keeping the plan supply fixed isolates execution of a composition rather than plan inference: every method receives the same per-arm atomic prompts from a common rule-based phase scheduler, so the benchmark measures compositional execution rather than compositional planning [Project page §ACG-Bench].
Dual π₀.₅ (one policy per arm) does not solve coordination — two independent policies in the same workspace reach only 5.53% generalization success and still under-perform shared-policy AE-VLA by 15.95 pp, because local obstruction and goal-fragment success do not compose [Project page §From sim to real; Abstract].
The three AE-VLA design choices (arm-token grouping, SkillLoRA, arm-wise attention) interact super-linearly: grouping alone 3.82%, grouping + SkillLoRA 6.00%, grouping + AWA 5.82%, all three 21.53% — the combination is load-bearing rather than any single lever [Project page §AE-VLA table].
SkillLoRA is a bank of 10 skill-specific low-rank adapters shared across arms and tasks, with a per-arm learned router that selects an adapter from the arm’s prompt and state features — distinct from monolithic task-specific LoRA finetuning in that the same adapter bank is reused across both arms and across tasks [Project page §AE-VLA].
Arm-wise attention keeps action attention within each arm’s stream while preserving access to shared global context — the global features can still gather observations from both arms, so the policy remains coupled through shared representation while direct cross-arm action attention is blocked [Project page §AE-VLA].
AE-VLA transfers from sim to real with large margins on bimanual-coordination held-out conditions: on real SO-101 robots AE-VLA succeeds 12/20 on Stack Bowls / Sync (all three baselines 0/20) and 10/20 on held-out Cube in Bowl (Dual π₀.₅ 4/20) [Project page §From sim to real].
Known limitations: pure synchronization remains the hardest axis (8.67% simulation), and in-domain success drops relative to Dual π₀.₅ (27.17% vs 41.33% sim; 60% vs 70% real) — generalization gains trade against in-domain peak, and on real Push Cubes / Sync Dual π₀.₅ beats AE-VLA 6/20 vs 2/20 [Project page §Where the approach still falls short].

Method

AE-VLA starts from a shared pretrained π₀.₅ backbone and action expert and introduces structure at three levels. Arm-token grouping gives each arm an explicit group of action tokens that both pass through the same action expert, with each group assigned to its arm’s output — so the model has explicit per-arm action-token identity rather than one undifferentiated action stream. SkillLoRA is a bank of 10 skill-specific low-rank adapters reused across arms and tasks; a learned router per arm selects an adapter from the arm’s prompt and state features, factoring the “which skill is this arm doing right now” question out of the base-weight representation. Arm-wise attention restructures the action-expert attention so each arm’s action tokens attend within their own stream while retaining access to shared global context features that gather observations from both arms — blocking direct cross-arm action attention but keeping the coupling through shared global representation.

ACG-Bench provides the controlled training-vs-held-out split: 6 in-domain task-condition pairs with demonstrations and 17 held-out compositions with zero training data, decomposed along four coordination axes (reorder, sync, sync+reorder, cross-task). Every method sees the same per-arm atomic prompts from a common rule-based phase scheduler, so the comparison isolates compositional execution rather than mixing in plan inference. Simulation evaluation runs 100 episodes per condition per method; real-SO101 evaluation uses 20 trials per condition across five unseen conditions (2 Reorder + 2 Sync + 1 Cross-task).

Results

Headline simulation generalization-success numbers on held-out ACG-Bench conditions: AE-VLA 21.53% vs Single π₀.₅ 2.94%, MA-VLA 3.06%, and two independently controlled π₀.₅ policies 5.53%. The three AE-VLA components compose super-linearly: Token Group 3.82%, Token Group + SkillLoRA 6.00%, Token Group + AWA 5.82%, all three 21.53%.

On real SO-101 across five unseen conditions, AE-VLA reaches 39.00% mean success vs 10.00% for the strongest baseline. Per-task: Stack Bowls / Sync 12/20 vs 0/20 across all three baselines (sharpest filed gap on a synchronization task); Cube in Bowl (cross-task held-out) 10/20 vs 4/20 for Dual π₀.₅.

Trade-off numbers worth tracking: in-domain simulation success is 27.17% for AE-VLA vs 41.33% for Dual π₀.₅ (−14.16 pp); in-domain real success 60% vs 70%; pure synchronization in simulation is only 8.67% — the hardest axis across all methods. On real Push Cubes / Sync, Dual π₀.₅ beats AE-VLA 6/20 vs 2/20, so the gains are not uniform across tasks.

Why it’s interesting

This is the first filed paper on the wiki to measure a VLA on held-out dual-arm coordination compositions under a controlled benchmark where both single-shared-policy and per-arm-policy baselines are run on the same source data — the Dual π₀.₅ baseline at 5.53% directly challenges the “give each arm its own policy” counter-recipe that otherwise seems reasonable. On the recipe-lever board tracked in VLA Models, AE-VLA stakes out a distinct cell from the π*0.6 / Embodied-R1.5 / clean-teleop levers: it keeps a stock π₀.₅ backbone unchanged and puts the generalization lever into a shared-policy architectural structure over the action expert (arm-grouping + per-arm skill router + attention topology) rather than into pretraining scale, RL, or data cleanliness. SkillLoRA with a bank-of-10 adapters routed per arm from prompt+state features is a distinctive instance of the hypernetwork-generated / routed-adapter direction in Parameter-Efficient Finetuning — reused across arms and tasks rather than being task-specific, directly in the spirit of the LoRWeB learned-basis-plus-router pattern. The sharpest contrast is with Empirical results fine-tuning π0.5 on a real manufacturing task‘s finding that LoRA on π₀.₅ underperforms full FT by 23–43 pp on a new embodiment — AE-VLA’s SkillLoRA bank + router operates at a different granularity (skill-specific, not task-specific; applied on top of a trained shared base rather than as a replacement for FT) and the super-linear composition with grouping + arm-wise attention suggests the LoRA is doing representation routing, not full embodiment adaptation.