Adaptive Action Chunking at Inference-time for Vision-Language-Action Models

TL;DR

AAC is a training-free inference-time strategy that picks the action-chunk size per step for a diffusion-/flow-matching-based VLA by looking at action entropy over a batch of parallel candidate chunks: use a large chunk when predicted actions are confident (low entropy), a small chunk when they are uncertain. The chunk size is chosen as the maximum differential point of the average-entropy curve subject to a minimum chunk-length floor, with translation/rotation modeled as continuous Gaussian differential entropy and gripper as discrete entropy. Applied on top of GR00T N1.5 and π0.5, AAC beats every fixed chunk size on RoboCasa (24 kitchen tasks) and LIBERO (four suites), lifts 3 real-world Realman single-arm tasks 67% → 82%, and holds under OOD position perturbation and across sample counts, at ~20 ms extra latency on an A800 with 20 samples.

Key claims

No single fixed chunk size is best across tasks or across time within a task — success on RoboCasa varies non-monotonically with chunk size, and different LIBERO suites prefer different sizes (Spatial best at 4, Goal best at 16) [§1, Fig. 1, Table 1].
Action entropy over a batch of parallel candidate chunks is usable as a cue for chunk-size selection: for continuous translation/rotation use Gaussian differential entropy from the sample covariance; for the discrete gripper channel use the empirical probability of the close-state [§4, Eq. 2–4].
The optimal chunk length is chosen at inference-time as the maximum differential point of the average-entropy-vs-chunk-size curve, floored by a minimum size $L$ to bound reactivity and per-call compute [§4, Eq. 5, Algorithm 1].
AAC is architecture-agnostic — it needs no additional training or architectural modification and layers on any diffusion-/flow-matching action head [§1, §4].
On RoboCasa (100 rollouts/task × 24 tasks), AAC lifts GR00T N1.5 average success from 59.7 (default L=16) → 62.0, with the largest gain on the Rotation subset (57.6 → 61.4) [Table 1].
On LIBERO (50 rollouts/task × 40 tasks), AAC lifts GR00T average 94.1 → 95.0, with the largest gain on the long-horizon LIBERO-Long suite (88.8 → 92.8) [Table 1].
AAC generalizes across VLA backbones — applied to π0.5 on LIBERO it lifts 97.0 → 97.9 average and 92.5 → 95.2 on LIBERO-Long [Table 2].
AAC holds under OOD position perturbation on LIBERO-Pro: GR00T 3.9 → 6.3 average, π0.5 30.9 → 34.8 [Table 3].
Sample-count sweep shows accuracy is near-saturated at 20 candidate chunks (94.1 → 95.0 → 95.5 as samples go 1 → 20 → 40) while latency scales sub-linearly thanks to batched parallel decoding — 83 ms → 106 ms → 157 ms on a single A800 [Table 4].
On 3 real-world Realman single-arm tasks (banana pick-and-place, E-Button pressing, long-horizon drawer close), AAC lifts GR00T average 67 → 82% over 20 trials/task and specifically reduces tabletop-collision failures by filtering high-entropy actions [§5.2.3, Table 5, Fig. 6].
Empirical chunk-size trajectory tracks task phase: large chunks during transport, small chunks around the grasp/manipulation phase, then large again during placement — a learned “human-intuitive” schedule from the entropy cue alone [§5.1.4, Fig. 3–4].

Method

At each inference step, AAC samples $N=20$ candidate action chunks (each of horizon $H$ ) in parallel from the pretrained flow-matching action head. Per-timestep action entropy is computed with three components matching a 7-DoF arm + gripper: 3-dim Gaussian differential entropy on translation offsets, 3-dim Gaussian differential entropy on rotation offsets, and 1-dim discrete entropy on the close/open gripper decision. The three per-timestep entropies are summed then cumulatively averaged over chunk length $L$ , giving a curve $\bar H(L)$ . The chosen chunk size is the maximum differential point of this curve subject to a minimum floor $L_{\min}$ that guarantees a non-trivial commitment; the first $L^\star$ actions are executed before the next inference call. Nothing about training changes — the pretrained GR00T N1.5 / π0.5 checkpoint is used as-is. The three baselines it is compared against inside the same policy are (i) vanilla fixed $L=16$ , (ii) sweeps over $L \in \{2,4,8,12\}$ , and (iii) BID / RTC / TV-BID style test-time chunk-selection methods that still fix $L$ . Real-robot fine-tuning uses 50 human-teleoperated demos per task on 8×A800 for 300 epochs; sim experiments use 100 MimicGen demos on RoboCasa and OpenVLA/π0.5-preprocessed data on LIBERO.

Results

RoboCasa Kitchen (24 tasks, 100 rollouts each): GR00T+AAC 62.0 vs default 59.7 and every fixed $L$ (47.0–61.2); Rotation subset 61.4 vs 57.6 default; Button subset 81.3 vs 80.3 default (near-ceiling) [Table 1].
LIBERO (4 suites, 40 tasks, 50 rollouts each): GR00T+AAC 95.0 vs 94.1 default; LIBERO-Long 92.8 vs 88.8 default [Table 1].
π0.5 backbone transfer: LIBERO 97.0 → 97.9 avg with AAC; LIBERO-Long 92.5 → 95.2 [Table 2].
LIBERO-Pro OOD perturbation (levels 0.2 / 0.3 / 0.4): GR00T avg 3.9 → 6.3 with AAC; π0.5 avg 30.9 → 34.8 [Table 3].
Sample count vs speed on A800 (single GPU): 20 samples ≈ +23 ms over 1 sample (83 → 106 ms), and success 94.1 → 95.0. At 40 samples: 157 ms / 95.5 [Table 4].
Real-world Realman single-arm (20 trials/task): banana 70 → 90, E-Button 65 → 75, long-horizon drawer 65 → 80; average 67 → 82 [Table 5].
Qualitative: chunk-size distribution vs timestep tracks task phase (large during transport, small during grasp/press, large during placement) [Fig. 3–4]; on the banana task AAC halts smoothly above the tabletop where vanilla GR00T over-descends and collides [Fig. 6].

Why it’s interesting

Sits at the intersection of two threads on the wiki. It is a new entry on the VLA Models recipe-lever board — specifically, it converts the fixed-chunk-size decision that BID, RTC, TV-BID, and SGAC all still parameterize as a hyperparameter into an inference-time adaptive quantity, without training. This is a strict sibling to Real-Time Robot Execution with Masked Action Chunking REMAC and πR²: Reactive Real-time Flow Policies πR², which both keep chunk size fixed and instead patch the async-inference execution model; AAC is orthogonal and could in principle be layered on top of either. It is also a distinct answer to the mechanism question posed by Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control? “Why does action chunking improve BC?” — that paper argues chunking helps mainly via non-Markovian expressivity + implicit ensembling, and would predict AAC’s improvements come from the implicit ensembling half (AAC exploits N=20 parallel candidate chunks that would otherwise be discarded). On Inference-Time Scaling this adds a robotics-specific entropy-guided per-step adaptive commitment primitive alongside RL²-VLA’s per-state activation gate and Learning to Fold’s Thompson-sampling bandits over multiple inference-time knobs — a distinct “how much to commit each step” axis, cheap enough to plug in at 20 ms extra latency.