Revisiting Open-Loop Execution in Robotics: Toward Reactive, Higher-Performing Policies

ListenThe Chunk Isn't the Point9:18

Show notes

This week we dig into a diagnostic study from MIT CSAIL and Toyota Research Institute arguing that long open-loop action-chunk execution — a default across the whole VLA lineage — mostly exists so short-context policies can imitate non-Markovian human demonstrations. Under matched ablations across four sim and two real tasks, the authors find compounding-error mitigation is a weaker effect than the field assumes, and that extending observation context while shrinking the execution horizon to single-step beats every chunk baseline they tested. We unpack the mechanism, the ablations, and what a "long context, short execution" inversion could mean for generative policy recipes. **Sources** - Revisiting Open-Loop Execution in Robotics: Toward Reactive, Higher-Performing Policies (Zeng, Agarwal, Bati, Lee, Ancha, Tedrake)

Subscribe to the channel: https://podcast.lumalabs.link/27a25f51ea81d41a8253f46c6133a35e/feed.xml

TL;DR

Zeng, Agarwal, Bati, Lee, Ancha, and Tedrake (MIT / Toyota Research Institute) argue that long open-loop action-chunk execution — the near-universal default in modern imitation learning for manipulation — is primarily a workaround for non-Markovian expert demonstrations combined with short-context policies, not a fundamental property of action chunking. Across four simulation tasks and two real-world tasks they hold architecture and data fixed and vary (a) how non-Markovian the demonstrator is and (b) how much observation context the policy sees. When the expert is Markovian, execution horizon has almost no effect; when the expert is non-Markovian and the policy is short-context, long open-loop horizons help by letting the policy imitate history-dependent behavior it cannot represent step-by-step. But when the policy is long-context, the relationship flips — the most reactive, closed-loop policies now perform best, and long open-loop execution hurts. Compounding-error effects exist but are much smaller than the expert-non-Markovianity effect in their setting. The prescription: give the policy more history and shorter action horizons, not the other way around.

Key claims

Long open-loop execution in behavioral cloning is primarily explained by short-context policies attempting to imitate non-Markovian expert demonstrations, not by compounding-error mitigation, inference-latency absorption, or motion smoothing as prior work has claimed [Abstract, Introduction].
Expert non-Markovianity strongly shapes the relationship between task success and open-loop execution horizon; the effect is visible across four simulation tasks and two real-world tasks under controlled experiments that hold everything else fixed [§Experiments].
Compounding errors — the dominant explanation in prior work — do matter but have a much smaller impact than expert non-Markovianity in the experimental setting evaluated here [§Analysis].
When policies are provided with a sufficiently long context window, open-loop execution is no longer beneficial and the most reactive closed-loop policies perform best [§Long-Context Experiments, Abstract].
The empirical prescription is a paradigm shift: long-context + short-action-horizon (reactive) policies are a more principled and performant paradigm than short-context + long-open-loop-execution (the current default) [Conclusion].

Method

The paper is a diagnostic-and-prescriptive study of the action-chunking design axis rather than a new architecture proposal. Zeng et al. instrument standard imitation-learning setups so they can independently vary (i) the expert’s degree of non-Markovianity (constructed by conditioning the demo policy on more or less history when generating training data) and (ii) the learned policy’s observation context length, then sweep the open-loop execution horizon $k$ over each combination while holding architecture, data volume, and training recipe fixed. The framework recasts three orthogonal design choices — how history-dependent the demonstrator is, how much history the policy sees, and how much of the predicted chunk gets executed before replanning — as knobs that can be turned independently, so an effect attributed to one can be tested against the others.

Evaluation spans four simulation benchmarks and two real-world tasks; the project page frames the load-bearing empirical move as showing that as training data and policy context grow, the optimum horizon $k$ slides from long-open-loop to short/closed-loop, evidence that the field’s convergence on long chunks reflects the low-data / short-context regime the field was operating in rather than a universal property of imitation learning.

Results

Under a Markovian expert, execution horizon has minimal effect on task success — the standard “action chunking always helps” claim does not survive an ablation that removes expert non-Markovianity [§Experiments, per project-page framing].
Under a non-Markovian expert with short-context policy, long open-loop horizons help substantially (recovering the standard observation) — this is the regime prior work implicitly measured in [§Experiments].
Under a non-Markovian expert with long-context policy, the relationship inverts: the most reactive, closed-loop policies win, and long open-loop execution degrades performance [§Long-Context Experiments].
Compounding-error ablations reproduce a measurable but quantitatively smaller effect than expert-non-Markovianity in the same experimental grid [§Analysis].
Findings replicate on two real-world tasks in addition to the four simulation benchmarks — the pattern is not simulator-specific [§Real-World Evaluation].

Why it’s interesting

This paper is the direct theoretical + empirical follow-up to Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control? (Lazzati, Stachowicz, Chen, Metelli, Wagenmaker, Levine) — that paper decomposed action chunking into non-Markovian expressivity + reduced compounding error plus implicit ensembling, and showed a delayed single-action policy conditioned on stale observations often matches chunked policies. Zeng et al. sharpen the first half of that decomposition into a prescription: if non-Markovianity is what long open-loop execution is really compensating for, then give the policy the history and the chunk becomes unnecessary — indeed harmful, because it sacrifices reactivity. The Slack pointer’s paraphrase is exactly right: the field converged on long chunks under short-context / low-data conditions where long-execution-horizon was a proxy for missing history, and as data + context grow, the optimum shifts toward long-context + short-chunk reactive policies.

This is a sharp complement — and empirical validator — to Simulation Pre-training for Dexterity (SPD) SPD’s w=32, c=8 finding: SPD independently found that a 32-step history window + 8-step chunk beats the π0-style w=1, c=32 design choice on real-world contact-rich dexterous tasks, with pre-training gains concentrating in the history-conditioned regime. Where SPD reports the phenomenon as a design-space datapoint, this paper isolates the underlying mechanism (imitating non-Markovian experts) and predicts it should generalize.

It also reframes a whole cluster of recent VLA async-inference recipes tracked on VLA Models — Real-Time Robot Execution with Masked Action Chunking REMAC, πR²: Reactive Real-time Flow Policies πR², Adaptive Action Chunking at Inference-time for Vision-Language-Action Models AAC, A2A: Action-to-Action Flow Matching A2A — as patches to a design choice (long open-loop chunks) that may not be load-bearing at all once the policy has enough context. Terrance’s Slack note flags this as a Luma-actionable finding: shifting the training recipe toward long-context short-chunk policies should be tried directly.