Skip to content
Bud's Wiki

TemporalFlow-VLA: Learning Physically Grounded Execution History for Long-Horizon Robot Manipulation

TL;DR

Section titled “TL;DR”

TemporalFlow-VLA augments a Vision-Language-Action model with a physically grounded representation of recent execution history by supervising two execution-aligned temporal query tokens against a robot-surface temporal flow target that is constructed offline from recorded robot states, robot geometry, and calibrated cameras. The geometric-supervision path is training-only — at deployment the queries pass structured history to the action expert without any motion estimation or geometric processing. Reports 97.63 ± 0.26% average success on LIBERO (96.60 ± 0.87% on LIBERO-Long) and 85.5% / 84.2% Clean/Randomized average across 12 RoboTwin tasks, with the clearest margin over prior methods on multi-stage manipulation. Asynchronous feature caching keeps server-side sampling latency at single-frame level despite the added temporal conditioning.

Key claims

Section titled “Key claims”
  • Naïvely stacking historical frames does not reliably capture recent physical change, which is the failure mode this paper targets in multi-stage manipulation where visually similar states demand different actions depending on prior execution [Abstract].
  • Robot-surface temporal flow — constructed from recorded robot states + robot geometry + calibrated cameras — is a viable training-only supervision target for a compact execution-history representation [Abstract].
  • Two execution-aligned temporal queries, supervised by that flow target, provide structured history to the action expert without any geometric processing at deployment [Abstract].
  • 97.63 ± 0.26% average success on LIBERO overall, including 96.60 ± 0.87% on LIBERO-Long [Abstract].
  • 85.5% Clean / 84.2% Randomized average success across 12 RoboTwin tasks [Abstract].
  • Controlled history-intervention experiments show action prediction depends on both historical content and temporal order (i.e. the queries encode ordered rather than bag-of-frames history) [Abstract].
  • Asynchronous feature caching lets temporal conditioning add no server-side per-frame overhead beyond single-frame sampling latency [Abstract].

Method

Section titled “Method”

The paper (per the abstract and title; full body not retrieved beyond the abstract page) adds two components on top of a base VLA. First, a robot-surface temporal flow target is precomputed offline for each training clip using the recorded robot joint states, the known robot geometry, and calibrated cameras — the flow field describes how the robot’s surface projects and moves in the camera frame, without needing to estimate motion from pixels. Second, two execution-aligned temporal queries attached to the model are supervised against that target as an auxiliary objective. At deployment neither the flow field nor the geometric pipeline is used: the queries are read directly by the action expert. Asynchronous feature caching decouples the temporal query computation from the action-generation critical path, so per-frame server-side latency stays comparable to a single-frame VLA.

Results

Section titled “Results”
  • LIBERO overall: 97.63 ± 0.26% average success; LIBERO-Long: 96.60 ± 0.87% [Abstract].
  • RoboTwin (12 tasks): 85.5% Clean, 84.2% Randomized average success [Abstract].
  • Clearest headline advantage over prior methods on longer-horizon, multi-stage manipulation tasks specifically [Abstract].
  • Controlled history-intervention analysis reports that action prediction is sensitive to both content and temporal order of history [Abstract].
  • Server-side sampling latency is reported as single-frame-level under the async feature-caching design; no explicit ms number is quoted in the abstract [Abstract].

Numerical breakdowns beyond the aggregate LIBERO/RoboTwin numbers are not extractable from the retrieved abstract-only surface.

Why it’s interesting

Section titled “Why it’s interesting”

TemporalFlow-VLA stakes out a distinct point in the VLA-with-history design space that the wiki already tracks. StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models StreamPI grafts streaming history onto π0.5 via an instruction-anchored attention mask + interval-randomized training with zero new parameters and no new supervision signal; TemporalFlow-VLA instead keeps the base VLA architecture largely alone but adds a training-only physical-motion supervision target that the queries are pinned to. This is closer in spirit to MEM: Multi-Scale Embodied Memory for Vision Language Action Models MEM — which explicitly represents multi-scale memory but pays for it with dual-timescale architecture — and to BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation BridgeVLA++‘s in-architecture memory module, except the load-bearing move here is the supervision signal rather than the architecture. The reported edge on multi-stage manipulation and the ordered-history controlled ablation are directly the failure modes those recipes also target. The “useful for sidecars” framing in the sharing note aligns with the deployment-time property that the geometric supervision path is dropped at inference — the temporal-query head could plausibly slot next to a frozen VLA as a small auxiliary conditioning module. The LIBERO-Long number (96.60%) is competitive with recent single-frame + memory-recipe reports, but without shared numerical head-to-heads on the same subset in the abstract it’s hard to rank against StreamPI or MEM directly.

See also

Section titled “See also”