TemporalFlow-VLA: Learning Physically Grounded Execution History for Long-Horizon Robot Manipulation

Jiarui Yang; Yehao Lu; Yuning Su; Yu Zhong; Yufeng Xie; Yazhou Zhang; Haiyu Lan; Kaixiang Lu

cs.ROAug 27, 2026

TemporalFlow-VLA: Learning Physically Grounded Execution History for Long-Horizon Robot Manipulation

Authors: Jiarui Yang, Yehao Lu, Yuning Su, Yu Zhong, Yufeng Xie, Yazhou Zhang, Haiyu Lan, Kaixiang Lu, +4 more

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Zhejiang University, Hangzhou, China. · Simon Fraser University, Burnaby, BC, Canada. · AgiBot, Shanghai, China.

Abstract

Vision-language-action (VLA) models leverage pretrained vision-language representations for robot control, yet simply adding historical frames does not reliably capture recent physical change. This is especially problematic in multi-stage manipulation, where visually similar states may require different actions depending on prior execution. To address this challenge, we present TemporalFlow-VLA, which learns compact execution history through physically grounded temporal supervision. Using recorded robot states, robot geometry, and calibrated cameras, we construct robot-surface temporal flow as a training-only target and supervise two execution-aligned temporal queries that provide structured history to the action expert. The geometric supervision path is not evaluated at deployment. TemporalFlow-VLA achieves 97.63 +/- 0.26% average success on LIBERO, including 96.60 +/- 0.87% on LIBERO Long, and 85.5%/84.2% Clean/Randomized success across 12 RoboTwin tasks. It shows its clearest advantage over prior methods on longer-horizon, multi-stage manipulation. Controlled history interventions show that action prediction depends on both historical content and temporal order. With asynchronous feature caching, temporal conditioning maintains single-frame-level server-side sampling latency without additional historical-encoding overhead. Overall, TemporalFlow-VLA provides a compact, physically grounded interface for exploiting ordered execution history without explicit motion estimation or geometric processing at deployment.

Figures & tables

Fig. 2: History-use diagnostic. The baseline is trained with chronologically ordered history. At evaluation, shuffling the three historical observations while fixing the current frame leaves the offline action flow-matching loss nearly unchanged, whereas removing history increases it by about 4.6%.

Fig. 3: Overview of TemporalFlow-VLA. Robot state, URDF geometry, and calibrated camera parameters generate robot-surface temporal-flow supervision offline. A parallel temporal module compresses historical RGB observations into flow-supervised temporal queries, which are exposed to the action expert through the model’s joint masked-attention operation.

Fig. 4: Directed masked-attention pattern. $Q_{8}$ summarizes short-range history, $Q_{15}$ may additionally read $Q_{8}$ and the longer-range frame, and action tokens access historical visual information only through the two temporal queries.

H.	Simulation Task	Ours		$\pi_{0}$		$\pi_{0.5}$		X-VLA † [ 21 ]		GigaWorld
H.	Simulation Task	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.	Clean	Rand.
1	Place Shoe	97	95	76	76	92	93	96	95	98	96
	Place Phone Stand	79	82	49	53	81	81	88	87	82	72
	Lift Pot	98	98	80	72	96	85	99	100	98	98
	Move Stapler Pad	62	62	41	24	56	42	78	73	92	82
	Average	84.0	84.3	61.5	56.3	81.3	75.3	90.3	88.8	92.5	87.0

TABLE I: RoboTwin 2.0 success rate (%), grouped by execution horizon following LingBot-VA [ 19 ] . The

\pi_{0}

and

\pi_{0.5}

task-level entries are from LingBot-VA (Easy/Hard). X-VLA † and GigaWorld entries are from Table 8 of GigaWorld-Policy [ 20 ] (Clean/Rand.); X-VLA † therefore denotes the RoboTwin re-evaluation reported there rather than the original X-VLA results [ 21 ] . The corresponding fixed/clean and randomized conditions are displayed as Clean/Rand. Best and second-best results are bold and underlined separately within each condition.

Method	Spatial	Object	Goal	Long	Avg.
$\pi_{0.5}$ [ 3 ]	98.8	98.2	98.0	92.4	96.85
GR00T N1.7 [ 23 ]	97.5	98.5	97.5	94.5	97.0
CronusVLA [ 15 ]	97.3	99.6	96.9	94.0	97.0
HAMLET [ 16 ]	99.0	100.0	99.2	92.2	97.7
MotionVLA [ 12 ]	96.2	98.0	96.2	91.2	95.4
Ours	$97.60{\pm}0.20$	$99.40{\pm}0.20$	$96.93{\pm}0.61$	$\mathbf{96.60{\pm}0.87}$	$97.63{\pm}0.26$

TABLE II: LIBERO success rate (%). Ours reports mean

\pm

SD over three seeds; published methods are shown as reported in their sources, with GR00T N1.7 percentages recomputed from the official success counts [ 22 , 23 ] .

Fig. 5: Asynchronous historical-feature caching. Historical observations are encoded during execution and stored in a timestamped feature cache, so replanning retrieves cached history while only the current observation remains on the synchronous inference path.

Fig. 6: Offline history perturbations on six RoboTwin tasks. Both removing and shuffling history increase action flow-matching loss relative to correct history; shuffling is most harmful on five tasks, while put bottles (dustbin) is more sensitive to history removal. The vertical axis is logarithmic.

Clean
Method	$H{=}1$	$H{=}2$	$H{=}3$	Avg.
$\pi_{0.5}$	81.3	76.8	79.0	79.0
Multi-frame	75.5	79.5	84.5	79.8
2Q w/o Flow	83.0	79.8	86.8	83.2
$Q_{8}$ only	86.5	79.8	88.3	84.8
$Q_{15}$ only	84.0	80.5	87.3	83.9

TABLE III: Online RoboTwin ablation by horizon (SR, %). Multi-frame : raw history; 2Q w/o Flow : two queries without flow supervision. Single-query variants use one supervised query. Bold/underline: best/second.

Fig. 7: Asynchronous historical-feature caching on LIBERO Long. Top: per-replan server-side policy sampling latency with and without caching. Bottom: cumulative latency saved over the same 15-replan segment.

Fig. 8: Real-robot evaluation on the AgiBot A3. Left: head/chest views and task setups; the head view provides training-only temporal-flow supervision. Middle: representative three-stage executions. Right: Ours and Baseline over three 15-trial rounds; dashed lines show the 45-trial means.

Explore similar work

Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models
Aug 31, 2026Xingyu Ding, Yuzhong Zhao, Chunhai Zhao +3Representation Learning Robotic Manipulation
Nanjing University, Nanjing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
MotionVLA: Injecting Geometric Motion into Vision-Language-Action Model
Jun 6, 2026Shanglin Yuan, Weiheng Zhao, Xianda Guo +4Long-Horizon Robotic Manipulation Visuomotor Policy Learning
Huazhong University of Science and Technology · D-Robotics · Wuhan University
MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models
Jun 8, 2026Hao Shi, Weiye Li, Bin Xie +6Long-Horizon Robotic Manipulation World Models for Robotics
Department of Automation, BNRist, Tsinghua University, Beijing, China · Department of Computer Science, The University of Hong Kong, Hong Kong, China · Dexmal, Beijing, China +1

Aug 31, 2026J/K move · Enter open · S save

Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models

Xingyu Ding, Yuzhong Zhao, Chunhai Zhao, Yinghuan Shi+2

Nanjing University, Nanjing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

Representation Learning Robotic Manipulation Vision-Language-Action Models Latent Space Alignment

Open paper AskPDF arXiv

Long-horizon robotic manipulation requires vision-language-action (VLA) models to track scene states and their evolution beyond the current observation. However, simply conditioning policies on observation history does not guarantee that the history is effectively utilized: action supervision constrains what the policy should do, but only indirectly constrains what its history representations should retain. To resolve this, we present Temporal Forcing, a 4D representation alignment framework that explicitly supervises latent temporal states and their transitions. Specifically, we first introduce a history pathway that compresses past observations into compact latent tokens. We then align these tokens and current-frame features with geometric targets from a pretrained 4D foundation model, providing direct supervision at both the state and transition levels. The 4D foundation model and alignment heads are used only for training-time supervision. Temporal Forcing improves average success from 96.6% to 98.8% on LIBERO, with the largest gain on LIBERO-Long (93.8% to 97.2%), and from 53.5% to 62.8% across twelve RoboTwin 2.0 tasks. Furthermore, Temporal Forcing increases full-task success from 20.0% to 43.3% on a physical multi-stage hidden-placement task. Controlled experiments show that 4D representation alignment is crucial for making observation history beneficial to the model. Code will be publicly available.

Abstract

Explore similar work

Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models

MotionVLA: Injecting Geometric Motion into Vision-Language-Action Model

MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models