Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining

Yipeng Chen; Wentao Tan; Lei Zhu; Fengling Li; Jingjing Li; Guoli Yang; Heng Tao Shen

cs.ROMar 2, 2026

Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining

Authors: Yipeng Chen, Wentao Tan, Lei Zhu, Fengling Li, Jingjing Li, Guoli Yang, Heng Tao Shen

Organizations: Tongji University · University of Technology Sydney · University of Electronic Science and Technology of China · Advanced Institute of Big Data

Abstract

Existing Vision-Language-Action (VLA) models often struggle to generalize to long-horizon tasks due to their heavy reliance on immediate observations. While recent studies incorporate retrieval mechanisms or extend context windows to handle procedural tasks, they often struggle to capture Non-Markovian dependencies, where optimal actions rely solely on specific past states rather than the current observation. To address this, we introduce Keyframe-Chaining VLA, a framework that extracts and links key historical frames to model long-horizon dependencies. Specifically, we propose an automatic keyframe selector that learns a discriminative embedding space, effectively identifying distinct state transitions. To capture task-critical information, we design a progress-aware query mechanism that dynamically retrieves historical frames based on their temporal relevance to the current execution phase. These selected keyframes are integrated into the VLA as interleaved visual tokens, explicitly grounding the policy in the long-horizon temporal context. Finally, we introduce a suite of four Non-Markovian manipulation tasks built upon the ManiSkill simulator to measure task success rates. Experimental results demonstrate that our method achieves superior performance, effectively tackling robot manipulation tasks characterized by long-horizon temporal dependencies. Code is available at https://github.com/cytoplastm/KC-VLA.

Figures & tables

Figure 1: (a) Unlike standard VLAs ( Kim et al., 2024 ; Team et al., 2024 ; Chi et al., 2025 ) that rely on dense sliding windows or context compression ( Jang et al., 2025 ; Koo et al., 2025 ) , our Keyframe-Chaining VLA explicitly selects sparse semantic keyframes, avoiding computational redundancy. (b) The other three paradigms in (a) suffer from limited temporal receptive fields, failing to capture distant critical cues. In contrast, our KSM-based retrieval achieves a global temporal receptive field, enabling the agent to access long-range historical information and effectively resolve state aliasing. (c) Our framework achieves state-of-the-art performance, significantly outperforming baselines in both ManiSkill memory-dependent tasks and real-world long-horizon deployment.

Figure 2: Overview of the Keyframe-Chaining VLA framework. The architecture operates via two synergistic modules. (Left) Keyframe Selection Module: Adopts a two-stage design where Stage I optimizes a visual encoder via unified metric learning, and Stage II employs a Task-Modulated Query network to compute phase-specific confidence scores $s_{t}$ . (Top Right) Greedy Temporal Smoothing: A stabilizing mechanism designed to filter detection jitter. It greedily updates provisional candidates to capture the latest valid state and finalizes the keyframe only after passing a temporal validation window. (Bottom Right) VLA Module: The policy integrates the sparse semantic history of detected keyframes with the current observation to generate actions via Flow Matching.

Figure 3: Overview of the four custom Non-Markovian ManiSkill tasks. Red bounding boxes indicate the ground-truth keyframes utilized for training. Please refer to the Appendix for the detailed definitions of keyframe events for each task.

Model / Configuration	Sampling	$\mathbf{N_{h}}$	$\mathbf{I}$	Spatial	Temporal	Identity	Counting	Average
$\pi_{0}$ ( Black et al., 2024 )	Dense	0	-	2.0	0.0	0.0	60.0	15.5
Diffusion Policy ( Chi et al., 2025 )		1	1	22.0	10.0	0.0	30.0	15.5
GR00T-N1.5 ( Bjorck et al., 2025 ) ( No History )		0	-	20.0	0.0	28.0	16.0	16.0
GR00T-N1.5 ( Short-term )	Dense	1	1	8.0	16.0	30.0	4.0	14.5
		2	1	18.0	52.0	12.0	0.0	20.0
		3	1	14.0	54.0	0.0	40.0	27.0

Table 1: Quantitative comparison of success rates (%) on non-Markovian tasks. Keyframe-Chaining VLA(ours) achieves robust and superior performance.

KSM Training Paradigm	P (%) $\uparrow$	R (%) $\uparrow$	F1 (%) $\uparrow$	FPR (%) $\downarrow$	FNR (%) $\downarrow$
w/o Metric Pre-training	90.7	81.7	85.1	9.3	18.3
Joint End-to-End	93.6	79.6	83.4	6.4	20.4
Ours (Two-stage)	97.5	97.5	97.5	2.5	2.5

Table 2: Comparative diagnostic evaluation of KSM paradigms. Metrics are averaged across four tasks and reported in percentage. All metrics are reported in percentage (%). Detailed task-wise results are provided in Appendix B .

Configuration	Spatial	Temporal	Identity	Counting	Average
w/o Prompt Refinement	56.0	96.0	100.0	100.0	88.0
Ours (Context-Aware)	70.0	98.0	100.0	100.0	92.0

Table 3: Ablation on Prompt Refinement. Comparing context-aware prompting against generic instructions.

Figure 4: Real-world Experimental Setup.

Model	Spatial	Temporal	Counting	Identity	Average
	(4 stages)	(3 stages)	(2 stages)	(2 stages)
DP	0.0 / 33.8	0.0 / 16.7	5.0 / 35.0	10.0 / 45.0	3.8 / 32.6
GR00T	0.0 / 47.5	0.0 / 30.0	10.0 / 45.0	15.0 / 50.0	6.3 / 43.1
Ours	10.0 / 58.8	50.0 / 75.0	80.0 / 90.0	55.0 / 77.5	48.75 / 75.3

Table 4: Real-world quantitative evaluation (20 trials per task). Results are reported as Success Rate / Completion Rate (%). Completion Rate measures the percentage of completed sub-stages (see Appendix for stage definitions). Our method demonstrates decisive advantages in multi-stage reasoning tasks.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Method	Precision	Recall	F1-Score
Temporal Sequencing	w/o Metric Pre-training	80.3	81.7	81.0
	Joint End-to-End	93.3	93.3	93.3
	Ours (Two-stage)	100.0	100.0	100.0
Counting & Latency	w/o Metric Pre-training	100.0	62.5	76.9
	Joint End-to-End	100.0	40.0	57.1
	Ours (Two-stage)	100.0	100.0	100.0

Appendix

Table 5: Detailed Task-wise KSM Diagnostic Metrics (%). We compare Precision (P), Recall (R), and F1-score across three training paradigms. Our method consistently achieves optimal or near-optimal performance across all categories.

Panel of Figure 5 — Figure 5: The structured system prompt used to condition the VLA policy. The {Language Instruction} placeholder is replaced by the specific natural language command for the current episode (e.g., “Stack the cubes based on their initial vertical order”).

Figure 6: Qualitative rollouts of Keyframe-Chaining VLA in real-world scenarios. Each row corresponds to one of the four Non-Markovian tasks. The snapshots demonstrate the agent’s ability to maintain long-horizon consistency, such as waiting for signal lights (row 3) or tracking swapped objects (row 4).

Table 12

Explore similar work

S $^2$ -VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation
Jun 26, 2026Zhipeng Xie, Zongyi Han, Xiangyi Wei +3Long-Horizon Robotic Manipulation Vision-Language-Action Models
School of Computer Science and Technology, East China Normal University, Shanghai 200062, China · 2State Key Laboratory of Submarine Geoscience, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China
LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies
Feb 25, 2026Yue Yang, Shuo Cheng, Yu Fang +4Long-Horizon Robotic Manipulation Robotic Manipulation
University of North Carolina at Chapel Hill · Georgia Institute of Technology · Carnegie Mellon University
ChainVLA: Chaining Vision-Language-Action Queries through a Unified Execution State for Long-Horizon Manipulation
Aug 3, 2026Yuzhi Huang, Weijue Bu, Ziyi Xiong +4Memory-Augmented VLMs Long-Horizon Robotic Manipulation
Shenzhen International Graduate School, Tsinghua University · China University of Mining and Technology · Shenzhen Technology University

Jun 26, 2026J/K move · Enter open · S save

S $^2$ -VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation

Zhipeng Xie, Zongyi Han, Xiangyi Wei, Shiliang Sun+2

School of Computer Science and Technology, East China Normal University, Shanghai 200062, China · 2State Key Laboratory of Submarine Geoscience, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China

Long-Horizon Robotic Manipulation Vision-Language-Action Models Multimodal Fusion

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have demonstrated strong capabilities in robotic manipulation, but their performance degrades significantly in long-horizon tasks due to cumulative error propagation. This limitation largely arises from static feature fusion mechanisms that rely on fixed weights to combine visual, language, and action representations, preventing the model from adapting to different phases of task execution. To address this limitation, we propose S

^2

-VLA, a framework that introduces a State-Space Guided Adaptive Attention (SSGAA) mechanism. SSGAA maintains a belief state that tracks task progression and generates dynamic gating weights to adaptively fuse information from three complementary sources visual features for spatial perception, task intents for high-level task planning, and temporal action sequences for execution consistency. This adaptive fusion allows the model to shift its focus throughout task execution, aligning with the evolving requirements of different task stages. Despite its compact 2B parameter size, S

^2

-VLA consistently outperforms larger 7B-scale models and achieves state-of-the-art performance on long-horizon manipulation benchmarks, including LIBERO and SimplerEnv. highlighting the importance of adaptive feature fusion for long-horizon robotic manipulation.

Abstract

Explore similar work

S2^22-VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation

LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies

ChainVLA: Chaining Vision-Language-Action Queries through a Unified Execution State for Long-Horizon Manipulation

S $^2$ -VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation