Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks

Honghui Wang; Zhi Jing; Jicong Ao; Shiji Song; Xuelong Li; Gao Huang; Chenjia Bai

cs.ROMar 10, 2026

Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks

Authors: Honghui Wang, Zhi Jing, Jicong Ao, Shiji Song, Xuelong Li, Gao Huang, Chenjia Bai

Organizations: Tsinghua University · Fudan University · Institute of Artificial Intelligence (TeleAI), China Telecom

Abstract

The high cost of collecting real-robot data has made robotic simulation a scalable platform for both evaluation and data generation. Yet most existing benchmarks concentrate on simple manipulation tasks such as pick-and-place, failing to capture the non-Markovian characteristics of real-world tasks and the complexity of articulated object interactions. To address this limitation, we present RuleSafe, a new articulated manipulation benchmark built upon a scalable LLM-aided simulation framework. RuleSafe features safes with diverse unlocking mechanisms, such as key locks, password locks, and logic locks, which require different multi-stage reasoning and manipulation strategies. These LLM-generated rules produce non-Markovian and long-horizon tasks that require temporal modeling and memory-based reasoning. We further propose VQ-Memory, a compact and structured temporal representation that uses vector-quantized variational autoencoders (VQ-VAEs) to encode past proprioceptive states into discrete latent tokens. This representation filters low-level noise while preserving high-level task-phase context, providing lightweight yet robust temporal cues that are compatible with existing Vision-Language-Action models (VLA). Extensive experiments on state-of-the-art VLA models and diffusion policies show that VQ-Memory consistently improves long-horizon planning, enhances generalization to unseen configurations, and enables more efficient manipulation with reduced computational cost. Project page: vqmemory.github.io

Figures & tables

Figure 1: Locking rules from the RuleSafe Benchmark, structured by (a) part phase (knob/handle) and (b) task phase (password cache). Because these governing variables are not directly observable from visual input, the tasks exhibit a non-Markovian property.

Figure 2: Overview of the memory-augmented policy formulation. The pretrained Vision-Language Model encodes current observations and task instructions, while the memory tokens $\boldsymbol{m}_{t}$ provide additional historical context from previous timesteps. The combined representation is fed into the Action Expert, which predicts the future action sequence conditioned on the state $q_{t}$ and injected noise.

Figure 3: Visualization of discrete tokens of joint trajectories with ( Bottom : vocabulary size = 4) and without clustering ( Top : vocabulary size = 256). Each column represents a time step, and each row corresponds to one of three different trajectories, where different colors indicate different tokens. Without clustering, high-level temporal regularities are obscured by fine-grained variations, whereas clustering emphasizes shared semantic patterns across trajectories, enabling clearer identification of task execution stages.

Figure 4: Statistics of Demonstration Generation in RuleSafe: Success Rate and Average Frames.

Method	rule_001		rule_020
Method	SR (%)	PS (%)	SR (%)	PS (%)
$\pi_{0}$	30.0	56.7	0.0	10.6
$\pi_{0}$ + raw memory	55.0	70.1	0.0	16.3
$\pi_{0}$ + VQ-memory	80.0	89.3	45.0	67.3

Table 1: Single-task performances of different memory types. “Raw memory” refers to using raw joint-state sequences as memory tokens. We report success rate (SR) and process score (PS).

Method	rule_020
Method	Success Rate (%)	Process Score (%)
DP3	5.0	14.1
+ VQ-memory	45.0	70.4
RDT	0.0	13.6
+ VQ-memory	35.0	61.2
CogACT	0.0	9.7

Table 2: Effect of VQ-Memory across models.

	Success	Process		Success	Process		Success	Process
	Rate	Score		Rate	Score		Rate	Score
rule_001			rule_002			rule_003
$\pi_{0}$	35.0	61.7	$\pi_{0}$	80.0	85.0	$\pi_{0}$	75.0	87.5
+VQ-Memory	60.0	73.3	+VQ-Memory	90.0	90.0	+VQ-Memory	95.0	97.5
rule_004			rule_005			rule_006
$\pi_{0}$	35.0	58.3	$\pi_{0}$	30.0	48.8	$\pi_{0}$	0.0	25.7

Table 3: Performance of

\pi_{0}

with and without VQ-Memory on the multi-task setting. We report success rate (%) and process score (%).

	Variant	Success Rate	Process Score
Number of Clusters	256	20.0	47.5
	32	35.0	57.5
	4	45.0	67.3
	2	30.0	54.4
Memory Length	20	25.0	53.8
	40	45.0	67.3

Table 4: Ablation on number of clusters and memory length. We report the success rate (%) and process score (%) on rule_020 .

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: List of 10 object instances used in RuleSafe.

Figure 6: Annotation example. For hands, we annotate contact points and motion directions. For articulated components such as knobs, we label key points and axes representing their operational interfaces.


Rule ID	Description
rule_001	The safe door remains locked unless both the knob and handle are open.
rule_002	The safe door remains locked unless the knob is open.
rule_003	The safe door remains locked unless the handle is open.
rule_004	The safe door remains locked unless both the knob and handle are open. Additionally, the knob cannot be opened unless the handle is open.
rule_005	The safe door remains locked unless the password ’001’ is entered. Changing the knob’s state from open to closed (or vice versa) inputs a ’0’, while changing the handle’s state inputs a ’1’.

Appendix

Table 5: Descriptions of all 20 rules in the RuleSafe benchmark.

Long-Horizon Robotic Manipulation

Long-horizon robotic manipulation studies how robots plan, remember, and recover across extended sequences of physical tasks.

See all 429 papers

RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
Mar 4, 2026Yinpei Dai, Hongze Fu, Jayjun Lee +6Memory-Augmented VLMs Robot Manipulation Benchmarks
University of Michigan · Stanford University · Figure AI
MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models
Jun 8, 2026Hao Shi, Weiye Li, Bin Xie +6Long-Horizon Robotic Manipulation World Models for Robotics
Department of Automation, BNRist, Tsinghua University, Beijing, China · Department of Computer Science, The University of Hong Kong, Hong Kong, China · Dexmal, Beijing, China +1
Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining
Mar 2, 2026Yipeng Chen, Wentao Tan, Lei Zhu +4Long-Horizon Robotic Manipulation Vision-Language-Action Models
Tongji University · University of Technology Sydney · University of Electronic Science and Technology of China +1

Mar 4, 2026J/K move · Enter open · S save

RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

Yinpei Dai, Hongze Fu, Jayjun Lee, Yuejiang Liu+5

University of Michigan · Stanford University · Figure AI

Memory-Augmented VLMs Robot Manipulation Benchmarks Long-Horizon Robotic Manipulation

Open paper AskPDF arXiv

Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms; however, their evaluations remain confined to narrow, non-standardized settings. This limits systematic understanding, comparison, and progress measurement. To address these challenges, we introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models in long-horizon, history-dependent scenarios. Our benchmark comprises 16 manipulation tasks constructed under a carefully designed taxonomy that evaluates temporal, spatial, object, and procedural memory. We further develop a suite of 14 memory-augmented VLA variants built on the π0.5 backbone to systematically explore different memory representations across multiple integration strategies. Experimental results show that the effectiveness of memory representations is highly task-dependent, with each design offering distinct advantages and limitations across different tasks. Videos and code can be found at our website https://robomme.github.io.

Abstract

Long-Horizon Robotic Manipulation

Explore similar work

RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models

Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining