Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation

Zaijing Li; Bing Hu; Rui Shao; Gongwei Chen; Dongmei Jiang; Pengwei Xie; Jianye Hao; Liqiang Nie

cs.ROFeb 22, 2026

Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation

Authors: Zaijing Li, Bing Hu, Rui Shao, Gongwei Chen, Dongmei Jiang, Pengwei Xie, Jianye Hao, Liqiang Nie

Organizations: Harbin Institute of Technology, Shenzhen · PengCheng Laboratory · Shenzhen Loop Area Institute · Huawei Noah’s Ark Lab

Abstract

Hierarchical Vision-Language-Action (VLA) models have rapidly become a dominant paradigm for robotic manipulation. It typically comprising a Vision-Language backbone for perception and understanding, together with a generative policy for action generation. However, its performance is increasingly bottlenecked by the action generation proceess. (i) Low inference efficiency. A pronounced distributional gap between isotropic noise priors and target action distributions, which increases denoising steps and the incidence of infeasible samples. (ii) Poor robustness. Existing policies condition solely on the current observation, neglecting the constraint of history sequence and thus lacking awareness of task progress and temporal consistency. To address these issues, we introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM). GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories, thereby shortening the generative path and reducing the umber of function evaluations (NFE). LCM dynamically models executed action sequence to infer task progress and injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory. Across three simulation benchmarks, OptimusVLA consistently outperforms strong baselines: it achieves 98.6% average success rate on LIBERO, improves over pi_0 by 13.5% on CALVIN, and attains 38% average success rate on RoboTwin 2.0 Hard. In Real-World evaluation, OptimusVLA ranks best on Generalization and Long-horizon suites, surpassing pi_0 by 42.9% and 52.4%, respectively, while delivering 2.9x inference speedup.

Figures & tables

Figure 1: Top : Comparison between the standard VLA architecture (left) and our proposed OptimusVLA (right). (ii) Poor robustness to temporal dependence. Middle : Illustration of how GPM ( blue ) and LCM ( green ) address two key limitations of existing VLA models: (i) Low inference efficiency due to a large prior–target gap. (ii) Poor robustness to temporal dependence. Bottom : Efficiency and performance comparison.

Figure 2: Overview of OptimusVLA framework. Given a task and the current observation, the Vision–Language backbone first encodes the inputs into a multimodal representation. GPM then retrieves a task-level prior based on this representation, while LBM dynamically encodes the historical action sequence to produce a consistency constraint. Finally, the flow policy denoises the initialization with an adaptive NFEs schedule to generate the action chunk.

Method	LIBERO
Method	Spatial	Object	Goal	Long	Avg.
DP [ 6 ]	78.3	92.5	68.3	50.5	72.4
Octo [ 42 ]	78.9	85.7	84.6	51.1	75.1
SpatialVLA [ 37 ]	88.2	89.9	78.6	55.5	78.1
$\pi_{0}$ -FAST [ 36 ]	96.4	96.8	88.6	60.2	85.5
CogACT [ 17 ]	97.2	98.0	90.2	88.8	93.6

Table 1: Performance comparison on LIBERO [ 26 ] . We report the average success rate on each task suite (500 rollouts).

Method	CALVIN (ABC $\rightarrow$ D)
Method	1/5	2/5	3/5	4/5	5/5	Avg. Len
RoboVLM [ 28 ]	98.0	93.6	85.4	77.8	70.4	4.25
ReconVLA [ 41 ]	95.6	87.6	76.9	69.3	64.1	3.95
OpenVLA [ 14 ]	91.3	77.8	62.0	52.1	43.5	3.27
UniVLA [ 3 ]	95.5	85.8	75.4	66.9	56.5	3.80
UP-VLA [ 53 ]	92.8	86.5	81.5	76.9	69.9	4.08

Table 2: Performance comparison on CALVIN [ 34 ] . We report the success rate of each track and average completion length (Avg. Len). † represents the result we reproduced.

Task	RDT [ 31 ]		ACT [ 57 ]		DP [ 6 ]		DP3 [ 49 ]		$\mathbf{\pi_{0}}$ [ 1 ]		$\mathbf{\pi_{0.5}}^{{\dagger}}$ [ 12 ]		OptimusVLA
Task	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR RK
Click Alarmclock	12%	4	4%	7	5%	6	14%	3	11%	5	18%	2	31% 1
Click Bell	9%	3	3%	4	0%	5	0%	5	3%	4	28%	2	46% 1
Dump Bin Bigbin	32%	3	1%	6	0%	7	53%	1	24%	5	29%	4	35% 2
Open Laptop	32%	4	0%	6	0%	6	7%	5	46%	2	38%	3	48% 1
Place Bread Skillet	1%	3	0%	4	0%	4	0%	4	1%	4	2%	2	4% 1

Table 3: Performance comparison on RoboTwin 2.0 [ 5 ] . We report per-task success rates (SR) and rank (RK) over 100 rollouts under Hard setting. † represents the result we reproduced. 16 tasks results are available in Appendix.

Ablation Setting		Simulation		Real-World
GPM	LCM	LIBERO-Long	CALVIN	Generalization Tasks
✓	✓	96.4	4.45	85.0
	✓	93.2 ( $\downarrow$ 3.3%)	4.28 ( $\downarrow$ 3.8%)	77.0 ( $\downarrow$ 9.4%)
✓		94.8 ( $\downarrow$ 1.7%)	4.38 ( $\downarrow$ 1.6%)	79.5 ( $\downarrow$ 6.5%)
		92.4 ( $\downarrow$ 4.1%)	4.26 ( $\downarrow$ 4.3%)	75.0 ( $\downarrow$ 11.8%)

Table 4: Ablation study of GPM and LCM on LIBERO-Long, CALVIN, and Real-World Generalization Tasks .

Metric	Num =6500			Num =1300		Num =130
Metric	$k{=}8$	$k{=}16$	$k{=}1$	$k{=}8$	$k{=}1$	$k{=}8$
SR	96.4	94.8	92.6	95.2	92.4	93.6

Table 5: Ablation study on memory size of GPM. We report average success rate on LIBERO-Long. Num . denotes the number of trajectories in the memory bank, k . denotes the number of retrieved trajectories.

Figure 3: Real-world task setup and evaluation results. We evaluate the performance of OptimusVLA against OpenVLA [ 14 ] , OpenVLA-OFT [ 15 ] , $\pi_{0}$ [ 1 ] , and $\pi_{0.5}$ [ 12 ] on the Generalization Tasks and Long-horizon Tasks suites.

Panel of Figure 4 — Figure 4: Training efficiency comparison between OptimusVLA and $\pi_{0.5}$ [ 12 ] . Initialized from same weights, OptimusVLA attains strong performance with substantially fewer training steps.

Figure 5: Inference efficiency comparison on LIBERO and Real-World. OptimusVLA attains strong performance with substantially fewer inference time and NFEs.

Figure 6: Qualitative results of OptimusVLA on simulation task and Real-World task. Top : On simulation task, we visualize the retrieval similarity $\bar{s}$ , adaptive noise scales $\lambda$ , and NFEs $N$ of key frames. Bottom : On Real-World task, we demonstrate the role of LCM in modeling temporal dependencies, whereas $\pi_{0.5}$ struggles to distinguish between similar observations.

Hyperparameter	Stage-1	Stage-2	Stage-3
Optimizer	AdamW	AdamW	AdamW
Learning Rate	5e-5	1e-4	1e-4
Steps	30000	1000	1000
Batch Size	512	64	64
Warm Up Ratio	0.10	-	-
Ema Decay	0.999	-	-

Table 6: Hyperparameter setting for each training phase.

Task	RDT [ 31 ]		ACT [ 57 ]		DP [ 6 ]		DP3 [ 49 ]		$\mathbf{\pi_{0}}$ [ 1 ]		$\mathbf{\pi_{0.5}}^{{\dagger}}$ [ 12 ]		OptimusVLA
Task	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR RK
Click Alarmclock	12%	4	4%	7	5%	6	14%	3	11%	5	18%	2	31% 1
Click Bell	9%	3	3%	4	0%	5	0%	5	3%	4	28%	2	46% 1
Dump Bin Bigbin	32%	3	1%	6	0%	7	53%	1	24%	5	29%	4	35% 2
Open Laptop	32%	4	0%	6	0%	6	7%	5	46%	2	38%	3	48% 1
Place Bread Skillet	1%	3	0%	4	0%	4	0%	4	1%	4	2%	2	4% 1

Table 7: Performance comparison on RoboTwin 2.0 [ 5 ] . We report per-task success rates (SR) and rank (RK) over 100 rollouts under Hard setting. † represents the result we reproduced.

	Generalization Tasks
Task name	place_one_fruit_on_plate	place_plate_on_tablecloth	stand_bottle_upright	place_cup_on_tablecloth
Description	Grasp a piece of fruit and place it onto the plate. The set of graspable fruits includes bananas, apples, lemons, and related varieties.	Grasp a plate and place it on the tablecloth. Both the plates and the tablecloths come in multiple styles.	Grasp a water bottle and then orient it to an upright pose. The bottles exhibit diverse shapes and appearances.	Grasp a cup and place it on the tablecloth. The cups exhibit diverse shapes and appearances.
# demonstrations	100	100	150	120
# rollouts	50	50	50	50
Performance (%)	88	90	74	88

Table 8: Overview of Generalization Tasks on Real-World.

	Long-Horizon Tasks
Task name	place_all_fruits_on_plate	place_2_fruits_in_2_position	place_2_obj_on_plate	place_blocks_on_tablecloth
Description	Grasp three different fruits and place them onto the plate. For each rollout, a different combination of fruits is randomly sampled.	Grasp one fruit from the bowl and place it onto the tablecloth, then grasp another fruit from the plate and place it into the bowl.	Grasp a cup and an apple in sequence and place them into the plate.	Grasp three distinct blocks and place them on the tablecloth. For each rollout, a different combination of blocks is randomly sampled.
# demonstrations	200	200	250	300
# rollouts	25	25	25	25
Performance (%)	64	60	68	64

Table 9: Overview of Long-horizon Tasks on Real-World.

Figure 7: Qualitative results of OptimusVLA on Real-World. From top to bottom, we illustrate four Generalization Tasks : Place one fruit on plate , Place plate on tablecloth , Stand bottle upright , and Place cup on tablecloth . Here, third denotes third-person view images, while wrist denotes images captured from the robot’s wrist-mounted camera.

Figure 8: Qualitative results of OptimusVLA on Real-World. From top to bottom, we illustrate four Long-horizon Tasks : Place all fruits on plate , Place one fruit on tablecloth, then place another fruit into bowl , Place cup and apple on plate , and Place 3 blocks on tablecloth . Here, third denotes third-person view images, while wrist denotes images captured from the robot’s wrist-mounted camera.

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 190 papers

D $^2$ -VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation
Sep 28, 2026Zijian Ye, Chengqi Wei, Wei Huang +9Memory-Augmented VLMs Long-Horizon Robotic Manipulation
The University of Hong Kong · Southern University of Science and Technology
VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting
Jul 7, 2025Juyi Lin, Amir Taherin, Arash Akbari +11Robotic Control Efficient VLA Model Inference
Northeastern University, Boston, USA · EmbodyX,San Mateo,USA
TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM
Jul 29, 2026Hengyi Xie, Chenfei Yao, Xianjin Wu +4Efficient VLA Model Inference Efficient VLA Models
Huazhong University of Science and Technology

Sep 28, 2026J/K move · Enter open · S save

D $^2$ -VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation

Zijian Ye, Chengqi Wei, Wei Huang, Anlin Zheng+8

The University of Hong Kong · Southern University of Science and Technology

Memory-Augmented VLMs Long-Horizon Robotic Manipulation Efficient VLA Model Inference Vision-Language-Action Models Robot Manipulation Benchmarks

Open paper AskPDF arXiv

Long-horizon manipulation requires robots to remember cues that are no longer in view while responding to moving objects. Yet vision-language-action (VLA) policies often rely on the latest observation, and refreshing their visual context typically requires another costly vision-language model (VLM) pass. We present D

^2

-VLA, which combines dual memory and dual-frequency control at the KV-cache interface of a pretrained VLA. D

^2

-VLA uses block-wise causal KV caching to encode observations incrementally and, guided by distinct temporal attention patterns, constructs separate historical KV read views for the VLM and action expert. Between periodic VLM updates, a gated adapter incorporates fresh visual features into the latest history-conditioned KV block, while a short fast-memory queue supports action replanning. We introduce DOMINO-Long, a ten-task benchmark requiring robots to use earlier visual cues when manipulating moving objects. D

^2

-VLA achieves complete-task success rates of 29.3% on DOMINO, compared with 9.6% for

π_{0.5}

and 17.2% for PUMA, and 60.0% on DOMINO-Long, compared with 35.4% and 20.6%, respectively. It improves success rates on eight real-robot tasks and reaches 97.5% on LIBERO-Long and 74.3% on RoboTwin 2.0.

Abstract

Efficient VLA Model Inference

Explore similar work

D2^22-VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation

VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

D $^2$ -VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation