Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation

Zaijing Li; Bing Hu; Rui Shao; Gongwei Chen; Dongmei Jiang; Pengwei Xie; Jianye Hao; Liqiang Nie

cs.ROFeb 22, 2026

Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation

Authors: Zaijing Li, Bing Hu, Rui Shao, Gongwei Chen, Dongmei Jiang, Pengwei Xie, Jianye Hao, Liqiang Nie

Organizations: Harbin Institute of Technology, Shenzhen · PengCheng Laboratory · Shenzhen Loop Area Institute · Huawei Noah’s Ark Lab

Abstract

Hierarchical Vision-Language-Action (VLA) models have rapidly become a dominant paradigm for robotic manipulation. It typically comprising a Vision-Language backbone for perception and understanding, together with a generative policy for action generation. However, its performance is increasingly bottlenecked by the action generation proceess. (i) Low inference efficiency. A pronounced distributional gap between isotropic noise priors and target action distributions, which increases denoising steps and the incidence of infeasible samples. (ii) Poor robustness. Existing policies condition solely on the current observation, neglecting the constraint of history sequence and thus lacking awareness of task progress and temporal consistency. To address these issues, we introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM). GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories, thereby shortening the generative path and reducing the umber of function evaluations (NFE). LCM dynamically models executed action sequence to infer task progress and injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory. Across three simulation benchmarks, OptimusVLA consistently outperforms strong baselines: it achieves 98.6% average success rate on LIBERO, improves over pi_0 by 13.5% on CALVIN, and attains 38% average success rate on RoboTwin 2.0 Hard. In Real-World evaluation, OptimusVLA ranks best on Generalization and Long-horizon suites, surpassing pi_0 by 42.9% and 52.4%, respectively, while delivering 2.9x inference speedup.

Figures & tables

Figure 1: Top : Comparison between the standard VLA architecture (left) and our proposed OptimusVLA (right). (ii) Poor robustness to temporal dependence. Middle : Illustration of how GPM ( blue ) and LCM ( green ) address two key limitations of existing VLA models: (i) Low inference efficiency due to a large prior–target gap. (ii) Poor robustness to temporal dependence. Bottom : Efficiency and performance comparison.

Figure 2: Overview of OptimusVLA framework. Given a task and the current observation, the Vision–Language backbone first encodes the inputs into a multimodal representation. GPM then retrieves a task-level prior based on this representation, while LBM dynamically encodes the historical action sequence to produce a consistency constraint. Finally, the flow policy denoises the initialization with an adaptive NFEs schedule to generate the action chunk.

Method	LIBERO
Method	Spatial	Object	Goal	Long	Avg.
DP [ 6 ]	78.3	92.5	68.3	50.5	72.4
Octo [ 42 ]	78.9	85.7	84.6	51.1	75.1
SpatialVLA [ 37 ]	88.2	89.9	78.6	55.5	78.1
$\pi_{0}$ -FAST [ 36 ]	96.4	96.8	88.6	60.2	85.5
CogACT [ 17 ]	97.2	98.0	90.2	88.8	93.6

Table 1: Performance comparison on LIBERO [ 26 ] . We report the average success rate on each task suite (500 rollouts).

Method	CALVIN (ABC $\rightarrow$ D)
Method	1/5	2/5	3/5	4/5	5/5	Avg. Len
RoboVLM [ 28 ]	98.0	93.6	85.4	77.8	70.4	4.25
ReconVLA [ 41 ]	95.6	87.6	76.9	69.3	64.1	3.95
OpenVLA [ 14 ]	91.3	77.8	62.0	52.1	43.5	3.27
UniVLA [ 3 ]	95.5	85.8	75.4	66.9	56.5	3.80
UP-VLA [ 53 ]	92.8	86.5	81.5	76.9	69.9	4.08

Table 2: Performance comparison on CALVIN [ 34 ] . We report the success rate of each track and average completion length (Avg. Len). † represents the result we reproduced.

Task	RDT [ 31 ]		ACT [ 57 ]		DP [ 6 ]		DP3 [ 49 ]		$\mathbf{\pi_{0}}$ [ 1 ]		$\mathbf{\pi_{0.5}}^{{\dagger}}$ [ 12 ]		OptimusVLA
Task	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR RK
Click Alarmclock	12%	4	4%	7	5%	6	14%	3	11%	5	18%	2	31% 1
Click Bell	9%	3	3%	4	0%	5	0%	5	3%	4	28%	2	46% 1
Dump Bin Bigbin	32%	3	1%	6	0%	7	53%	1	24%	5	29%	4	35% 2
Open Laptop	32%	4	0%	6	0%	6	7%	5	46%	2	38%	3	48% 1
Place Bread Skillet	1%	3	0%	4	0%	4	0%	4	1%	4	2%	2	4% 1

Table 3: Performance comparison on RoboTwin 2.0 [ 5 ] . We report per-task success rates (SR) and rank (RK) over 100 rollouts under Hard setting. † represents the result we reproduced. 16 tasks results are available in Appendix.

Ablation Setting		Simulation		Real-World
GPM	LCM	LIBERO-Long	CALVIN	Generalization Tasks
✓	✓	96.4	4.45	85.0
	✓	93.2 ( $\downarrow$ 3.3%)	4.28 ( $\downarrow$ 3.8%)	77.0 ( $\downarrow$ 9.4%)
✓		94.8 ( $\downarrow$ 1.7%)	4.38 ( $\downarrow$ 1.6%)	79.5 ( $\downarrow$ 6.5%)
		92.4 ( $\downarrow$ 4.1%)	4.26 ( $\downarrow$ 4.3%)	75.0 ( $\downarrow$ 11.8%)

Table 4: Ablation study of GPM and LCM on LIBERO-Long, CALVIN, and Real-World Generalization Tasks .

Metric	Num =6500			Num =1300		Num =130
Metric	$k{=}8$	$k{=}16$	$k{=}1$	$k{=}8$	$k{=}1$	$k{=}8$
SR	96.4	94.8	92.6	95.2	92.4	93.6

Table 5: Ablation study on memory size of GPM. We report average success rate on LIBERO-Long. Num . denotes the number of trajectories in the memory bank, k . denotes the number of retrieved trajectories.

Figure 3: Real-world task setup and evaluation results. We evaluate the performance of OptimusVLA against OpenVLA [ 14 ] , OpenVLA-OFT [ 15 ] , $\pi_{0}$ [ 1 ] , and $\pi_{0.5}$ [ 12 ] on the Generalization Tasks and Long-horizon Tasks suites.

Panel of Figure 4 — Figure 4: Training efficiency comparison between OptimusVLA and $\pi_{0.5}$ [ 12 ] . Initialized from same weights, OptimusVLA attains strong performance with substantially fewer training steps.

Figure 5: Inference efficiency comparison on LIBERO and Real-World. OptimusVLA attains strong performance with substantially fewer inference time and NFEs.

Figure 6: Qualitative results of OptimusVLA on simulation task and Real-World task. Top : On simulation task, we visualize the retrieval similarity $\bar{s}$ , adaptive noise scales $\lambda$ , and NFEs $N$ of key frames. Bottom : On Real-World task, we demonstrate the role of LCM in modeling temporal dependencies, whereas $\pi_{0.5}$ struggles to distinguish between similar observations.

Hyperparameter	Stage-1	Stage-2	Stage-3
Optimizer	AdamW	AdamW	AdamW
Learning Rate	5e-5	1e-4	1e-4
Steps	30000	1000	1000
Batch Size	512	64	64
Warm Up Ratio	0.10	-	-
Ema Decay	0.999	-	-

Table 6: Hyperparameter setting for each training phase.

Task	RDT [ 31 ]		ACT [ 57 ]		DP [ 6 ]		DP3 [ 49 ]		$\mathbf{\pi_{0}}$ [ 1 ]		$\mathbf{\pi_{0.5}}^{{\dagger}}$ [ 12 ]		OptimusVLA
Task	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR	RK	SR RK
Click Alarmclock	12%	4	4%	7	5%	6	14%	3	11%	5	18%	2	31% 1
Click Bell	9%	3	3%	4	0%	5	0%	5	3%	4	28%	2	46% 1
Dump Bin Bigbin	32%	3	1%	6	0%	7	53%	1	24%	5	29%	4	35% 2
Open Laptop	32%	4	0%	6	0%	6	7%	5	46%	2	38%	3	48% 1
Place Bread Skillet	1%	3	0%	4	0%	4	0%	4	1%	4	2%	2	4% 1

Table 7: Performance comparison on RoboTwin 2.0 [ 5 ] . We report per-task success rates (SR) and rank (RK) over 100 rollouts under Hard setting. † represents the result we reproduced.

	Generalization Tasks
Task name	place_one_fruit_on_plate	place_plate_on_tablecloth	stand_bottle_upright	place_cup_on_tablecloth
Description	Grasp a piece of fruit and place it onto the plate. The set of graspable fruits includes bananas, apples, lemons, and related varieties.	Grasp a plate and place it on the tablecloth. Both the plates and the tablecloths come in multiple styles.	Grasp a water bottle and then orient it to an upright pose. The bottles exhibit diverse shapes and appearances.	Grasp a cup and place it on the tablecloth. The cups exhibit diverse shapes and appearances.
# demonstrations	100	100	150	120
# rollouts	50	50	50	50
Performance (%)	88	90	74	88

Table 8: Overview of Generalization Tasks on Real-World.

	Long-Horizon Tasks
Task name	place_all_fruits_on_plate	place_2_fruits_in_2_position	place_2_obj_on_plate	place_blocks_on_tablecloth
Description	Grasp three different fruits and place them onto the plate. For each rollout, a different combination of fruits is randomly sampled.	Grasp one fruit from the bowl and place it onto the tablecloth, then grasp another fruit from the plate and place it into the bowl.	Grasp a cup and an apple in sequence and place them into the plate.	Grasp three distinct blocks and place them on the tablecloth. For each rollout, a different combination of blocks is randomly sampled.
# demonstrations	200	200	250	300
# rollouts	25	25	25	25
Performance (%)	64	60	68	64

Table 9: Overview of Long-horizon Tasks on Real-World.

Figure 7: Qualitative results of OptimusVLA on Real-World. From top to bottom, we illustrate four Generalization Tasks : Place one fruit on plate , Place plate on tablecloth , Stand bottle upright , and Place cup on tablecloth . Here, third denotes third-person view images, while wrist denotes images captured from the robot’s wrist-mounted camera.

Figure 8: Qualitative results of OptimusVLA on Real-World. From top to bottom, we illustrate four Long-horizon Tasks : Place all fruits on plate , Place one fruit on tablecloth, then place another fruit into bowl , Place cup and apple on plate , and Place 3 blocks on tablecloth . Here, third denotes third-person view images, while wrist denotes images captured from the robot’s wrist-mounted camera.

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 190 papers

Sep 28, 2026

D $^2$ -VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation

Long-horizon manipulation requires robots to remember cues that are no longer in view while responding to moving objects. Yet vision-language-action (VLA) policies often rely on the latest observation, and refreshing their visual context typically requires another costly vision-language model (VLM) pass. We present D

^2

-VLA, which combines dual memory and dual-frequency control at the KV-cache interface of a pretrained VLA. D

^2

-VLA uses block-wise causal KV caching to encode observations incrementally and, guided by distinct temporal attention patterns, constructs separate historical KV read views for the VLM and action expert. Between periodic VLM updates, a gated adapter incorporates fresh visual features into the latest history-conditioned KV block, while a short fast-memory queue supports action replanning. We introduce DOMINO-Long, a ten-task benchmark requiring robots to use earlier visual cues when manipulating moving objects. D

^2

-VLA achieves complete-task success rates of 29.3% on DOMINO, compared with 9.6% for

π_{0.5}

and 17.2% for PUMA, and 60.0% on DOMINO-Long, compared with 35.4% and 20.6%, respectively. It improves success rates on eight real-robot tasks and reaches 97.5% on LIBERO-Long and 74.3% on RoboTwin 2.0.

Zijian Ye, Chengqi Wei, Wei Huang +9

The University of Hong Kong · Southern University of Science and Technology

Memory-Augmented VLMs Long-Horizon Robotic Manipulation Efficient VLA Model Inference Vision-Language-Action Models Robot Manipulation Benchmarks

Ask PDF

Jul 7, 2025

VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

Recent large-scale Vision Language Action (VLA) models have shown superior performance in robotic manipulation tasks guided by natural language. However, current VLA models suffer from two drawbacks: (i) generation of massive tokens leading to high inference latency and increased training cost, and (ii) insufficient utilization of generated actions resulting in potential performance loss. To address these issues, we develop a training framework to finetune VLA models for generating significantly fewer action tokens with high parallelism, effectively reducing inference latency and training cost. Furthermore, we introduce an inference optimization technique with a novel voting-based ensemble strategy to combine current and previous action predictions, improving the utilization of generated actions and overall performance. Our results demonstrate that we achieve superior performance compared with state-of-the-art VLA models, achieving significantly higher success rates and 39

\times

faster inference than OpenVLA with 46 Hz throughput on edge platforms, demonstrating practical deployability. The code is available at https://github.com/LukeLIN-web/VOTE.

Juyi Lin, Amir Taherin, Arash Akbari +11

Northeastern University, Boston, USA · EmbodyX,San Mateo,USA

Robotic Control Efficient VLA Model Inference Efficient VLA Models Vision-Language-Action Models VLMs for Robotics VLA Model Training

Ask PDF

Jul 29, 2026

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

Vision-language-action (VLA) models commonly adopt an LLM-centric

V \to L \to A

pathway, processing visual observations and language instructions through a large language model before predicting robot actions. Although effective, this design incurs substantial computation and memory overhead. In this work, we introduce TurboVLA, a compact VLA architecture built on a direct

V + L \to A

mapping. Instead of using a large language model as the central interface between perception and action, TurboVLA independently encodes visual observations and language instructions, directly exchanges information between them through lightweight bidirectional vision-language interaction, and predicts continuous action chunks with a compact decoder. This simple design directly constructs task-conditioned representations while avoiding the overhead of an LLM-centered execution pathway. On LIBERO, TurboVLA achieves 97.6% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GiB inference VRAM on a consumer-grade RTX 4090. Notably, a 0.4B TurboVLA achieves 88.06% success on RoboTwin 2.0, even matching or outperforming substantially larger VLA policies. These results demonstrate that the simple

V + L \to A

design of TurboVLA can achieve high performance without requiring an LLM-centric execution pathway, offering a new perspective on how vision, language, and action can be connected for efficient robotic manipulation. Code is available at https://github.com/H-EmbodVis/TurboVLA.

Hengyi Xie, Chenfei Yao, Xianjin Wu +4

Huazhong University of Science and Technology

Efficient VLA Model Inference Efficient VLA Models Robotic Manipulation Vision-Language-Action Models

Ask PDF

Abstract

Efficient VLA Model Inference

Explore similar work

D2^22-VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation

VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

D $^2$ -VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation