CondenseVLA: Learnable History Condensation for Efficient Multi-Frame VLA

Feiyang Hong; Yujie Wei; Bo Zhao; Xiu Su; Hongxun Yao; Shuo Yang

cs.CVOct 19, 2026

CondenseVLA: Learnable History Condensation for Efficient Multi-Frame VLA

Authors: Feiyang Hong, Yujie Wei, Bo Zhao, Xiu Su, Hongxun Yao, Shuo Yang

Organizations: Harbin Institute of Technology, Shenzhen · Harbin Institute of Technology · Shanghai Jiao Tong University · Central South University

Abstract

Robot manipulation often depends on short-term cues - such as brief contacts, temporary occlusions, and intermediate object states - that are ambiguous in a single frame and can only be resolved by integrating observations over time. However, injecting historical information solely by enlarging the history window is computationally costly and frequently yields limited gains due to substantial redundancy and task-agnostic, inflexible sampling of observations. We propose CondenseVLA, a multi-frame VLA policy with a learnable decoder-side history condensation module for using short-term visual context in action decoding. Given a sequence of cached per-frame features from the VLM backbone, CondenseVLA distills observation history into a fixed set of slot tokens using learnable queries and multi-head cross-attention. Simple auxiliary regularization penalizes overlap among slot attention maps and decorrelates slot embeddings, encouraging complementary attention and reducing slot redundancy. Across the evaluated tasks, CondenseVLA improves Simpler-WidowX success from 58.7% to 69.8% (+11.1 percentage points), achieves 97.2% on LIBERO, and reaches 52.8% on real-world tasks. It achieves 21.6% success on memory-dependent RMBench tasks. These results support selective condensation of bounded recent visual evidence for action decoding, while harder multi-observation tasks expose the limits of a finite history window. Project page: https://github.com/FeiyangHong/CondenseVLA

Figures & tables

Figure 1: We propose CondenseVLA, a decoder-side Learnable History Condensation (LHC) model that distills informative multi-frame context into compact slot tokens. The left panel highlights history-dependent failures of single-step VLA execution: in Pick and Place Twice, the policy forgets which plate to place next; in Swap Cubes, it forgets which cube should be picked next. Across multiple benchmarks, including Simpler, LIBERO, and real-world robot experiments, CondenseVLA consistently outperforms the CronusVLA baseline.

Figure 2: Overview of CondenseVLA. Frozen vision/language weights and a trainable multimodal projector encode each observation. LHC condenses cached history using slot queries and value-only time embeddings. The decoder expands the current feature separately; historical features use stop-gradient during training.

Method	Spoon on Towel	Carrot on Plate	Stack Cube	Eggplant in Basket	Avg. Success
TraceVLA [ 54 ]	12.5	16.6	16.6	65.0	27.7
Octo-base [ 11 ]	15.8	12.5	0.0	41.7	17.5
UniVLA [ 4 ]	52.8	55.6	2.8	80.6	47.9
CogACT [ 24 ]	75.0	50.0	16.7	79.2	55.2
Magma [ 45 ]	37.5	29.2	20.8	91.7	44.8
RT-1-X [ 34 ]	0.0	4.2	0.0	0.0	1.1

Table 1: Mean success rate (%) on Simpler-WidowX. Boldface marks lower-block best results; source and protocol differences are detailed in Appendix B.1 .

Method	Spatial	Object	Goal	Long	Avg. Success
Diffusion Policy [ 7 ]	78.3	92.5	68.3	50.5	72.4
OpenVLA [ 18 ]	84.7	88.4	79.2	53.7	76.5
OpenVLA-OFT [ 19 ]	97.6	98.4	97.9	94.5	97.1
$\pi_{0}$ [ 2 ]	96.8	98.8	95.8	85.2	94.2
TraceVLA [ 54 ]	84.6	85.2	75.1	54.1	74.8
SpatialVLA [ 38 ]	88.2	89.9	78.6	55.5	78.1

Table 2: Mean success rate (%) on LIBERO. Repeated-evaluation and input protocols are reported in Appendix B.2 .

Method	$M(1)$	$M(n)$	Overall
OpenVLA (current frame)	5.4	3.8	4.7
CronusVLA (dense, $M=20$ )	18.2	9.5	14.3
CondenseVLA ( $M=20$ , $K=6$ )	28.8	12.5	21.6

Table 3: RMBench success rate (%).

M(1)

M(n)

denote memory-demand categories, distinct from history length

M

. Overall averages all nine tasks.

Figure 3: Real-world execution snapshots on the Galaxea R1 Lite platform. The figure shows representative stage-wise trajectories for two long-horizon tasks: Pick and Place Twice and Swap Cubes.

Method	Put Apple on Plate		Open then Close Drawer		Pick and Place Twice		Swap Cubes		Put Objects in Drawer		Avg. Success
	Grasp	Success	Open	Success	PnP Once	Success	First Cube	Success	Obj. 1 Placed	Success
OpenVLA-OFT [ 19 ]	52.0	48.0	56.0	44.0	48.0	16.0	48.0	8.0	12.0	0.0	23.2
$\pi_{0}$ [ 2 ]	72.0	72.0	68.0	60.0	56.0	24.0	64.0	12.0	24.0	16.0	36.8
CronusVLA [ 23 ]	68.0	66.0	52.0	48.0	46.0	40.0	58.0	50.0	28.0	22.0	45.2
CondenseVLA	72.0	70.0	56.0	54.0	52.0	50.0	66.0	62.0	32.0	28.0	52.8

Table 4: Success rate (%) over 50 trials per task and method on Galaxea R1 Lite. Intermediate stages precede final success; Avg. averages final success only. Put Objects in Drawer requires both insertions and drawer closure.

Panel of Figure 4 — Figure 4: (a) Component ablations under the original 24-rollout protocol. The final configuration (labeled “+Reg.”) replaces all-pairs with middle-only slot decorrelation. (b) Success rate under the same ablation protocol and inference speed at fixed $K=6$ ; Baseline denotes dense CronusVLA. Speed (Hz) is the reciprocal of measured policy-forward latency, using an A100-SXM4-80GB, batch size 1, and 10-step DDIM. Appendix C.5 gives the profiling protocol and component breakdown.

$M\backslash K$	$K=3$	$K=6$	$K=10$
$M=10$	59.4	63.5	56.2
$M=20$	57.3	70.8	62.5
$M=30$	53.1	60.4	64.6

Table 5: K/M scaling on Simpler-WidowX. Each cell reports success rate (%).

Method	$M$	$K$	SR (%)	Gap (pp)
CondenseVLA (learned)	20	6	70.8	0.0
Static-Index Selection	20	6	55.2	-15.6
Static-Weight Selection	20	6	54.2	-16.6
Strided History Sampling	20	6	59.4	-11.4

Table 6: History-selection comparisons on Simpler-WidowX; static templates are inference-time replacements. Gap is variant SR minus learned SR, in percentage points (pp).

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Simpler-WidowX	Real-world
Learning rate	$2\times 10^{-5}$	$2\times 10^{-5}$
History length $M$	$20$	$20$
Number of slots $K$	$6$	$6$
Batch size	$64$	$64$
Training epochs	$4$	$17$
Diffusion steps at inference	$10$	$10$

Appendix

Table 7: Run-level training hyperparameters for Simpler-WidowX and real-world experiments.

Setting	Spatial	Object	Goal	Long
Learning rate	$2\times 10^{-5}$	$2\times 10^{-5}$	$2\times 10^{-5}$	$2\times 10^{-5}$
History length $M$	$10$	$10$	$10$	$10$
Number of slots $K$	$3$	$3$	$3$	$3$
Global batch size	$64$	$64$	$64$	$64$
Training epochs	$13$	$21$	$18$	$25$
Diffusion steps at inference	$10$	$10$	$10$	$10$

Appendix

Table 8: Suite-wise LIBERO fine-tuning hyperparameters. Each suite is fine-tuned separately from the same multi-frame post-training initialization.

Module (Parameters)	Submodule	Layer Num	Input Shape	Output Shape
VLM backbone (7.47B)	SigLIP vision encoder	$1+27+1$	$(B\cdot F,3,H,W)$	$(B\cdot F,P,1152)$
	DINOv2 vision encoder	$1+24+1$	$(B\cdot F,3,H,W)$	$(B\cdot F,P,1024)$
	LLaMA2 language backbone	$1+32+0$	$(B\cdot F,P+T,D)$	$(B,F,D)$
MM projector (71.4M)	No-align fused-GELU MLP	$1$	$(B\cdot F,P,2176)$	$(B\cdot F,P,D)$
LHC (100.8M)	Base queries + slot embeddings	$1$	two $(K,D)$ tables	$(K,D)$
	Query MLP (SiLU)	$2$ linear	$(K,D+1)$	$(K,D)$

Appendix

Table 9: Module summary for

M=20,K=6

. Vision encoders and the language model are frozen; the separately listed MM projector, LHC, and decoder are trainable. Counts are reported totals, with LHC verified analytically from its layer definitions. Here

B

is batch size,

F

encoded frames,

P

visual tokens,

T

text tokens,

A

action horizon,

D=4096

, and

d=768

	CronusVLA		CondenseVLA
Task	Mean $\pm$ SD	95% CI	Mean $\pm$ SD	95% CI
Spoon on Towel	$66.7\pm 0.0$	[55.2, 76.5]	$80.6\pm 6.4$	[70.0, 88.0]
Carrot on Plate	$51.4\pm 8.7$	[40.1, 62.6]	$66.7\pm 0.0$	[55.2, 76.5]
Stack Cube	$22.2\pm 6.4$	[14.2, 33.1]	$33.3\pm 8.3$	[23.5, 44.8]
Eggplant in Basket	$94.4\pm 6.4$	[86.6, 97.8]	$98.6\pm 2.4$	[92.5, 99.8]
Avg.	$58.7\pm 1.6$	[52.9, 64.2]	$69.8\pm 1.0$	[64.3, 74.8]

Appendix

Table 10: Simpler fixed-checkpoint evaluation: mean

\pm

evaluation-seed SD (%), and pooled Wilson 95% intervals. Each task uses

3\times 24

rollouts; Avg. pools 288 rollouts for its interval.

	Spoon on Towel		Carrot on Plate		Stack Cube		Eggplant in Basket
Method	Grasp	Success	Grasp	Success	Grasp	Success	Grasp	Success	Avg.
RT-1-X [ 34 ]	16.7	0.0	20.8	4.2	8.3	0.0	0.0	0.0	1.1
RoboVLMs [ 27 ]	70.8	50.0	45.8	37.5	70.8	0.0	91.7	83.3	42.7
Magma [ 45 ]	62.5	37.5	41.7	29.2	79.2	20.8	95.8	91.7	44.8
SpatialVLA [ 38 ]	25.0	20.8	41.7	37.5	79.2	41.7	91.7	83.3	45.8
HAMLET [ 20 ]	91.7	75.0	83.3	62.5	75.0	16.7	100.0	100.0	63.5

Appendix

Table 11: Original Simpler-WidowX detailed results (%): grasp and final success. The local CronusVLA and CondenseVLA rows use the original 24-rollout-per-task evaluation; Avg. averages final success.

Method	Input setting	Spatial	Object	Goal	Long	Avg.
Diffusion Policy [ 7 ]	no state, no wrist	78.3	92.5	68.3	50.5	72.4
Dita [ 14 ]	no state, no wrist	84.2	96.3	85.4	63.8	82.4
OpenVLA [ 18 ]	no state, no wrist	84.7	88.4	79.2	53.7	76.5
TraceVLA [ 54 ]	no state, no wrist	84.6	85.2	75.1	54.1	74.8
SpatialVLA [ 38 ]	no state, no wrist	88.2	89.9	78.6	55.5	78.1
OpenVLA-OFT [ 19 ]	wrist + state	97.6	98.4	97.9	94.5	97.1

Appendix

Table 12: LIBERO success rates (%) under different input protocols. CondenseVLA reports mean

\pm

SD across three evaluation seeds of fixed trained checkpoints; other rows report means. The

\pi_{0.5}

average retains the source precision.

Category	Task	OpenVLA	CronusVLA	CondenseVLA
$M(1)$	Observe and Pick Up	0.0	5.0	12.0
	Rearrange Blocks	5.0	23.0	39.0
	Put Back Block	9.0	35.0	46.0
	Swap Blocks	10.0	21.0	35.0
	Swap T	3.0	7.0	12.0
	Average	5.4	18.2	28.8

Appendix

Table 13: Full RMBench success rates (%). Each task has 100 rollouts, so each task-level percentage also gives the number of successful rollouts. The category averages weight tasks equally; Overall weights all nine tasks equally.

Figure 5: Real-world deployment setup on the Galaxea R1 Lite platform, showing the camera, robot body, and manipulator used in our experiments.

Variant	Pair rule	Avg. success (%)	$\Delta$ (pp)
All-pairs	all off-diagonal pairs	68.8	-2.0
Middle-only	fixed interior indices	70.8	0.0
Dynamic middle masking	sample-adaptive middle slots	52.1	-18.7
Gate-by-attention	attention-similarity gating	64.6	-6.2

Appendix

Table 14: Comparison of slot-redundancy reduction variants on Simpler-WidowX.

\Delta=\mathrm{SR}_{\mathrm{variant}}-\mathrm{SR}_{\mathrm{middle-only}}

is a signed difference in percentage points (pp).

Figure 6: Slot-zero ablation on Simpler-WidowX. Each bar shows the success rate after zeroing one slot at inference time; the dashed line indicates the full-model result. The uneven drop across slots suggests that the learned slots are not fully interchangeable, while the limited degradation for some slots still indicates partial redundancy.

Method	History setting	Avg. success (%)
CronusVLA [ 23 ]	original 6-frame setting	58.4
CronusVLA [ 23 ]	$M=10$	57.3
CronusVLA [ 23 ]	$M=20$	53.1
CronusVLA [ 23 ]	$M=30$	47.9
CondenseVLA	$M=10$ , $K=6$	63.5
CondenseVLA	$M=20$ , $K=6$	70.8

Appendix

Table 15: History-length success rates on Simpler-WidowX under the original 24-rollout-per-task ablation protocol. Runtime is reported separately in Table 18 .

Method	Construction	Avg. success (%)	Gap to learned (pp)
Static-Index Selection	fixed endpoint gather	55.2	-15.6
Static-Weight Selection	fixed endpoint kernel	54.2	-16.6
Strided History Sampling	fixed interval gather	59.4	-11.4
CondenseVLA (learned)	input-adaptive attention	70.8	0.0

Appendix

Table 16: History-selection comparisons at matched history and output budgets. Static-Index and Static-Weight replace LHC at inference without retraining. Gap is variant SR minus learned SR, in percentage points (pp).

Group	Setting	Best SR (%)
Slot regularization	slot-regularization weight = 0.005	64.6
Slot regularization	slot-regularization weight = 0.01	58.3

Appendix

Table 17: Reported best-checkpoint success rates at two alternative slot-regularization weights on Simpler-WidowX.

Method	History	VLM	LHC	Decoder	Sum	Total
Dense CronusVLA	$M=6$	108.37	0.00	106.00	214.37	214.89
Dense CronusVLA	$M=10$	108.11	0.00	110.09	218.20	219.07
CondenseVLA	$M=10,K=6$	108.42	2.26	106.44	217.12	219.62
Dense CronusVLA	$M=20$	108.28	0.00	120.32	228.60	229.52
CondenseVLA	$M=20,K=6$	108.15	4.52	106.85	219.52	220.88
Dense CronusVLA	$M=30$	107.94	0.00	130.54	238.48	239.97

Appendix

Table 18: Policy latency in ms per step under one profiling protocol. Boldface marks lower aggregate values within matched

M

pairs. The original

M=6

dense setting is an unmatched reference.

Panel of Figure 7 — Figure 7: Occlusion overlays on SimplerEnv (Eggplant-in-Basket). Highlighted regions show larger cosine-distance changes in the cognition feature under local occlusion. Each heatmap is normalized separately, so colors do not compare absolute sensitivity across frames.

Figure 8: Task-grouped attention summaries (“episode” in the image refers to a task group). A: per-slot centroid shifts relative to the across-task mean. B: mean effective support standardized across tasks, separately for each slot. C: mean temporal centroids. The views show task-dependent redistribution of temporal position and attention coverage.

Figure 9: Task-grouped slot statistics on Simpler-WidowX (image rows labeled “episode” are task groups). A: mean per-step effective support. B: standard deviation of per-step temporal centroids within each task group. C: normalized entropy of top-attended history indices. Together, the panels illustrate structured but flexible temporal retrieval across tasks.

Method	Put Apple on Plate		Open then Close Drawer		Pick and Place Twice		Swap Cubes		Put Objects in Drawer		Avg. Success
	Grasp	Success	Open	Success	PnP Once	Success	First Cube	Success	Obj. 1 Placed	Success
OpenVLA-OFT [ 19 ]	52.0	48.0	56.0	44.0	48.0	16.0	48.0	8.0	12.0	0.0	23.2
$\pi_{0}$ [ 2 ]	72.0	72.0	68.0	60.0	56.0	24.0	64.0	12.0	24.0	16.0	36.8
CronusVLA [ 23 ]	68.0	66.0	52.0	48.0	46.0	40.0	58.0	50.0	28.0	22.0	45.2
CondenseVLA	72.0	70.0	56.0	54.0	52.0	50.0	66.0	62.0	32.0	28.0	52.8

Appendix

Table 19: Success rate (%) over 50 real-world trials per task and method. All intermediate and final metrics use the same 50-trial denominator; Avg. averages final success only. Put Objects in Drawer requires closure after both insertions.

	CronusVLA		CondenseVLA
Task	Successes (%)	95% CI	Successes (%)	95% CI
Put Apple on Plate	33/50 (66.0)	[52.2, 77.6]	35/50 (70.0)	[56.2, 80.9]
Open then Close Drawer	24/50 (48.0)	[34.8, 61.5]	27/50 (54.0)	[40.4, 67.0]
Pick and Place Twice	20/50 (40.0)	[27.6, 53.8]	25/50 (50.0)	[36.6, 63.4]
Swap Cubes	25/50 (50.0)	[36.6, 63.4]	31/50 (62.0)	[48.2, 74.1]
Put Objects in Drawer	11/50 (22.0)	[12.8, 35.2]	14/50 (28.0)	[17.5, 41.7]

Appendix

Table 20: Real-world final success over 50 trials per task. Counts, rates (%), and Wilson 95% intervals are reported for CronusVLA and CondenseVLA. Aggregate pools 250 trials across the five tasks.

Panel of Figure 10 — Figure 10: Representative successful real-world rollouts. Put Objects in Drawer ends with drawer closure after both objects are inserted. The long-horizon examples highlight the need for temporally consistent stage tracking across multi-step execution.

Panel of Figure 11 — Figure 11: Representative real-world failure cases. In the simple-task portion, Put Apple on Plate typically fails when the apple is not grasped successfully, and Open then Close Drawer fails when the policy misses the drawer handle during opening or closing, sometimes leading to direct pushing of the drawer front instead of handle-based interaction. In the long-horizon portion, Pick and Place Twice may fail by selecting the wrong target plate; Swap Cubes may fail by grasping the wrong cube and returning it; Put Objects in Drawer may fail through object–drawer collision during placement or through the same handle-grasp failure modes observed in the drawer task.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,194 papers

ActMem-VLA: Action-History Memory with Dual-Expert Denoising for Long-Horizon Vision-Language-Action Policies
Sep 29, 2026Yaxin Zhao, Dianye Huang, Chenwei Wang +2Memory-Augmented VLMs Long-Horizon Robotic Manipulation
Harbin Institute of Technology, Harbin, China. · Medical Intelligence and Robotic Cognition (MIRoC) Lab, Department of Mechanical Engineering, The University of Hong Kong (HKU), Hong Kong SAR, China. · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China
EvoMem-VLA: State-Evolution Memory for Long-Horizon Robot Manipulation
Oct 4, 2026Yuheng Na, Zhide Zhong, Junjie He +7Memory-Augmented VLMs Long-Horizon Robotic Manipulation
Xi’an Jiaotong University · The Hong Kong University of Science and Technology (Guangzhou) · Xbotics Embodied AI Community +1
Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation
Jul 8, 2026Hongyu Qu, Jianzhe Gao, Xiaobin Hu +6Memory-Augmented VLMs Robotic Manipulation
Nanjing University of Science and Technology · Zhejiang University · National University of Singapore

Sep 29, 2026J/K move · Enter open · S save

ActMem-VLA: Action-History Memory with Dual-Expert Denoising for Long-Horizon Vision-Language-Action Policies

Yaxin Zhao, Dianye Huang, Chenwei Wang, Chenguang Yang+1

Harbin Institute of Technology, Harbin, China. · Medical Intelligence and Robotic Cognition (MIRoC) Lab, Department of Mechanical Engineering, The University of Hong Kong (HKU), Hong Kong SAR, China. · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China

Memory-Augmented VLMs Long-Horizon Robotic Manipulation Visuomotor Policy Learning Vision-Language-Action Models

Open paper AskPDF arXiv

Vision-language-action (VLA) models have driven rapid progress in robotic manipulation, demonstrating strong fine-grained control and promising performance on long-horizon tasks. However, many existing VLAs lack explicit access to interaction history, making them vulnerable to perceptual aliasing: similar current observations and robot states at different task stages may induce action ambiguity and lower success rate. Existing methods incorporate temporal or progress cues through feature conditioning, action-prior modification, or sampling guidance. However, methods that jointly fine-tune memory modules and the base VLA incur additional policy-training costs, motivating the separation of trainable history-conditioned steering from frozen base-policy refinement. We propose ActMem-VLA, a dual-expert handover architecture that augments a frozen, fine-tuned VLA with a memory plugin comprising a Mamba-based memory module and a lightweight PreAction Expert (PAE). Specifically, Mamba encodes executed-action history into memory that conditions PAE alongside current context. With these inputs, PAE steers task progression during early, high-noise denoising, then passes the partially denoised action to the frozen Action Expert (AE) to refine action details during the remaining low-noise steps. The fine-tuned base VLA remains frozen throughout training, while only the Mamba module and PAE are jointly optimized. On LIBERO-Mem, ActMem-VLA achieves 80.8% average success across all ten tasks, compared with 65.2% for

π_{0.5}

and 49.5% for MemoryVLA, while introducing only 3.45% additional parameters. Across four real-world tasks, it improves the average success rate over

π_{0.5}

by 28.8%.

Abstract

Vision-Language-Action Models

Explore similar work

ActMem-VLA: Action-History Memory with Dual-Expert Denoising for Long-Horizon Vision-Language-Action Policies

EvoMem-VLA: State-Evolution Memory for Long-Horizon Robot Manipulation

Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation