FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models

Jiuyi Xu; Chengyuan Qian; Zhengzhong Tu; Yang Sui; Yangming Shi

cs.ROOct 15, 2026

FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models

Authors: Jiuyi Xu, Chengyuan Qian, Zhengzhong Tu, Yang Sui, Yangming Shi

Organizations: Colorado School of Mines · Texas A&M University · Microsoft AI

Abstract

Vision-Language-Action (VLA) models present a promising paradigm for general-purpose robot manipulation, but closed-loop deployment is often limited by inference efficiency: at each timestep, high-resolution, often multi-view images are encoded into hundreds of visual tokens that dominate transformer computation and latency. Token compression via attention is a natural solution but is challenging for VLA control because (1) decision-critical tokens are determined by the task, not generic visual saliency, and (2) the token budget varies substantially across steps, so static metrics and budgets waste computation or discard control-critical tokens. We introduce FastVLA, a training-free, task-aware, and temporally adaptive visual token compression framework. At each step, FastVLA scores token importance via task-to-vision attention and adaptively sets the compression rate from the importance-distribution uniformity, measured by an inverse-Simpson metric with online sliding-window calibration. We evaluate FastVLA on OpenVLA, OpenVLA-OFT, and CogACT across LIBERO and SIMPLER, and on a physical Franka Research 3 robot. In simulation, it reduces LLM-backbone FLOPs by 35-59% and action latency by 9-32% (a 1.09 $\times$ -1.46 $\times$ speedup) while maintaining task success, achieving a comparable or better efficiency-performance trade-off than existing plug-and-play token compression/pruning/reuse baselines. On the real robot, it cuts end-to-end latency by 20% (1.25 $\times$ speedup) at comparable success. Code is available at https://github.com/jiuyixu25/FastVLA.

Figures & tables

Figure 1: Overview of FastVLA, our temporally adaptive, training-free visual token compression framework for Vision-Language-Action models. At each action prediction step, FastVLA estimates task-conditioned patch importance via attention, measures distributional uniformity over selected vision tokens, and dynamically adjusts the compression rate using an online sliding-window calibration. By retaining task-relevant yet information-diverse tokens across timesteps, our approach reduces LLM backbone computation while preserving action performance.

Figure 2: Average success rate on the LIBERO benchmark under different visual token pruning rates, comparing three attention-based importance scoring strategies: task-to-vision, vision-to-vision, and all-to-vision.

Figure 3: Visual token importance distributions across action steps in a single trial on the LIBERO-Long benchmark.

Figure 4: Temporal comparison of Shannon entropy and inverse Simpson concentration measures over action prediction steps in a single trial on the LIBERO-Long benchmark.

Method	Success Rate (%) $\uparrow$					FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
Method	Spatial	Object	Goal	10	Average	FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
OpenVLA	85.8	88.8	77.4	53.4	76.4	1.86	186.64	5.36	42.28	1.00 $\times$
+ VLA-Cache	86.0	88.0	75.4	54.4	76.0	1.36	203.37	4.92	31.83	0.92 $\times$
+ FlashVLA ∗	82.6	86.6	78.8	46.8	73.7	1.08	-	-	-	1.56 $\times$
+ FastVLA	85.3 $\pm$ 0.1	87.2 $\pm$ 0.2	77.6 $\pm$ 0.2	53.2 $\pm$ 0.4	75.8 $\pm$ 0.2	1.20	161.53	6.19	26.91	1.16 $\times$
OpenVLA-OFT	98.2	98.2	97.6	94.0	97.0	3.98	118.52	67.50	65.99	1.00 $\times$

Table 1: Comparison of different VLA acceleration methods on the LIBERO benchmark. For FastVLA,

\rho

is set per suite and backbone: on OpenVLA-OFT,

\rho{=}0.8

(Spatial),

0.7

(Object),

0.75

(Goal),

0.6

(10); on OpenVLA,

\rho{=}0.35

(Spatial),

0.5

(Object),

0.4

(Goal),

0.3

(10). The remaining hyperparameters are fixed throughout:

\tau{=}0.1

and a calibration window

|W|{=}10

. FastVLA SR results are averaged over three random seeds and reported as mean

\pm

standard error. Methods marked with ∗ could not be reproduced in our setup, and their results are quoted from the original papers for reference only.

Panel of Figure 5 — Figure 5: Qualitative rollouts on two manipulation domains. Top: a long-horizon task from the LIBERO-Long suite (simulation). Bottom: a real-world pick-and-place task. In each strip, eight frames are sampled uniformly over time (left to right) from the external third-person camera, where the full observation is shown alongside its masked/pruned counterpart processed by FastVLA . We further provide complete rollout videos with pruning effect by FastVLA in the supplementary material.

SIMPLER	Method	Success Rate (%) $\uparrow$					FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
SIMPLER	Method	PickCan	MoveNear	Drawer	DrawerApple	Average	FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
Visual Matching	CogACT-base	93.7	86.7	75.9	53.7	77.5	1.85	174.48	5.73	82.53	1.00 $\times$
Visual Matching	+ FastVLA	93.7 $\pm$ 2.0	85.4 $\pm$ 0.7	75.5 $\pm$ 0.9	53.6 $\pm$ 0.2	77.1 $\pm$ 0.8	1.16	159.59	6.27	66.10	1.09 $\times$
Variant Aggregation	CogACT-base	89.1	76.7	50.8	27.0	60.9	1.81	174.94	5.72	82.74	1.00 $\times$
Variant Aggregation	+ FastVLA	89.5 $\pm$ 1.3	77.2 $\pm$ 1.0	52.4 $\pm$ 0.3	28.3 $\pm$ 0.4	61.9 $\pm$ 0.6	1.13	157.06	6.37	65.34	1.11 $\times$

Table 2: Comparison of different VLA acceleration methods on the SIMPLER benchmark. FastVLA SR results are averaged over three random seeds and reported as mean

\pm

standard error. For FastVLA we set

\rho{=}0.7

(PickCan),

0.5

(MoveNear), and

0.3

(Drawer, DrawerApple), with

\tau{=}0.1

and a calibration window

|W|{=}10

fixed throughout.

Task	Method	Success Rate $\uparrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	Speedup $\uparrow$
Pick	OpenVLA-OFT	84.0%	121.13	66.04	1.0 $\times$
Pick	+ FastVLA	80.0%	96.90	82.56	1.25 $\times$
Pick and place	OpenVLA-OFT	68.0%	120.01	66.67	1.0 $\times$
Pick and place	+ FastVLA	66.0%	97.77	81.82	1.23 $\times$

Table 3: Real-world evaluation on a Franka Research 3 robot.

Method	Spatial		Object		Goal		10
Method	Tokens	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$
Fixed @ nominal	102 (80%)	96.7 $\pm$ 0.1	154 (70%)	97.0 $\pm$ 0.2	128 (75%)	95.1 $\pm$ 0.5	205 (60%)	91.9 $\pm$ 0.4
Fixed, budget-matched	105	96.5 $\pm$ 0.1	157	97.1 $\pm$ 0.1	132	95.7 $\pm$ 0.3	207	92.0 $\pm$ 0.3
Adaptive	104.3	97.8 $\pm$ 0.1	156.3	98.0 $\pm$ 0.1	131.9	97.0 $\pm$ 0.5	206.8	93.1 $\pm$ 0.5

Table 4: Compute-matched comparison of fixed and adaptive token selection across four LIBERO suites. Tokens (and target compression rate) are reported separately for each suite. Results are averaged over three random seeds and reported as mean

\pm

standard error.

Calibration	Spatial		Object
Calibration	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens
Min–Max	97.8	104.3	98.0	156.3
Rank	97.6	103.0	98.0	159.5
Calibration	Goal		10
Calibration	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens
Min–Max	97.0	131.9	93.1	206.8

Table 5: Comparison of min–max and rank-based calibration across four LIBERO suites. Success rates and token usage are reported separately for each suite.

Diversity Metric	Goal		10
Diversity Metric	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens
Shannon	94.0	129.6	90.8	205.5
Gini	95.3	130.3	91.1	205.2
Inverse-Simpson	97.0	131.9	93.1	206.8

Table 6: Ablation of diversity metrics across the LIBERO-Goal and LIBERO-10 suites. Success rates and token usage are reported separately for each suite.

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 174 papers

Oct 6, 2026

VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models

Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action-level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full-context teacher, with ground-truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real-world manipulation tasks show that VLA-ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance-efficiency trade-off than existing frozen-VLA pruning methods and demonstrate the value of action-level supervision for visual token selection. Code is available at https://github.com/du-owen/VLA-ACL.

Owen Du, Yang Yue, Jie Zhang +3

ETH Zurich · Tsinghua University

Efficient VLA Models VLMs for Robotics Visual Token Pruning

Ask PDF

Mar 11, 2026

DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inference

Vision-Language-Action (VLA) models enable generalist robotic manipulation but suffer from high inference latency. This bottleneck stems from the massive number of visual tokens processed by large language backbones. Existing methods either prune or merge tokens uniformly, degrading the spatial reasoning essential for robotic control. We present DepthCache, a training-free framework that leverages depth as a structural prior for visual token compression. It partitions observations into depth-based regions and applies spatially differentiated merge ratios, preserving the near-field workspace while compressing the distant background. To exploit temporal redundancy, DepthCache distributes the merging process across consecutive frames, ensuring consistent representations while reducing per-step computation. A motion-adaptive pipeline further optimizes auxiliary view compression based on end-effector dynamics. The framework requires no model modification, generalizing across diverse VLA architectures. On the LIBERO benchmark, DepthCache achieves up to 1.28x inference speedup with less than 1% average success rate degradation across three VLA models (pi_0.5, OpenVLA, GR00T), whereas pruning and merging baselines incur 4--24% degradation at comparable compression. Real-world experiments on a physical manipulator demonstrate that DepthCache enables faster task throughput and more responsive closed-loop control in latency-sensitive scenarios.

Yuquan Li, Lianjie Ma, Han Ding +1

School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. · School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China.

Efficient VLA Model Inference Visual Token Compression Vision-Language-Action Models Visual Token Merging VLMs for Robotics

Ask PDF

Jul 14, 2026

Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference

Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment. We identify two primary sources of temporal redundancy in existing VLA pipelines: repeated visual encoding of highly similar consecutive frames and multi step iterative sampling in diffusion based policies. To address this, we propose a system level acceleration strategy that reduces computation in both perception and action generation. On the perception side, we incrementally update only tokens corresponding to dynamic scene regions instead of re-encoding entire frames. On the policy side, we compress diffusion sampling into a compact 2-step schedule through efficiency oriented training while preserving action precision. Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance, achieving up to 98% success rate on general manipulation benchmarks. Our codes will be released on Github.

Yuzhou Wu, Yuxin Zheng, Muchun Niu +6

1Tianji KernalMind co ltd · 2Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · 3Shanghai Jiao Tong University, Shanghai, China +2

Efficient VLA Model Inference Efficient VLA Models Robotic Manipulation Vision-Language-Action Models Diffusion Model Inference Acceleration

Ask PDF

Abstract

Efficient VLA Model Inference

Explore similar work

VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models

DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inference

Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference