FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models

Jiuyi Xu; Chengyuan Qian; Zhengzhong Tu; Yang Sui; Yangming Shi

cs.ROOct 15, 2026

FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models

Authors: Jiuyi Xu, Chengyuan Qian, Zhengzhong Tu, Yang Sui, Yangming Shi

Organizations: Colorado School of Mines · Texas A&M University · Microsoft AI

Abstract

Vision-Language-Action (VLA) models present a promising paradigm for general-purpose robot manipulation, but closed-loop deployment is often limited by inference efficiency: at each timestep, high-resolution, often multi-view images are encoded into hundreds of visual tokens that dominate transformer computation and latency. Token compression via attention is a natural solution but is challenging for VLA control because (1) decision-critical tokens are determined by the task, not generic visual saliency, and (2) the token budget varies substantially across steps, so static metrics and budgets waste computation or discard control-critical tokens. We introduce FastVLA, a training-free, task-aware, and temporally adaptive visual token compression framework. At each step, FastVLA scores token importance via task-to-vision attention and adaptively sets the compression rate from the importance-distribution uniformity, measured by an inverse-Simpson metric with online sliding-window calibration. We evaluate FastVLA on OpenVLA, OpenVLA-OFT, and CogACT across LIBERO and SIMPLER, and on a physical Franka Research 3 robot. In simulation, it reduces LLM-backbone FLOPs by 35-59% and action latency by 9-32% (a 1.09 $\times$ -1.46 $\times$ speedup) while maintaining task success, achieving a comparable or better efficiency-performance trade-off than existing plug-and-play token compression/pruning/reuse baselines. On the real robot, it cuts end-to-end latency by 20% (1.25 $\times$ speedup) at comparable success. Code is available at https://github.com/jiuyixu25/FastVLA.

Figures & tables

Figure 1: Overview of FastVLA, our temporally adaptive, training-free visual token compression framework for Vision-Language-Action models. At each action prediction step, FastVLA estimates task-conditioned patch importance via attention, measures distributional uniformity over selected vision tokens, and dynamically adjusts the compression rate using an online sliding-window calibration. By retaining task-relevant yet information-diverse tokens across timesteps, our approach reduces LLM backbone computation while preserving action performance.

Figure 2: Average success rate on the LIBERO benchmark under different visual token pruning rates, comparing three attention-based importance scoring strategies: task-to-vision, vision-to-vision, and all-to-vision.

Figure 3: Visual token importance distributions across action steps in a single trial on the LIBERO-Long benchmark.

Figure 4: Temporal comparison of Shannon entropy and inverse Simpson concentration measures over action prediction steps in a single trial on the LIBERO-Long benchmark.

Method	Success Rate (%) $\uparrow$					FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
Method	Spatial	Object	Goal	10	Average	FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
OpenVLA	85.8	88.8	77.4	53.4	76.4	1.86	186.64	5.36	42.28	1.00 $\times$
+ VLA-Cache	86.0	88.0	75.4	54.4	76.0	1.36	203.37	4.92	31.83	0.92 $\times$
+ FlashVLA ∗	82.6	86.6	78.8	46.8	73.7	1.08	-	-	-	1.56 $\times$
+ FastVLA	85.3 $\pm$ 0.1	87.2 $\pm$ 0.2	77.6 $\pm$ 0.2	53.2 $\pm$ 0.4	75.8 $\pm$ 0.2	1.20	161.53	6.19	26.91	1.16 $\times$
OpenVLA-OFT	98.2	98.2	97.6	94.0	97.0	3.98	118.52	67.50	65.99	1.00 $\times$

Table 1: Comparison of different VLA acceleration methods on the LIBERO benchmark. For FastVLA,

\rho

is set per suite and backbone: on OpenVLA-OFT,

\rho{=}0.8

(Spatial),

0.7

(Object),

0.75

(Goal),

0.6

(10); on OpenVLA,

\rho{=}0.35

(Spatial),

0.5

(Object),

0.4

(Goal),

0.3

(10). The remaining hyperparameters are fixed throughout:

\tau{=}0.1

and a calibration window

|W|{=}10

. FastVLA SR results are averaged over three random seeds and reported as mean

\pm

standard error. Methods marked with ∗ could not be reproduced in our setup, and their results are quoted from the original papers for reference only.

Panel of Figure 5 — Figure 5: Qualitative rollouts on two manipulation domains. Top: a long-horizon task from the LIBERO-Long suite (simulation). Bottom: a real-world pick-and-place task. In each strip, eight frames are sampled uniformly over time (left to right) from the external third-person camera, where the full observation is shown alongside its masked/pruned counterpart processed by FastVLA . We further provide complete rollout videos with pruning effect by FastVLA in the supplementary material.

SIMPLER	Method	Success Rate (%) $\uparrow$					FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
SIMPLER	Method	PickCan	MoveNear	Drawer	DrawerApple	Average	FLOPs (T) $\downarrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	CUDA Late. (ms) $\downarrow$	Speedup $\uparrow$
Visual Matching	CogACT-base	93.7	86.7	75.9	53.7	77.5	1.85	174.48	5.73	82.53	1.00 $\times$
Visual Matching	+ FastVLA	93.7 $\pm$ 2.0	85.4 $\pm$ 0.7	75.5 $\pm$ 0.9	53.6 $\pm$ 0.2	77.1 $\pm$ 0.8	1.16	159.59	6.27	66.10	1.09 $\times$
Variant Aggregation	CogACT-base	89.1	76.7	50.8	27.0	60.9	1.81	174.94	5.72	82.74	1.00 $\times$
Variant Aggregation	+ FastVLA	89.5 $\pm$ 1.3	77.2 $\pm$ 1.0	52.4 $\pm$ 0.3	28.3 $\pm$ 0.4	61.9 $\pm$ 0.6	1.13	157.06	6.37	65.34	1.11 $\times$

Table 2: Comparison of different VLA acceleration methods on the SIMPLER benchmark. FastVLA SR results are averaged over three random seeds and reported as mean

\pm

standard error. For FastVLA we set

\rho{=}0.7

(PickCan),

0.5

(MoveNear), and

0.3

(Drawer, DrawerApple), with

\tau{=}0.1

and a calibration window

|W|{=}10

fixed throughout.

Task	Method	Success Rate $\uparrow$	Latency (ms) $\downarrow$	Control Freq. (Hz) $\uparrow$	Speedup $\uparrow$
Pick	OpenVLA-OFT	84.0%	121.13	66.04	1.0 $\times$
Pick	+ FastVLA	80.0%	96.90	82.56	1.25 $\times$
Pick and place	OpenVLA-OFT	68.0%	120.01	66.67	1.0 $\times$
Pick and place	+ FastVLA	66.0%	97.77	81.82	1.23 $\times$

Table 3: Real-world evaluation on a Franka Research 3 robot.

Method	Spatial		Object		Goal		10
Method	Tokens	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$
Fixed @ nominal	102 (80%)	96.7 $\pm$ 0.1	154 (70%)	97.0 $\pm$ 0.2	128 (75%)	95.1 $\pm$ 0.5	205 (60%)	91.9 $\pm$ 0.4
Fixed, budget-matched	105	96.5 $\pm$ 0.1	157	97.1 $\pm$ 0.1	132	95.7 $\pm$ 0.3	207	92.0 $\pm$ 0.3
Adaptive	104.3	97.8 $\pm$ 0.1	156.3	98.0 $\pm$ 0.1	131.9	97.0 $\pm$ 0.5	206.8	93.1 $\pm$ 0.5

Table 4: Compute-matched comparison of fixed and adaptive token selection across four LIBERO suites. Tokens (and target compression rate) are reported separately for each suite. Results are averaged over three random seeds and reported as mean

\pm

standard error.

Calibration	Spatial		Object
Calibration	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens
Min–Max	97.8	104.3	98.0	156.3
Rank	97.6	103.0	98.0	159.5
Calibration	Goal		10
Calibration	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens
Min–Max	97.0	131.9	93.1	206.8

Table 5: Comparison of min–max and rank-based calibration across four LIBERO suites. Success rates and token usage are reported separately for each suite.

Diversity Metric	Goal		10
Diversity Metric	SR (%) $\uparrow$	Tokens	SR (%) $\uparrow$	Tokens
Shannon	94.0	129.6	90.8	205.5
Gini	95.3	130.3	91.1	205.2
Inverse-Simpson	97.0	131.9	93.1	206.8

Table 6: Ablation of diversity metrics across the LIBERO-Goal and LIBERO-10 suites. Success rates and token usage are reported separately for each suite.

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 174 papers

VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models
Oct 6, 2026Owen Du, Yang Yue, Jie Zhang +3Efficient VLA Models VLMs for Robotics
ETH Zurich · Tsinghua University
DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inference
Mar 11, 2026Yuquan Li, Lianjie Ma, Han Ding +1Efficient VLA Model Inference Visual Token Compression
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. · School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China.
Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference
Jul 14, 2026Yuzhou Wu, Yuxin Zheng, Muchun Niu +6Efficient VLA Model Inference Efficient VLA Models
1Tianji KernalMind co ltd · 2Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · 3Shanghai Jiao Tong University, Shanghai, China +2

Oct 6, 2026J/K move · Enter open · S save

VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models

Owen Du, Yang Yue, Jie Zhang, Jiaqi Pi+2

ETH Zurich · Tsinghua University

Efficient VLA Models VLMs for Robotics Visual Token Pruning

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action-level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full-context teacher, with ground-truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real-world manipulation tasks show that VLA-ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance-efficiency trade-off than existing frozen-VLA pruning methods and demonstrate the value of action-level supervision for visual token selection. Code is available at https://github.com/du-owen/VLA-ACL.

Abstract

Efficient VLA Model Inference

Explore similar work

VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models

DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inference

Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference