Pruning is a typical acceleration technique for compute-bound models by removing computation on unimportant values. Recently, it has been applied to accelerate Vision-Language-Action (VLA) model inference. However, existing acceleration methods focus on local information from the current action step and ignore the global context, leading to >20% success rate drop and limited speedup in some scenarios. In this paper, we point out spatial-temporal consistency in VLA tasks: input images in consecutive steps exhibit high similarity, and propose the key insight that token selection should combine local information with global context of the model. Based on this, we propose SpecPrune-VLA, a training-free, two-level pruning method with heuristic control. (1) Action-level static pruning. We leverage global history and local attention to statically reduce visual tokens per action. (2) Layer-level dynamic pruning. We prune tokens adaptively per layer based on layer-wise importance. (3) Lightweight action-aware controller: We classify actions as coarse- or fine-grained by the speed of the end effector and adjust pruning aggressiveness accordingly. Extensive experiments show that SpecPrune-VLA achieves up to 1.57× speedup in LIBERO simulation and 1.70× on real-world tasks, with negligible success rate degradation.
Figures & tables
Figure 1: (a) The mainstream inference dataflow of VLA models. (b) Latency breakdown in four typical VLA models during each action generation. * In DB-OFT, we report the time of LLM backbone and Action Expert per denoising step, with the tokenizer’s single forward pass time averaged evenly across the 10 steps. (c) The practical arithmetic intensity of four models in different cases in the roofline model of NVIDIA A800 GPU.
Figure 2: Overview of SpecPrune-VLA. We prune the visual tokens with global and local information with a lightweight action-aware controller.
Figure 3: Insight 1: (a) Layers of different depth focus on different information. (b)(c)(d) In pick and place task, random pruning causes important tokens missed out and pruning based on global attention retain important tokens and ensure success rate. (e) The performance of different guiding layer selection. Insight 2: (a) The recall score of Top-30 globally important token sets of consecutive inference remains high in different camera view. (b) The visualization of globally important tokens between consecutive steps.
Figure 4: (I) Attention entropy differs across layers, but are similar throughout the task. (II) (a) Comparison of the hitrate between leveraging the first one, two, and three layers. (b) LLM latency comparison between leveraging the first one, two, and three layers.
Figure 5: Adaptive selection strategy for frame comparison.
Figure 6: (a) The task process consists of four stages, categorized into coarse- and fine-grained actions based on velocity. (b) Typical failures in fine-grained stages.
Figure 7: (a) and (b) Ablation study on base K value. K values are chosen to maximize the Recall rate. (c) and (d) Ablation study on prune rate.
LIBERO benchmark
Success Rate (%)
Avg. Speedup
Avg. SR(%)
FLOPs
Method
Spatial
Object
Goal
Long
OpenVLA-OFT
97.6%
96.5%
97.9%
94.5%
1.00 ×
96.6%
100%
+ FastV [ECCV24]
94.6%
95.8%
94.0%
88.8%
1.44 ×
93.3%
57%
+ DivPrune [CVPR25]
92.4%
91.2%
89.0%
84.8%
1.46 ×
89.4%
54%
+ SparseVLM [ICML25]
96.8%
94.2%
97.6%
93.6%
1.28 ×
95.6%
77%
+ VLA-Cache [NIPS25]
99.0%
97.7%
97.4%
93.6%
1.07 ×
96.9%
83%
Table 1: Performance Evaluation (Success Rate and Average Speedup)
Figure 8: Ablation study of three techniques in LIBERO-spatial
Real world task
Success Rate (%)
Avg. Latency (ms)
Avg. Speedup
Method
Pick&Place
TransferTube
PickUpCup
MultiCubeTask
OpenVLA-OFT
96.7%
85.0%
91.7%
95.0%
187.7
1.00×
Ours
96.7%
82.0%
90.0%
95.0%
110.2
1.70×
Table 2: Performance comparison on real-world robot tasks.
Table 11
Figure 9: Extended evaluation on NVIDIA 3090 GPU
Figure 10: Left: Real world robot, a Flexiv Rizon4 arm equipped with a gripper and three Intel cameras. The cameras are mounted separately on the wrist, on its side, and above its head. Right: Four real world evaluation tasks.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
SimplerEnv
Success Rate (%)
Avg. Speedup
Avg. SR(%)
Method
Spatial
Goal
Object
Long
OpenVLA-OFT
92.8%
97.2%
97.6%
95.2%
1.00 ×
95.7%
+ Ours ( α =0.8)
92.8%
96.6%
97.2%
94.4%
1.48 ×
95.3%
Appendix
Table 5: Performance Evaluation (Success Rate and Speedup) on RTX 4090 GPU
Figure 11: Performance evaluation on LIBERO-plus and VLA-Arena benchmark.
Figure 12: The task process is categorized into coarse- and fine-grained actions based on velocity.
SimplerEnv Visual Matching
Success Rate (%)
Avg. Speedup
Avg. SR(%)
Method
PickCokeCan
MoveNear
Open/CloseDrawer
AppleDrawer
CogACT
91.3%
85.0%
71.8%
50.8%
1.00 ×
74.7%
+ FastV [ECCV24]
92.6%
81.4%
69.8%
52.4%
1.21 ×
74.1%
+ VLA-Cache [NIPS25]
92.0%
83.3%
70.5%
51.6%
1.20 ×
74.4%
+ Ours ( α =0.8)
91.3%
83.8%
71.6%
50.2%
1.28 ×
74.2%
+ Ours&SC ( α =0.8)
91.7%
83.8%
71.2%
50.0%
1.42 ×
74.1%
Appendix
Table 6: Performance Evaluation of CogACT (Success Rate and Average Speedup) in the SimplerEnv benchmark, visual matching task
Table 7: Ablation study of success rate under different threshold settings on LIBERO-spatial.
Figure 13: The prefilling latency of eager attention and flash attention under different input length on A800 GPU. When original input length is less than 2048 (4 images) and 30% sparsity is applied on input, eager attention achieves consistently lower latency.
Prune Ratio α
0.6
0.8
1.0
Pruned Tokens
382
360
336
Visual Retention
25.4%
29.8%
34.4%
Appendix
Table 8: Number of pruned tokens and visual retention rate under different prune ratios.
Similarity calculation
First 2 layers
Top-k sorting
Attention Entropy
Our Method
Original Model
1.8±0.1ms
5.5±0.2ms
2.5±0.1ms
1.0±0.2ms
72–78ms
109.0ms
Appendix
Table 9: Latency breakdown per module and comparison with the original model.
Method
Latency (ms) / Speedup
Avg. Latency(ms)
Avg. Speedup
Spatial
Object
Goal
Long
OpenVLA-OFT
109.0 / 1.00 ×
109.0 / 1.00 ×
109.0 / 1.00 ×
109.0 / 1.00 ×
109.0
1.00 ×
SparseVLM
85.3 / 1.28 ×
85.3 / 1.28 ×
85.3 / 1.28 ×
85.3 / 1.28 ×
85.3
1.28 ×
VLA-Cache
101.8 / 1.07 ×
101.8 / 1.07 ×
101.8 / 1.07 ×
101.8 / 1.07 ×
101.8
1.07 ×
EfficientVLA
71.7 / 1.52 ×
71.7 / 1.52 ×
71.7 / 1.52 ×
71.7 / 1.52 ×
71.7
1.52 ×
Ours( α = 0.8)
74.8 / 1.46 ×
74.5 / 1.46 ×
74.8 / 1.46 ×
75.8 / 1.44 ×
75.1
1.46 ×
Appendix
Table 10: Performance Evaluation (Latency and Speedup)
Prune Ratio α
Success Rate (%) / Speedup
Avg. SR (%)
Avg. Speedup
Spatial
Object
Goal
10
None (OpenVLA-OFT)
97.6 / 1.00 ×
96.5 / 1.00 ×
97.9 / 1.00 ×
94.5 / 1.00 ×
96.6
1.00 ×
α =1.0
97.6 / 1.41 ×
96.3 / 1.43 ×
97.9 / 1.40 ×
94.0 / 1.38 ×
96.5
1.40 ×
α =0.8
97.6 / 1.46 ×
95.8 / 1.46 ×
97.7 / 1.46 ×
93.4 / 1.44 ×
96.1
1.46 ×
α =0.6
97.4 / 1.52 ×
93.2 / 1.50 ×
96.0 / 1.52 ×
92.2 / 1.51 ×
94.7
1.51 ×
Appendix
Table 11: Ablation study on prune rate α . Success rates (%) and speedup factors are reported per task. Our method achieves optimal trade-off at α=0.8 , achieving 1.46× average speedup with minimal accuracy drop.
Hyperparameter
Value
# GPUs
8 × NVIDIA H200 (141GB VRAM)
Learning rate (LR)
5e-4 (decays to 5e-5 after 50K steps for all tasks)
Total batch size
64 (8 per GPU)
# Train steps
“Pick up the cube and place it in the pan.”: 80K
“Remove the cube from the tube rack.”: 100K
“Pick up the purple cube and put it in the pot.”: 80K
Appendix
Table 12: OpenVLA-OFT hyperparameters for real-world experiments. Includes parallel decoding, action chunking, continuous actions with L1 regression, and additional inputs (wrist/side cameras + robot state).
Figure 14: Retained tokens during tasks across four datasets and different views.
Figure 15: The real-world experiment visualization of OpenVLA-OFT and our method across various tasks.
Real-time inference of vision-language-action (VLA) models is essential for robotic control. While visual token pruning has shown strong potential for accelerating inference, most existing methods mainly base pruning decisions on shallow-layer cues and risk discarding visual information required by deep layers. To address this issue, we propose SAFE-Pruner, a plug-and-play pruning framework that incorporates attention cues of future layers into pruning decisions. Specifically, we identify semantic attention consistency, the tendency that VLA models concentrate their attention probability mass on the same semantic entity across execution steps. Based on this observation, we design a forward-looking strategy to forecast the token saliency in deep layers, which prevents the premature removal of critical tokens and leads to more stable acceleration. We further introduce an adaptive subtask division strategy to detect abrupt attention shifts, thereby improving forecasting accuracy and pruning reliability. Extensive experiments in simulation and real-world settings demonstrate that our method achieves up to 1.89x speedup with a minimal degradation in success rate of less than 1.7%, while outperforming state-of-the-art methods by up to 1.9%.
Shilin Ma, Chubin Zhang, Changyuan Wang +6
Tsinghua Shenzhen International Graduate School, Tsinghua University · GigaAI
Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action-level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full-context teacher, with ground-truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real-world manipulation tasks show that VLA-ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance-efficiency trade-off than existing frozen-VLA pruning methods and demonstrate the value of action-level supervision for visual token selection. Code is available at https://github.com/du-owen/VLA-ACL.
Visual token pruning is an effective way to accelerate vision-language models and is especially useful for vision-language-action (VLA) inference, where many visual tokens must be processed before predicting robot actions. Existing pruning methods usually estimate which tokens can be pruned based on attention scores or feature diversity, retaining tokens that are either highly attended or visually different from others. However, most of them use fixed pruning schedules, such as pruning once at a preset layer or pruning at uniformly spaced layers. Such schedules can be risky for VLA models, because the model may not know which visual regions matter for the action in early layers. Tokens that look unimportant at first may become useful after the model combines visual observations with the language instruction. In this work, we propose SAPrune, a training-free visual token pruning framework for efficient VLA inference. Instead of pruning at fixed layers, SAPrune uses a small calibration set to observe how action-to-visual attention changes across layers, and chooses pruning layers only after the attention pattern becomes more reliable. At each selected layer, SAPrune applies a dual-path pruning rule: one path protects strongly attended visual tokens from pruning, while the other prevents useful surrounding context from being discarded. Experiments on LIBERO, SIMPLER, and real-world robotic tasks show that SAPrune prunes 87.5% of visual tokens and achieves up to 1.718x inference speedup while maintaining competitive task success rates.