ActPrune: Action-Grounded Visual Token Pruning for Efficient Vision-Language-Action Models
Authors: Sungwon Moon, Minji Kim, Hyemin Boo, Daeun Song, Jiyoung Lee
Organizations: Human-Centered Artificial Intelligence Research Institute, Ewha Womans University · Division of Artificial Intelligence & Software, Ewha Womans University
Vision-language-action (VLA) models perform versatile manipulation tasks from language instructions, but their inference cost limits how fast a robot can respond in closed-loop control. Visual token pruning reduces this cost; however, prior methods estimate token redundancy from encoded visual embeddings or attention, only after the visual encoder has processed the entire observation. We propose ActPrune, a training-free token pruning method that moves the pruning decision ahead of the vision encoder by grounding it in a geometric prior derived from the robot's own action. The action chunk predicted at the previous control step is already available before the current observation is encoded, and it specifies the end-effector and camera motion between consecutive observations. The same action affects the two standard camera views asymmetrically: in the fixed third-person view, ActPrune retains patches along the projected end-effector trajectory together with semantic anchors, while in the wrist view, it prunes patches that remain unchanged after compensating for the action-induced camera motion. Extensive experiments on two simulation environments (LIBERO and VLABench) and four real-robot tasks demonstrate that ActPrune achieves up to 1.61× speedup and 59.5% FLOPs reduction with negligible degradation in success rates. The project site is http://mmai.ewha.ac.kr/actprune/.
Figures & tables
Fig. 1: Overview of ActPrune . (a) By grounding token pruning in executed actions, (b) ActPrune achieves the largest FLOPs reduction and highest speedup at 25% token retention in a training-free manner, while preserving competitive task performance.
Fig. 2: The overall framework of ActPrune . The previous action chunk At−1 derives view-specific pruning signals before visual encoding. Pruned patches are compressed into context tokens, and encoder attention applies the remaining budget.
Method
Spatial
Object
Goal
Long
Avg.(%) ↑
FLOPs(T) ↓
Latency(ms) ↓
Speedup( × ) ↑
OpenVLA-OFT (100%)
99.0
97.0
97.4
92.6
96.5
4.739
86.173
1.00 ×
Retained 50% (50% ↓ )
+FastV
98.6
97.6
97.2
91.2
96.2
3.123
66.19
1.30 ×
+SparseVLM
98.0
96.2
97.2
80.8
93.1
3.388
71.89
1.20 ×
+DivPrune
97.8
98.2
97.2
92.8
96.5
3.015
72.33
1.19 ×
+VLA-Cache
97.0
96.2
97.1
90.6
95.2
3.509
78.72
1.09 ×
TABLE I: Performance comparison with OpenVLA-OFT on LIBERO at 50%, 25%, and 12.5% visual token retention. The best is bold, and the second is underlined.
Fig. 3: Real-world experimental setup. (a) A 7-DoF Franka Research 3 with UMI-style fingertips, observed by an exterior third-person camera and a wrist-mounted camera. (b) Four manipulation tasks.
Method
Spatial
Object
Goal
Long
Avg.(%) ↑
FLOPs(T) ↓
Latency(ms) ↓
Speedup( × ) ↑
π0.5 (100%)
98.0
98.0
99.4
90.8
96.6
4.975
178.3
1.00 ×
Retained 50% (50% ↓ )
+FastV
97.4
98.6
96.2
89.6
95.5
3.952
178.1
1.00 ×
+SparseVLM
98.4
98.2
96.2
91.2
96.0
4.143
177.1
1.01 ×
+DivPrune
98.2
96.6
96.8
88.8
95.1
3.825
180.1
0.99 ×
+VLA-Cache
98.0
97.0
95.2
90.2
95.1
4.091
181.4
0.98 ×
TABLE II: Performance comparison with π0.5 on LIBERO at 50%, 25%, and 12.5% visual token retention.
Fig. 4: Qualitative results of retained (color) and merged/pruned (gray) tokens on the four LIBERO tasks at 25% retained ratio. The leftmost column shows the observation, and the subsequent columns visualize the pruning results over the episode, with the third-person view on the top row and the wrist view on the bottom row.
Method
Con.
Flower
Book
Chem.
Drink
Fruit
Mahj.
Paint.
Poker
Toy
Avg.(%)
FLOPs(T) ↓
π0.5 (100%)
60.0
20.0
51.0
57.8
33.3
40.0
60.0
20.0
60.0
32.0
42.0
4.975
Retained 50% (50% ↓ )
+FastV
56.0
6.0
27.1
36.2
28.0
30.0
8.5
23.8
34.0
6.0
25.6
3.447
+SparseVLM
28.0
2.0
12.5
18.6
22.2
25.0
9.1
33.3
6.0
5.0
16.2
3.542
+DivPrune
52.0
6.0
24.8
32.8
26.8
40.0
23.4
28.6
30.0
16.0
28.0
3.257
+VLA-Cache
55.0
0.0
34.4
30.0
22.2
30.0
33.3
20.0
30.0
10.0
26.5
3.674
TABLE III: Performance comparison with π0.5 on VLABench at 50%, 25%, and 12.5% visual token retention. Task names are abbreviated for brevity (e.g., ‘Con.’ for ‘add condiment’, ‘Chem.’ for ‘select chemistry tube’).
Third-person View
Wrist View
Acc.(%) ↑
FLOPs(T) ↓
Patch
Emb.
Patch
Emb.
✓
✗
✗
✗
93.5
3.990
✓
✓
✗
✗
94.8
4.044
✗
✗
✓
✗
94.9
4.100
✗
✗
✓
✓
96.0
4.058
✓
✓
✓
✓
91.8
3.063
TABLE IV: Analysis of design choices for ActPrune on LIBERO with π0.5 are reported. The retained ratio is 25%.
TABLE V: ActPrune ablation experiments with π0.5 on LIBERO at retained ratio 25%.
TABLE VI: ActPrune ablation experiments with OpenVLA-OFT on LIBERO at retained ratio 25%.
Fig. 5: The effectiveness of patch-level retained ratio γpatch with π0.5 on LIBERO at full retained ratio 25% .
Method
Success (%) ↑
FLOPs (T) ↓
Latency (ms) ↓
Tool
Doll
Towel
Banana
π0.5 (100%)
70
60
60
50
5.021
174.8
Retained 50% (50% ↓ )
+FastV
60
50
50
30
3.996
167.6
+SparseVLM
60
50
40
40
4.187
169.3
+DivPrune
60
40
40
20
3.869
172.3
TABLE VII: Performance of our method on π0.5 for real-robot tasks at 50%, 25%, and 12.5% visual token retention.