Organizations: School of Computer Science and Technology, Tongji University · Agency for Science, Technology and Research (A*STAR), Singapore · School of Computer Science, University of Leeds · Electronic and Electrical Engineering, Brunel University of London
Vision-language-action (VLA) policies have demonstrated strong generalization in robotic manipulation, yet processing dense visual tokens often incurs high inference latency. Visual token pruning offers a practical way to reduce this cost, but identifying a compact set of tokens that preserves reliable action generation remains challenging. Existing methods largely rely on static or immediate heuristic cues to estimate token importance. We introduce APS-VLA, a framework that learns visual token selection from robot demonstrations while keeping the pretrained VLA backbone frozen. A lightweight Decision Influence Estimator scores tokens conditioned on current visual observations, language instructions, and robot proprioception. Its training combines two complementary signals: action supervision matches both the full-input policy predictions and demonstrated actions, while an auxiliary visual transition objective uses demonstrated actions to predict subsequent visual features from the selected representation. At deployment, the estimator performs a single hard selection and packing step before the policy's native action-generation procedure; the auxiliary predictor is removed. Experiments with OpenVLA-OFT on LIBERO and with π0.5 and XR-1 on RoboCasa365 demonstrate inference acceleration with modest changes in task success. Across five real-world tasks on two dual-arm platforms, APS-VLA reduces mean policy inference latency from 137.25 to 93.25 ms, achieving a 1.47× speedup with an average success rate of 61%, compared with 60% for the unpruned baseline. Together, these results support APS-VLA as a modular, plug-in framework for efficient robotic control across VLA architectures, preserving the underlying policy parameters and native action-generation procedure.
Figures & tables
Figure 1: Why visual token selection needs both action and future visual cues. Placing a cube in a bowl requires both precise gripper–cube alignment and accurate target release. Future visual supervision alone (top) misses fine-grained contact cues and fails to grasp, whereas action supervision alone (middle) secures the grasp but loses destination context, releasing outside the bowl. Joint supervision (bottom) retains both cues for successful placement .
Figure 2: Overview of APS-VLA. A shared visual selector is trained with a weighted combination of action supervision and future visual supervision while the VLA backbone remains frozen. Deployment uses one hard selection and packing step before native action generation; the auxiliary predictor is removed.
Figure 3: Decision Influence Estimator (DIE). Visual context, the instruction, and robot state are fused and added to each visual token feature. A shared scorer then produces one score per token. Robot state includes proprioceptive inputs such as end-effector pose and gripper state.
Table 1: Comparison with the baseline and representative VLA methods on LIBERO. Success rates are reported in percentages. All methods, including APS-VLA and the unpruned OpenVLA-OFT baseline, are reproduced on NVIDIA A800 GPUs under the same evaluation conditions.
π0.5
XR-1
Method
A-S
C-S
C-U
Latency
Speedup
A-S
C-S
C-U
Latency
Speedup
Unpruned
39.60
7.10
1.20
70.29–70.30
1.00×
80.23
57.10
32.10
217.25
1.00×
FastV [ 5 ]
36.47
5.67
0.30
56.24
1.25×
78.41
52.67
25.67
159.74
1.36×
SAFE-Pruner [ 19 ]
37.50
6.33
0.87
55.35
1.27×
78.10
49.23
25.92
164.58
1.32×
VLA-ACL [ 7 ]
38.97
6.33
0.77
52.46
1.34×
79.93
52.10
29.33
149.83
1.45×
APS-VLA
39.15
6.71
1.21
51.50
1.37×
79.77
55.67
30.97
142.93
1.52×
Table 2: Visual token pruning on RoboCasa365. We report success rates (%) on seen atomic (A-S), seen composite (C-S), and unseen composite (C-U) tasks, policy inference latency (ms), and speedup relative to each unpruned policy. APS-VLA averages success rates and speedups over the 256- and 128-token budgets. All methods are reproduced on NVIDIA A800 GPUs under the same evaluation conditions.
Tokens
Pruning
A
B
256
50%
95.17
96.58
128
75%
94.58
95.75
64
87.5%
92.75
93.92
Table 3: Mean development success (%) across four LIBERO suites at different visual token budgets.
Figure 5: Representative keyframes from five real-world manipulation tasks. Frames within each task are ordered chronologically, highlighting object interactions and transitions between task stages.
Success rate (%) ↑
Inference latency (ms) ↓
Task
π0.5
APS-VLA
π0.5
APS-VLA
Speedup ↑
Close Drawer
80.00
82.50
131.25
91.25
1.44×
Close Microwave
75.00
72.50
134.50
92.50
1.45×
Red Cube into Bowl
72.50
72.50
132.75
90.75
1.46×
Fold Clothes
37.50
37.50
142.50
95.25
1.50×
Clear Toys and Wipe Table
35.00
40.00
145.25
96.50
1.51×
Table 6: Real-world task success rates (%) and policy inference latency (ms). Each method is evaluated over 40 trials per task. Speedup is relative to the corresponding unpruned π0.5 baseline.
Updates
Aligned L1↓
Shuffled L1↓
Aligned Succ. (%) ↑
Shuffled Succ. (%) ↑
500
0.016451
0.022377
91.33
87.00
1,000
0.015625
0.021379
93.67
88.33
1,500
0.013377
0.020357
94.00
89.00
2,000
0.011973
0.019251
94.33
89.33
Table 4: Temporal-target control on LIBERO-Long ( K=128 ): teacher-action L1 (task-weighted, averaged over three seeds) and closed-loop success rate.
λ
Seed 17
Seed 23
Seed 41
Mean ± SD
0.03
91
94
91
92.00±1.73
0.1
94
92
95
93.67±1.53
0.3
92
91
94
92.33±1.53
Table 5: Auxiliary-weight sensitivity on LIBERO-Long ( K=128 , h=8 ). Development success (%) on 100 shared initial states per model; the final column is mean ± sample standard deviation over training seeds, in percentage points.
Human-Centered Artificial Intelligence Research Institute, Ewha Womans University · Division of Artificial Intelligence & Software, Ewha Womans University