POKE: Probe Offline, Keep Efficiently for Vision-Language-Action Models
Authors: Jaehyuk Lee, Yeajin Lee, Suhyun Bae, Donghun Lee
Organizations: Department of Mathematics, Korea University, Seoul, Republic of Korea · Program in Actuarial Science and Financial Engineering, Korea University, Seoul, Republic of Korea
Visual token pruning reduces the inference cost of Vision-Language-Action (VLA) models, but attention-based scores do not directly measure a token's effect on the predicted action. We define action sensitivity as the magnitude of the first-order action change when a token is replaced by its camera's mean token and estimate it for all tokens with a single backward pass. Computing it online is costly: scoring on the full (unpruned) graph needs an extra forward pass at every control step. Scoring on the pruned forward pass is cheaper but causes scoring-graph lock-in: removed tokens lose their direct paths through later layers and tend to receive lower scores, making those locations less likely to return. On replayed observations, even with a random conditioning mask, pruned-graph scoring re-selects 59% of the mask's tokens, versus 6% expected by chance; attention histories show a similar bias. POKE (Probe Offline, Keep Efficiently) computes full-graph action sensitivities offline and distills them into a student with under 0.3M parameters. At inference, the student scores the current observation without backpropagation or previous masks, and score-guided diversity selection retains the tokens. Keeping 32 of 512 visual tokens on LIBERO, POKE achieves 95.2% success on OpenVLA-OFT and 90.8% on π0.5 at 1.85× and 1.26× speedups, outperforming attention-, diversity-, and distillation-based pruners, with larger margins on LIBERO-Plus.
Figures & tables
Figure 1: Motivation on OpenVLA-OFT. (a) LIBERO-Goal, 64/512 tokens; rows: third-person/wrist; clear patches: retained. Attention misses the target plate kept by action sensitivity. (b) LIBERO-Long, B=32 , 9 closed-loop episodes (mean, IQR). Left: cumulative number of distinct tokens ever retained; pruned-graph scoring keeps returning to the same tokens. Right: share of score assigned to tokens outside the previous mask; 19% on the pruned graph vs. 77% on the full graph (dotted: uniform scores).
Figure 2: Method overview. (a) Offline training: a frozen full-graph VLA produces single-probe action-sensitivity labels, which are distilled into a lightweight student. (b) Inference: student predicts token scores, and MMDP selects diverse tokens from scoring candidates.
B=128
B=64
B=32
Method
Spa.
Obj.
Goal
Long
Avg.( ↑ )
× ( ↑ )
Spa.
Obj.
Goal
Long
Avg.( ↑ )
× ( ↑ )
Spa.
Obj.
Goal
Long
Avg.( ↑ )
× ( ↑ )
OpenVLA-OFT
Vanilla
98.6
97.2
97.6
93.4
96.70
1.00
98.6
97.2
97.6
93.4
96.70
1.00
98.6
97.2
97.6
93.4
96.70
1.00
Random
95.6
94.6
95.0
89.8
93.75
1.43
72.4
80.8
52.4
33.8
59.85
1.78
43.8
15.6
8.8
10.0
19.55
1.89
FastV
97.4
97.2
95.6
91.8
95.50
1.40
95.8
94.6
93.6
84.2
92.05
1.73
91.6
80.8
92.6
43.2
77.05
1.88
DivPrune
98.0
96.4
93.6
90.2
94.55
1.37
95.8
93.2
88.8
72.4
87.55
1.67
76.4
90.4
80.8
40.8
72.10
1.84
VLA-Cache
97.0
98.0
97.4
90.8
95.80
1.28
96.0
92.4
94.2
80.2
90.70
1.44
90.4
88.8
90.4
70.4
85.00
1.47
Table 1: LIBERO success rates (%) and speedups over Vanilla ( × ) with B retained visual tokens. † : our adaptation. Gray: online diagnostic probes (§ 4.5 ), excluded from bold along with Vanilla; on π0.5 their backward passes run in eager mode, so their speedups are against eager Vanilla (Table 26 ). POKE uses its B=32 checkpoints at all budgets; timing details in Appendix 13 .
Method
Spatial
Object
Goal
Long
Avg.( ↑ )
OpenVLA
Vanilla
84.6
84.0
74.0
51.4
73.50
FastV
61.2
74.2
58.8
36.4
57.65
VLA-Cache
67.4
71.0
60.2
34.2
58.20
VLA-Pruner
75.6
76.2
61.4
42.0
63.80
POKE (top- k )
72.8
81.2
70.0
44.2
67.05
POKE (MMDP)
78.8
82.2
72.4
53.6
71.75
Table 2: Additional policies and benchmark at B=32 of 256 visual tokens (success, %). Top: autoregressive OpenVLA on LIBERO. Bottom: open-pi-zero π0 (Bridge checkpoint) on SimplerEnv WidowX visual matching, 240 episodes per task; its student is trained on real BridgeData V2 frames. Details and full results appear in Appendices 10 and 11 .
Figure 3: Qualitative rollouts on OpenVLA-OFT Long task 6 ( B=32 ) from the same initial state, showing frames and the applied third-person mask. VLA-Pruner times out at step 520 (red); POKE (top- k ) succeeds at step 220 (green). Figure 4 adds more results.
Method
Camera
Robot
Lang.
Light
Backgr.
Noise
Layout
Total
OpenVLA-OFT
Vanilla (%)
66.4
41.0
80.7
91.9
89.6
74.4
83.7
74.4
FastV
0.56
0.51
0.74
0.63
0.73
0.58
0.66
0.64
DivPrune
0.46
0.35
0.54
0.47
0.56
0.38
0.54
0.48
VLA-Pruner
0.66
0.69
0.89
0.86
0.83
0.78
0.81
0.79
MAP †
0.72
0.87
0.85
0.89
0.95
0.69
0.85
0.82
POKE (top- k )
0.82
0.89
0.97
0.98
0.96
0.93
0.92
0.92
Table 3: LIBERO-Plus retention by perturbation category at B=32 on a stratified 25% subset (2,508 perturbed tasks, one episode each), pooled over the four suites. Vanilla: success (%); other rows: success relative to Vanilla. Policies and students are unchanged (trained on unperturbed demonstrations). Per-suite success appears in Appendix 12 .
OpenVLA-OFT
π0.5
Suite
Teacher
Top- k
MMDP
Top- k
MMDP
Object
Attention
93.4
96.0
82.8 ± 1.62
94.4 ± 2.02
Action sens.
96.8
97.6
86.0 ± 1.44
96.4 ± 2.84
Long
Attention
80.40 ± 2.38
83.20 ± 1.28
62.8 ± 0.48
71.8 ± 2.46
Action sens.
83.93 ± 0.81
87.53 ± 1.86
68.8 ± 0.98
76.4 ± 1.12
Table 4: Teacher supervision at B=32 (success, %). Only the teacher labels differ (Appendix 8 ). ± : mean ± SD over three training seeds, separate from the Table 1 checkpoint; other entries use the Table 1 checkpoint.
Success rate
Overlap
Method
Current-frame signal
pruned
full
pruned
full
VP: action attention only
–
3.4
69.4
0.995
0.664
VP: combined
Semantic attention
78.6
82.4
0.515
0.364
SpecPrune † : history only
–
4.8
84.4
1.000
0.431
SpecPrune † : combined
Local attention + frame difference
72.6
80.6
0.545
0.343
Action-sensitivity probe (top- k )
–
59.6
81.8
0.85
0.41
Table 5: History-graph interventions (OFT Long, B=32 , single runs). Pruned/full: graph used to compute history or probe scores; current-frame signals are computed before the cut; overlap: Jaccard. VP denotes VLA-Pruner; SpecPrune † uses its fixed-budget static stage (Appendix 9.5 ).
Long
Goal
Teacher
GPU-h
Top- k
MMDP
GPU-h
Top- k
MMDP
P=1
0.7
68.4
76.4
0.3
77.0
93.6
P=3
2.0
70.2
80.0
0.9
86.8
94.0
P=5
3.3
72.4
80.4
1.1
85.2
93.4
P=8
5.4
75.4
83.0
2.3
92.8
94.2
Online probe (full, top- k )
–
79.4
–
–
93.2
–
Table 6: Teacher probe count on π0.5 Long and Goal ( B=32 ; success %, 500 episodes). One checkpoint per P ; P=1 is from Table 1 . Label costs are GPU-hours; Long P=3,5 are scaled from measured P=8 . Inference cost is unchanged.
Spatial
Object
Goal
Long
Task
Top- k
MMDP
Top- k
MMDP
Top- k
MMDP
Top- k
MMDP
0
88
100
92
94
92
92
66
72
1
86
98
74
100
88
98
66
88
2
100
100
86
100
94
98
74
84
3
92
100
88
94
50
84
84
90
4
60
88
88
90
96
98
60
58
Table 11: Task-level POKE success rates (%) on π0.5 at B=32 . Each task is evaluated over 50 episodes. Top- k and MMDP denote the two POKE selection rules. Entries report single-checkpoint results, rather than averages over training seeds; task IDs are suite-specific.
Figure 4: All methods on OpenVLA-OFT Long task 6 ( B=32 ), from the same initial state as Figure 3 . Layout and borders follow Figure 3 .
Method
Carrot
Eggplant
Spoon
Stack
Pooled
FastV
19/24 (0.54)
9/40 ( 9.3×10−6 )
21/37 (0.048)
40/31 (0.34)
89/132 (0.0046)
POKE (top- k )
22/24 (0.88)
37/38 (1.00)
30/45 (0.11)
39/21 (0.027)
128/128 (1.00)
POKE (MMDP)
19/21 (0.87)
52/27 (0.0066)
35/24 (0.19)
46/19 (0.0011)
152/91 ( 1.1×10−4 )
Table 21: SimplerEnv paired comparison with VLA-Pruner ( π0 , B=32 ). Entries: episodes where only the method succeeds / only VLA-Pruner succeeds (two-sided exact McNemar p ), 240 episodes per task.
Method
Spatial
Object
Goal
Long
Avg.( ↑ )
Ret.( ↑ )
OpenVLA-OFT
Vanilla
82.7
83.8
64.0
67.8
74.6
1.00
FastV
77.0
33.2
44.4
36.0
47.7
0.64
DivPrune
53.3
27.8
42.6
19.4
35.8
0.48
VLA-Pruner
71.7
63.3
50.6
50.5
59.0
0.79
MAP †
71.8
63.2
59.3
50.8
61.3
0.82
POKE (top- k )
81.3
75.7
59.6
58.6
68.8
0.92
Table 22: LIBERO-Plus success (%) by suite at B=32 . Avg.: unweighted mean over suites; Ret.: Avg. relative to Vanilla. The Total column of Table 3 pools tasks across suites and can differ slightly.
Method
Dev./random ↓
Below random ↑
Tasks < VP
FastV
0.371
0.914
2/10
VLA-Pruner
0.332
0.938
–
POKE (top- k )
0.252
0.981
10/10
POKE (MMDP)
0.244
0.973
10/10
Table 36: Mask fidelity at shared observations (OFT Long, B=32 , 625 frames from Vanilla rollouts). Deviation is normalized by a random mask (lower is better); below random: fraction of frames with a smaller deviation than the random mask. VP’s temporal signal is updated along the shared trajectory. This measures mask quality on common reference states, not on each method’s own deployment states.
Vision-language-action (VLA) models perform versatile manipulation tasks from language instructions, but their inference cost limits how fast a robot can respond in closed-loop control. Visual token pruning reduces this cost; however, prior methods estimate token redundancy from encoded visual embeddings or attention, only after the visual encoder has processed the entire observation. We propose ActPrune, a training-free token pruning method that moves the pruning decision ahead of the vision encoder by grounding it in a geometric prior derived from the robot's own action. The action chunk predicted at the previous control step is already available before the current observation is encoded, and it specifies the end-effector and camera motion between consecutive observations. The same action affects the two standard camera views asymmetrically: in the fixed third-person view, ActPrune retains patches along the projected end-effector trajectory together with semantic anchors, while in the wrist view, it prunes patches that remain unchanged after compensating for the action-induced camera motion. Extensive experiments on two simulation environments (LIBERO and VLABench) and four real-robot tasks demonstrate that ActPrune achieves up to 1.61× speedup and 59.5% FLOPs reduction with negligible degradation in success rates. The project site is http://mmai.ewha.ac.kr/actprune/.
Sungwon Moon, Minji Kim, Hyemin Boo +2
Human-Centered Artificial Intelligence Research Institute, Ewha Womans University · Division of Artificial Intelligence & Software, Ewha Womans University
Pruning is a typical acceleration technique for compute-bound models by removing computation on unimportant values. Recently, it has been applied to accelerate Vision-Language-Action (VLA) model inference. However, existing acceleration methods focus on local information from the current action step and ignore the global context, leading to >20% success rate drop and limited speedup in some scenarios. In this paper, we point out spatial-temporal consistency in VLA tasks: input images in consecutive steps exhibit high similarity, and propose the key insight that token selection should combine local information with global context of the model. Based on this, we propose SpecPrune-VLA, a training-free, two-level pruning method with heuristic control. (1) Action-level static pruning. We leverage global history and local attention to statically reduce visual tokens per action. (2) Layer-level dynamic pruning. We prune tokens adaptively per layer based on layer-wise importance. (3) Lightweight action-aware controller: We classify actions as coarse- or fine-grained by the speed of the end effector and adjust pruning aggressiveness accordingly. Extensive experiments show that SpecPrune-VLA achieves up to 1.57× speedup in LIBERO simulation and 1.70× on real-world tasks, with negligible success rate degradation.
Hanzhen Wang, Jiaming Xu, Yushun Xiang +4
Shanghai Jiao Tong University · SII · Infinigence-AI
Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action-level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full-context teacher, with ground-truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real-world manipulation tasks show that VLA-ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance-efficiency trade-off than existing frozen-VLA pruning methods and demonstrate the value of action-level supervision for visual token selection. Code is available at https://github.com/du-owen/VLA-ACL.