Organizations: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China · University of Macau, Macau, China · School of Data Science and Engineering, Xingzhi College, South China Normal University, Guangzhou, China · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, China · School of Intelligence Science and Technology, University of Science and Technology Beijing, Beijing, China · College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China
Efficient visual representation is a central image-processing challenge in embodied manipulation, where policies repeatedly process dense visual-token sequences during closed-loop control. Existing methods rank or prune tokens using semantic relevance, VLM attention, cross-frame redundancy, or motion in the action space. These signals may discard task-relevant regions when instruction-related appearance and observed image motion are not yet spatially aligned. We introduce Interaction-Aligned Pruning (IAprune), a training-free method that treats per-frame budget setting and within-budget token selection as two linked decisions. Semantic--motion spatial agreement guides the choice between Conservative and Aggressive coverage, and the resulting region size is mapped to a calibrated dynamic budget. Continuous semantic and motion responses rank the existing tokens, while geometric residual correction redirects fixed selection slots toward under-represented boundaries without increasing the sequence length. Across four embodied manipulation policies, three simulation benchmarks, and a real-robot platform, IAprune provides a favorable accuracy--efficiency trade-off, matching the unpruned policy on LIBERO with a 1.54× speedup and reaching 1.48× acceleration on a real robot. Phase-wise analysis shows that retention gains are largest under tight budgets early in an episode, while fixed-budget analysis confirms that geometry replaces low-priority tokens with boundary and contact-region evidence rather than retaining more tokens. Our project website is: IAprune.com.
Figures & tables
Fig. 1: Comparison of Perception-First vs. Interaction-First token pruning paradigms. Perception-First baselines (Top) prematurely lose the manipulation target due to early semantic misalignment—a vulnerability that simple temporal stacking fails to resolve without explicit interaction modeling. In contrast, IAprune (Bottom) combines IoU-aware conservative-to-aggressive scheduling with budget-preserving structural reallocation, preserving the physical target while reducing redundant visual computation.
Fig. 2: Overview of IAprune. Semantic and motion evidence serve two complementary roles: their spatial agreement selects a Conservative or Aggressive coverage mode, while their continuous responses define the base relevance of each visual token. The selected mode defines a provisional region, whose size is converted into a dynamic per-frame budget Kt under a target average retention ratio rtar . Geometric evidence then performs budget-preserving structural reallocation through residual correction without changing Kt . The resulting compact visual sequence is fed into the unchanged embodied manipulation policy for action generation.
Fig. 3: Overview of the evaluation benchmarks and tasks. The simulation evaluation includes LIBERO, VLABench, and CALVIN ABC-D. The real-world evaluation covers simple, long-horizon, and dual-arm manipulation.
Fig. 4: Physical platform for real-world evaluation. The head-mounted primary camera and wrist cameras provide complementary observations to the embodied manipulation policy, while the external RealSense camera is used only to record task execution. The shared workspace supports both single-arm manipulation and coordinated dual-arm operation.
ZED 2i binocular camera mounted on the head; RGB stream from the left lens only
Wrist View
Wide-angle RGB wrist camera per arm; single-arm uses head + active wrist; dual-arm uses head + both wrists
Data Spec.
All observations standardized to 640 × 480; Intel RealSense D455 used only for external recording
OS / Hardware
Ubuntu 22.04 workstation with a single NVIDIA A100 GPU
Backbone
π0.5
Inference
Action chunk size 50; first 20 actions of each predicted chunk are executed
TABLE I: Real-Robot Deployment Details. Hardware and inference configuration used for physical experiments.
Method
DreamVLA
OpenPi (π0,π0.5)
LIBERO Success (%)
CALVIN Seq. Length ↑
LIBERO Success (%)
VLABench Success (%)
Spa.
Obj.
Goal
Long
1
2
3
4
5
Avg.
Spa.
Obj.
Goal
Long
Con.
Paint.
Poker.
Avg.
Vanilla Baseline: 100% Token Retention
Vanilla
84.5
91.5
89.5
89.5
97.8
94.5
89.5
83.8
77.5
4.43
94.2
98.2
94.6
82.8
56.0
30.0
52.0
46.0
Retention: 70%
FastV
86.5
86.5
84.0
87.5
97.6
94.5
89.0
82.8
76.8
4.40
93.5
96.1
92.2
80.5
44.0
28.0
32.0
34.6
TABLE II: Comprehensive Performance Comparison. Success rates are reported on LIBERO and VLABench, and average sequence length is reported on CALVIN at nominal retention labels of 30%, 50%, and 70%. For IAprune, these labels denote target average retention ratios. Baseline labels follow their original budget definitions and may not correspond to identical realized token counts.
Method
CKPT
Success Rate (%)
Speedup
Spa.
Obj.
Goal
Long
Avg.
Embodied Manipulation Policies
OpenVLA [ 2 ]
OpenVLA (7B)
84.7
88.4
79.2
53.7
76.5
-
WorldVLA [ 4 ]
Cham. (7B)
87.6
96.2
83.4
60.0
81.8
-
NORA [ 42 ]
Qwen-VL (3B)
85.6
87.8
77.0
45.0
73.9
-
SmolVLA [ 43 ]
SmolVLM (2.2B)
93.0
94.0
91.0
77.0
88.8
-
TABLE III: LIBERO results for embodied manipulation policies and visual-token pruning methods. Representative policies are included for context. All pruning methods use the OFT 7B backbone, and speedup is reported relative to the unpruned OpenVLA-OFT policy.
Ret.
Method
Early segment
Late segment
TTR ↑
BTR ↑
CZR ↑
TTR ↑
BTR ↑
CZR ↑
30%
SparseVLM
0.016
0.049
0.100
0.077
0.083
0.125
VLA-Cache
0.267
0.271
0.312
0.304
0.395
0.356
DivPrune
0.374
0.311
0.313
0.444
0.415
0.400
IAprune
0.733
0.672
0.710
0.543
0.507
0.565
50%
SparseVLM
0.207
0.334
0.571
0.273
0.391
0.545
TABLE IV: Segmentation-grounded token retention on VLABench. TTR, BTR, and CZR measure the retained fractions of target-object, target-boundary, and contact-zone tokens. The early segment is the first 30% of each episode, and the late segment is the remaining 70%.
Fig. 5: Visualization of interaction-aligned pruning on LIBERO. Bottom: visual-token retention changes from Conservative to Aggressive coverage. Middle: the alignment score (IoU) selects the coverage mode. Top: the overlap (purple) between the semantic (red) and observed-motion (blue) masks determines the alignment score.
Fig. 6: Effect of budget-preserving structural reallocation. Under the same per-frame Top- Kt budget, geometry-guided residual correction retains more object-boundary tokens near potential contact regions.
Token type
Available
Semantic only
+Sobel
Avg. change
Object body
22
21.3 (97%)
22.0 (100%)
+0.7
Semantic drift
18
14.5 (81%)
0 6.8 (38%)
−7.7
Object edge
26
0 7.9 (30%)
24.1 (93%)
+16.2
Noise edge
38
0 2.6 (7%)
0 6.4 (17%)
+3.8
Other/background
152
30.7 (20%)
17.7 (12%)
−13.0
Top- K total
256 (100%)
77.0 (keep30%)
77.0 (keep30%)
–
TABLE V: Budget-preserving structural reallocation. Geometry-guided residual correction reallocates a fixed budget toward object edges near potential contact regions. Both variants use Kt=77 .
Budget
Sem.
+Mot.
+Edge
+M+E
Full IAP
w/o Edge
OpenVLA-OFT
30%
91.0
90.7 (-0.3)
91.8 (+0.8)
91.4 (+0.4)
97.1 (+6.1)
96.1 (+5.1)
50%
94.0
94.0 (+0.0)
94.8 (+0.8)
94.2 (+0.2)
97.5 (+3.5)
96.8 (+2.8)
70%
95.3
95.5 (+0.2)
95.9 (+0.6)
96.0 (+0.7)
97.8 (+2.5)
97.2 (+1.9)
DreamVLA
30%
80.8
80.2 (-0.6)
82.0 (+1.2)
81.5 (+0.7)
88.8 (+8.0)
87.2 (+6.4)
TABLE VI: Mechanism ablation across backbones. Average success rate under matched average retention. Sem. is the semantic-only variant; +Mot. and +Edge add one cue at a time; +M+E combines both cues without coverage-mode switching; and w/o Edge. removes structural reallocation from full IAprune. Parentheses report percentage-point changes from Sem.
Fig. 7: Qualitative visual comparison under an extreme target average retention of 12.5%. Across four manipulation cases, IAprune preserves more continuous target- and manipulator-adjacent regions, while FastV and VLA-Cache tend to miss the target or keep fragmented redundant patches.
Average-Budget Tracking
Target
Observed
Dev. (pp)
Ret. Range
Lat. Range (ms)
30%
30.9%
+0.9
29.5–31.9%
78.2–82.6
50%
49.7%
−0.3
48.9–50.0%
89.2–91.7
70%
67.7%
−2.3
67.6–67.9%
98.5–99.9
TABLE VII: Budget tracking and threshold sensitivity on OpenVLA-OFT/LIBERO. The upper table reports observed average retention together with retention and latency ranges across the evaluated IoU thresholds. The lower table reports latency in milliseconds and success rate in percent as θiou varies.
Method
70% retention
50% retention
30% retention
Mem.
Time
Mem.
Time
Mem.
Time
Memory in GB; CUDA time in ms; lower is better.
DreamVLA on LIBERO
Vanilla (100%)
Unpruned: 2.661 GB memory / 101.47 ms CUDA time
FastV
2.626
84.55
2.615
76.87
2.608
72.96
SparseVLM
2.621
83.85
2.612
75.16
2.602
73.76
TABLE VIII: Memory and runtime on LIBERO. Peak GPU memory and per-step CUDA runtime at nominal retention labels of 70%, 50%, and 30%. For IAprune, these labels denote the target average retention ratio rtar ; baseline labels follow their original budget definitions.
Fig. 8: Latency breakdown of IAprune. Visual-token processing in the multimodal backbone is the main component that decreases with token retention. In the measured setting, vision encoding, text-token processing, and action decoding remain nearly constant.
Fig. 9: Real-World Task Demonstrations. From top to bottom: simple (single-step), long-horizon (multi-step), and dual-arm (coordinated) manipulation tasks, showing head and wrist camera perspectives.
Method
Success Rate (%)
Latency (ms) ↓
Simple
Long
Dual-Arm
Avg.
Single
Dual
Vanilla (100%)
80.0
64.0
44.0
62.7
88.1
124.3
FastV
60.0
44.0
20.0
41.3
61.6
88.2
VLA-ADP
72.0
52.0
32.0
52.0
65.3
94.2
IAprune
84.0
68.0
44.0
65.3
59.7 (1.48 × )
84.6 (1.47 × )
TABLE IX: Real-robot results on three tasks. We report success rate and inference latency for the Simple, Long, and Dual-arm tasks. Vanilla denotes the unpruned π0.5 policy, and values in parentheses are speedups relative to Vanilla.
Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action-level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full-context teacher, with ground-truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real-world manipulation tasks show that VLA-ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance-efficiency trade-off than existing frozen-VLA pruning methods and demonstrate the value of action-level supervision for visual token selection. Code is available at https://github.com/du-owen/VLA-ACL.
Existing VLA pruning strategies primarily select individual visual tokens according to task-level semantic relevance, while overlooking the spatial information required for robotic manipulation. To examine this limitation, we construct a simple Stride baseline that uniformly samples tokens along the flattened one-dimensional visual sequence, representing a purely geometric pruning strategy. Surprisingly, Stride outperforms semantic pruning and random pruning at certain pruning ratios, but collapses when the token budget is only slightly reduced. We characterize this phenomenon through the spatial coverage radius, defined as the largest spatial blind spot induced by the retained token set after pruning. Our analysis reveals a strong correlation between the spatial structure of retained tokens and task success, suggesting that reliable VLA pruning requires preserving not only task-relevant tokens but also the spatial scaffold of the scene. Motivated by this diagnosis, we propose GeoScaffold, a training-free visual token pruning method that partitions each image into spatial regions, allocates inter-region token budgets using task-relevance weights, and selects intra-region scaffold tokens via farthest point sampling to reduce the local coverage radius. On pi 0.5 and LIBERO, GeoScaffold retains only 20% of visual tokens while preserving a 93.2% average success rate, and achieves a 1.78 times prefill speedup over the unpruned baseline.
Jiayu Chen, Shuyong Gao, Jingkai Jia +6
Fudan University · The Hong Kong Polytechnic University
Visual token pruning is an effective way to accelerate vision-language models and is especially useful for vision-language-action (VLA) inference, where many visual tokens must be processed before predicting robot actions. Existing pruning methods usually estimate which tokens can be pruned based on attention scores or feature diversity, retaining tokens that are either highly attended or visually different from others. However, most of them use fixed pruning schedules, such as pruning once at a preset layer or pruning at uniformly spaced layers. Such schedules can be risky for VLA models, because the model may not know which visual regions matter for the action in early layers. Tokens that look unimportant at first may become useful after the model combines visual observations with the language instruction. In this work, we propose SAPrune, a training-free visual token pruning framework for efficient VLA inference. Instead of pruning at fixed layers, SAPrune uses a small calibration set to observe how action-to-visual attention changes across layers, and chooses pruning layers only after the attention pattern becomes more reliable. At each selected layer, SAPrune applies a dual-path pruning rule: one path protects strongly attended visual tokens from pruning, while the other prevents useful surrounding context from being discarded. Experiments on LIBERO, SIMPLER, and real-world robotic tasks show that SAPrune prunes 87.5% of visual tokens and achieves up to 1.718x inference speedup while maintaining competitive task success rates.