Organizations: Institute of Software, Chinese Academy of Sciences, Beijing, China. · University of Chinese Academy of Sciences, Beijing, China. · LimX Dynamic, Shenzhen, China. · Nanjing University, Nanjing, China. · Nanyang Technological University, Singapore. · Zhejiang University, Zhejiang, China.
Vision-Language-Action (VLA) models have achieved significant breakthroughs by leveraging Large Vision Language Models (VLMs) to jointly interpret instructions and visual inputs. However, the substantial increase in visual tokens, particularly from multi-view inputs, poses serious challenges to real-time robotic manipulation. Existing acceleration techniques for VLMs, such as token pruning, often result in degraded performance when directly applied to VLA models, as they overlook the relationships between different views and fail to account for the dynamic and task-specific characteristics of robotic operation. To address this, we propose BFA++, a dynamic token pruning framework designed specifically for VLA models. BFA++ introduces a hierarchical pruning strategy guided by two-level importance predictors: an intra-view predictor highlights task-relevant regions within each image to suppress spatial noise, while an inter-view predictor identifies critical camera views throughout different manipulation phases to reduce cross-view redundancy. This design enables efficient token selection while preserving essential visual cues, resulting in improved computational efficiency and higher manipulation success rates. Evaluations on the RoboTwin benchmark and real-world robotic tasks demonstrate that BFA++ consistently outperforms existing methods. BFA++ improves the success rate by about 10% on both the π0 and RDT models, achieving speedup of 1.8X and 1.5X, respectively. Our results highlight that context-sensitive and task-aware token pruning serves as a more effective strategy than full visual processing, enabling faster inference and improved manipulation accuracy in real-world robotic systems.
Figures & tables
Fig. 1 : The comparison between BFA++ and other methods. Compared to existing token pruning methods, our approach not only improves inference speed but also significantly increases the success rate.
Fig. 2 : The pipeline of BFA++. The left figure shows our plug-and-play module for token pruning, and the right figure further details it. After obtaining the offline annotated inter-view importance Sintergt and intra-view importance Sintragt , we train the two importance predictor, and predict these two importance scores ( Sinter , Sintra ). Based on the predicted intra-view importance, we perform local pruning on the tokens. Then, based on both inter-view importance and intra-view importance, we conduct global pruning to obtain the final pruned tokens.
Fig. 3 : Analyzing the importance of different viewpoints. We remove one or two view images to see how the performance of a trained π0 model is affected. This experiment demonstrates that the importance of different viewpoints varies across different stages of operation.
Fig. 4 : The annotation system of BFA++. For inter-view importance, we provide three optional methods (VLM, manual annotation, bounding boxes to detect overlaps). For intra-view importance, we use the Grounding-SAM method to identify task-related regions. Details of the VLM annotation method can be found in BFA [ 15 ] .
Fig. 5 : The local and global token prune in π0 +BFA++.
Fig. 6 : Task description of real world experiments.
Model
Hammer Beat
Block Handover
Bottles Pick
Blocks Stack
Mug Hang
Bottle Adjust
Shoe Place
Cup Place
Mean
FPS
π0 [ 9 ]
0.78
0.52
0.65
0.54
0.07
0.46
0.59
0.36
0.496
6.5 Hz
π0 +DART [ 20 ]
0.70
0.53
0.64
0.41
0.08
0.50
0.49
0.30
0.456
9.8 Hz
π0 +BFA [ 15 ]
0.80
0.62
0.59
0.60
0.07
0.52
0.65
0.34
0.524
11.1 Hz
π0 +BFA++
0.84
0.66
0.70
0.60
0.15
0.58
0.65
0.48
0.583
10.3 Hz
RDT [ 14 ]
0.83
0.78
0.48
0.15
0.05
0.48
0.61
0.38
0.470
1.0 Hz
RDT+DART [ 20 ]
0.71
0.70
0.58
0.07
0.03
0.48
0.41
0.24
0.403
1.6 Hz
TABLE I : Comparison of baseline and BFA++ in RoboTwin. Our method surpasses nearly all baseline methods across various metrics.
Method
Bottle Handover
Grasp Chicken
Slide Bag
Insert Cable
Banana Hard
Mean
FPS
π0 [ 9 ]
0.30
0.45
0.15
0.10
0.35
0.27
5.8 Hz
π0 +DART [ 20 ]
0.20
0.50
0.05
0.00
0.20
0.19
8.5 Hz
π0 +BFA [ 15 ]
0.45
0.65
0.20
0.20
0.40
0.38
10.1 Hz
π0 +BFA++
0.60
0.65
0.30
0.30
0.55
0.48
9.4 Hz
RDT [ 14 ]
0.35
0.50
0.05
0.05
0.20
0.23
1.2 Hz
RDT+DART [ 20 ]
0.25
0.35
0.00
0.00
0.20
0.16
1.8 Hz
TABLE II : Comparison in real-world environments with success rate and inference speed. Our method demonstrates substantial performance improvements over baseline approaches.
Method
Adjust Bottle
Beat Hammer
Handover Block
Open Laptop
Mean
FPS
π0 [ 9 ]
0.56
0.21
0.08
0.46
0.328
6.4 Hz
π0 +DART [ 20 ]
0.40
0.08
0.05
0.30
0.208
10.0 Hz
π0 +BFA [ 15 ]
0.59
0.24
0.18
0.46
0.368
11.3 Hz
π0 +BFA++
0.63
0.25
0.19
0.55
0.405
10.5 Hz
TABLE III : Comparison in OOD environments with success rate and inference speed in RoboTwin2. Our method shows impressive improvement.
Fig. 7 : T-SNE of token with baseline and our BFA++. Tokens from the left view are omitted as they are entirely pruned by BFA++, and BFA++ shows less similar token distributions across views.
Fig. 8 : Grad-Cam heatmaps with baseline and our BFA++. Our method exhibits significantly more concentrated attention on the gripper and interactive objects.
Fig. 9 : The visualization of the final pruned tokens with BFA++, predicted intra-view importance, and inter-view importance. In figures (a) and (b), each image is composed of three views (left, head, right wrist). Figure (a) represents the token importance predicted by Intra-IP, while figure (b) shows the final token retention visualization. Figure (c) illustrates how the inter-view importance predicted by Inter-IP changes throughout the manipulation process shown in the bottom portion.
Method
Block Hammer
Bottle Adjust
Shoe Place
Cup Place
Mean
FPS
w/o Inter-IP
0.76
0.41
0.56
0.50
0.558
10.8 Hz
w/o Intra-IP
0.74
0.51
0.56
0.35
0.540
11.0 Hz
w/o Hierarchical
0.73
0.49
0.58
0.46
0.565
10.4 Hz
w/o Adaptive Weight
0.84
0.53
0.60
0.46
0.608
10.4 Hz
Adaptive Pooling
0.75
0.48
0.55
0.46
0.560
11.9 Hz
Random Drop
0.70
0.51
0.52
0.41
0.535
11.5 Hz
TABLE IV : Ablation on components and method of BFA++ with π0 . The results show that all our components are effective and the designed BFA++ pruning structure is superior to alternative methods.
Fig. 10 : The ablation of the final prune ratio of BFA++. As the prune ratio increases, the control frequency accelerates, and the success rate first increases and then decreases. Our choice of prune ratio is balanced, as it not only accelerates the process but also maintains a relatively high success rate.