Recent advances in Multimodal Large Language Models (MLLMs) have expanded reasoning capabilities into 3D domains, enabling fine-grained spatial understanding. However, the substantial size of 3D MLLMs and the high dimensionality of input features introduce considerable inference overhead, which limits practical deployment on resource constrained platforms. To overcome this limitation, this paper presents Efficient3D, a unified framework for visual token pruning that accelerates 3D MLLMs while maintaining competitive accuracy. The proposed framework introduces a Debiased Visual Token Importance Estimator (DVTIE) module, which considers the influence of shallow initial layers during attention aggregation, thereby producing more reliable importance predictions for visual tokens. In addition, an Adaptive Token Rebalancing (ATR) strategy is developed to dynamically adjust pruning strength based on scene complexity, preserving semantic completeness and maintaining balanced attention across layers. Together, they enable context-aware token reduction that maintains essential semantics with lower computation. Comprehensive experiments conducted on five representative 3D vision and language benchmarks, including ScanRefer, Multi3DRefer, Scan2Cap, ScanQA, and SQA3D, demonstrate that Efficient3D achieves superior performance compared with unpruned baselines, with a +2.57% CIDEr improvement on the Scan2Cap dataset. Therefore, Efficient3D provides a scalable and effective solution for efficient inference in 3D MLLMs. The code is released at: https://github.com/sol924/Efficient3D
Figures & tables
Figure 1: (a) illustrates that the initial layers in the 3D MLLM focus more on visual tokens on ScanRefer using LLaVA-1.5-7B. (b) compares global prediction and DVTIE in terms of correctly pruned visual tokens. (c) shows visual token pruning under simple and complex questions, where simple questions typically involve a single object, while complex questions involve multiple objects.
Figure 2: Overview of the Efficient3D framework. First, we perform unpruned training on a pretrained 3D MLLM and extract importance scores of visual tokens. Next, we use the visual importance scores as the supervision targets for training the proposed DVTIE. During inference, the 3D MLLM leverages the predicted importance scores from the DVTIE to perform visual token pruning. Furthermore, we propose an ATR strategy that adjusts the pruning ratio adaptively according to scene complexity.
Table 1: Performance comparison with previous visual token compression methods. “Pruning Ratio” denotes the average ratio of pruned visual tokens. “Score Ratio” is obtained by calculating the average ratio of each score relative to Baseline. † denotes training-free methods.
Method
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
Flops
Fast3D [ 34 ]
51.33
52.20
35.97
13.79
55.90
× 0.65
DVTIE + SAP [ 34 ]
51.50
52.12
36.17
13.75
55.99
× 0.65
DVTIE + ATR
52.98
52.39
36.87
14.20
56.72
× 0.38
Table 2: Inference efficiency of Efficient3D (DVTIE + ATR). SAP denotes the dynamic pruning module originally in Fast3D.
Figure 3: Visualization of DVTIE network under different visual token pruning ratios. The results in average pruning ratios of 35%, 65%, and 90%, respectively. Retained objects are marked in orange. Red boxes highlight key objects referenced in prompts.
Attention Map
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
10% layers
9.24
14.57
25.74
11.82
51.47
30% layers
32.66
28.23
30.31
12.95
52.87
( L−K ) layers
50.35
50.02
35.75
14.42
54.20
DVTIE
51.35
51.44
35.96
14.37
55.55
Table 3: Performance with attention maps from different layers. The visual token pruning ratio is set to 90% for all experiments. Predicting attention maps via our DVTIE compared with attention maps aggregated from ( L−K ) layers of the 3D MLLM where L = 32 and K = 2.
K layers
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
0
50.30
50.16
32.88
14.12
54.45
1
51.23
51.40
35.76
14.27
55.63
2
51.35
51.44
35.96
14.39
55.55
3
51.05
51.46
35.81
14.39
55.50
Table 4: Effect of the number of removed shallow initial layers K on the DVTIE training targets under an average pruning ratio of 90%. Here, K and L denote the number of removed shallow initial layers and the total number of layers in the 3D MLLM.
Methods
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
cross-attention
50.88
51.04
35.81
13.92
54.89
high-rank
51.28
51.41
35.72
14.22
55.62
DHCT (Ours)
51.35
51.44
35.96
14.37
55.55
Table 5: Performance of using Dynamic Hybrid Compensation Attention Transformer (DHCT) layers in the DVTIE module under 90% average pruning ratio.
Low-Rank
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
32
51.30
51.16
35.97
14.27
55.15
64
51.35
51.44
35.96
14.37
55.55
128
22.43
28.97
26.51
12.46
53.03
Table 6: Performance of using low-rank dimensions in the DHCT module under a 90% average pruning ratio.
Training target
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
aself
31.91
39.30
30.75
10.08
52.20
aprompt
41.33
41.22
32.97
12.27
53.15
atext
45.90
44.72
32.61
13.00
52.20
aself+aprompt
50.33
49.22
35.97
14.29
55.15
aprompt+atext
50.19
50.18
35.91
14.43
54.45
aself+aprompt+atext
51.35
51.44
35.96
14.37
55.55
Table 7: Performance of aggregating different attention maps as the DVTIE’s training target.
Visual Token Pruning
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
10% layers + ATR
10.04
14.03
25.75
11.72
52.49
30% layers + ATR
33.55
29.58
29.91
13.28
52.34
( L−K ) layers + ATR
52.24
51.53
36.22
14.26
56.03
DVTIE + ATR
52.98
52.39
36.87
14.20
56.72
Table 8: Performance of ATR strategy with Different Attention Map Sources. Note that L = 32 and K = 2.
Scaling Factor λ
ScanRefer
Multi3DRefer
Scan2Cap
ScanQA
SQA3D
0
50.28
50.56
35.16
13.60
54.51
0.1
50.72
50.93
35.49
14.01
54.97
0.2
51.35
51.44
35.96
14.37
55.55
0.3
50.93
51.49
35.28
14.41
55.10
Table 9: Performance of the λ parameter in training the DVTIE network under a 90% average pruning ratio.
While 3D Vision-Language Models (3D VLMs) have demonstrated remarkable spatial reasoning capabilities, they suffer from massive visual token counts that create severe computational bottlenecks during inference. Existing token pruning methods primarily rely on diversity-based selection, discarding similar tokens to maximize dispersion. However, in 3D environments, this approach frequently drops representative prototype tokens in favor of outliers, breaking the multi-view consistencies and geometric structures essential for spatial reasoning. In this paper, we propose a paradigm shift for 3D VLM token pruning: from maximizing diversity to preserving visual evidence coverage. We introduce CoverPrune, a training-free framework that formulates inference-time token pruning as an Optimal Transport (OT) problem. To overcome the intractable combinatorial subset selection inherent in this formulation, we design the Feature-Spatial-Temporal (FST) transport cost and target capacity, along with an efficient Spatial-Guided Greedy Selection (SGS) algorithm to approximate the OT objective. Furthermore, we propose CoverPrune-Lite, an accelerated variant utilizing spatially structured local matching for minimal overhead. Extensive experiments across multiple 3D visual-spatial reasoning benchmarks demonstrate that our methods achieve state-of-the-art token efficiency, maintaining robust reasoning performance even under highly aggressive pruning budgets. Visit our project website at https://github.com/Brucess/CoverPrune.
Peng Ling, Yingda Yin, Lingting Zhu +5
Shenzhen International Graduate School, Tsinghua University · LIGHTSPEED
Recent Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on 2D question answering tasks. However, extending these models to the 3D question answering remains challenging, as they typically require multiple views of the scene, which incurs substantial computational cost at inference. To mitigate this issue, existing solutions rely on strategic frame selection or token-merging algorithms that require preprocessing in advance all frames of the scene, i.e., an offline fashion. In contrast, we propose the first online token-pruning method that can be integrated seamlessly with current MLLM models for 3D question answering tasks, without additional training and with lower memory usage.Our key insight is to project each input frame into a shared voxel space using depth information and camera pose, identifying spatially-overlapped regions across frames and selectively pruning redundant image tokens before they enter the language model. Our method enables efficient online processing while reducing up to 50% of token usage. We apply this approach to Qwen2.5-VL-7B and Qwen3-VL-8B, demonstrating improved performance on the ScanQA, SQA3D, and OpenEQA-HM3D benchmarks.
Ruei-Chi Lai, Bolivar Solarte, Chin-Hsuan Wu +2
National Tsing Hua University · Industrial Technology Research Institute ITRI · University of Toronto +1
3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy due to duplicated observations and large uninformative regions, leading to high computational cost. Although visual token compression has shown promise in accelerating 2D VLMs, it fails to capture the structured nature of 3D scenes and leads to incomplete spatial coverage and loss of fine-grained details. In this paper, we propose HiSC, a training-free framework for hierarchical spatial clustering token compression in 3D VLMs. HiSC lifts token compression from token-level selection to cluster-level processing by organizing tokens into spatially grounded clusters using joint geometric and semantic cues. Specifically, we first introduce a spatial graph-based merging (SGraM) strategy that models cross-view redundancy as spatial connectivity and consolidates physically consistent regions, effectively merging extremely similar redundant tokens prior to LLM inference. We then propose a spatial clustering-based pruning (SCluP) paradigm within LLM inference, which performs hierarchical compression across clusters and within clusters, preserving object instance completeness while retaining fine-grained details for important regions. Extensive experiments on diverse 3D reasoning benchmarks show validate the effectiveness of HiSC, particularly under high visual token pruning ratios. Besides, HiSC achieves over 90% token reduction with minimal performance degradation. Code is accessible at https://github.com/elecreak/HiSC.