Efficient3D: A Unified Framework for Adaptive and Debiased Token Reduction in 3D MLLMs

Yuhui Lin; Siyue Yu; Yuxing Yang; Guangliang Cheng; Jimin Xiao

cs.CVApr 3, 2026

Efficient3D: A Unified Framework for Adaptive and Debiased Token Reduction in 3D MLLMs

Authors: Yuhui Lin, Siyue Yu, Yuxing Yang, Guangliang Cheng, Jimin Xiao

Organizations: Xi’an Jiaotong-Liverpool University · University of Liverpool

Abstract

Recent advances in Multimodal Large Language Models (MLLMs) have expanded reasoning capabilities into 3D domains, enabling fine-grained spatial understanding. However, the substantial size of 3D MLLMs and the high dimensionality of input features introduce considerable inference overhead, which limits practical deployment on resource constrained platforms. To overcome this limitation, this paper presents Efficient3D, a unified framework for visual token pruning that accelerates 3D MLLMs while maintaining competitive accuracy. The proposed framework introduces a Debiased Visual Token Importance Estimator (DVTIE) module, which considers the influence of shallow initial layers during attention aggregation, thereby producing more reliable importance predictions for visual tokens. In addition, an Adaptive Token Rebalancing (ATR) strategy is developed to dynamically adjust pruning strength based on scene complexity, preserving semantic completeness and maintaining balanced attention across layers. Together, they enable context-aware token reduction that maintains essential semantics with lower computation. Comprehensive experiments conducted on five representative 3D vision and language benchmarks, including ScanRefer, Multi3DRefer, Scan2Cap, ScanQA, and SQA3D, demonstrate that Efficient3D achieves superior performance compared with unpruned baselines, with a +2.57% CIDEr improvement on the Scan2Cap dataset. Therefore, Efficient3D provides a scalable and effective solution for efficient inference in 3D MLLMs. The code is released at: https://github.com/sol924/Efficient3D

Figures & tables

Figure 1: (a) illustrates that the initial layers in the 3D MLLM focus more on visual tokens on ScanRefer using LLaVA-1.5-7B. (b) compares global prediction and DVTIE in terms of correctly pruned visual tokens. (c) shows visual token pruning under simple and complex questions, where simple questions typically involve a single object, while complex questions involve multiple objects.

Figure 2: Overview of the Efficient3D framework. First, we perform unpruned training on a pretrained 3D MLLM and extract importance scores of visual tokens. Next, we use the visual importance scores as the supervision targets for training the proposed DVTIE. During inference, the 3D MLLM leverages the predicted importance scores from the DVTIE to perform visual token pruning. Furthermore, we propose an ATR strategy that adjusts the pruning ratio adaptively according to scene complexity.

	Method	Pruning	ScanRefer		Multi3DRefer		Scan2Cap		ScanQA		SQA3D		Score
	Method	Ratio	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	C	B-4	EM	EM-R	Ratio
Chat-Scene [ 27 ]	Baseline	0%	56.21	50.42	58.14	53.24	76.35	35.86	84.23	13.57	53.98	56.80	100.00%
	Random Pruning	35%	36.06	31.54	40.85	36.62	72.77	34.90	81.61	12.75	52.74	55.28	84.42%
		65%	14.67	12.51	19.92	18.04	31.14	26.61	76.42	12.28	51.02	53.41	60.38%
		90%	3.50	2.83	9.59	9.25	21.74	23.36	67.94	10.65	48.48	50.99	47.80%
	ToMe ${}^{\dagger}_{\text{ICLR23}}$ [ 2 ]	35%	50.43	45.39	51.95	47.70	73.43	35.76	83.46	13.39	52.64	55.12	94.69%

Table 1: Performance comparison with previous visual token compression methods. “Pruning Ratio” denotes the average ratio of pruned visual tokens. “Score Ratio” is obtained by calculating the average ratio of each score relative to Baseline. † denotes training-free methods.

Method	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D	Flops
Fast3D [ 34 ]	51.33	52.20	35.97	13.79	55.90	$\times$ 0.65
DVTIE + SAP [ 34 ]	51.50	52.12	36.17	13.75	55.99	$\times$ 0.65
DVTIE + ATR	52.98	52.39	36.87	14.20	56.72	$\times$ 0.38

Table 2: Inference efficiency of Efficient3D (DVTIE + ATR). SAP denotes the dynamic pruning module originally in Fast3D.

Figure 3: Visualization of DVTIE network under different visual token pruning ratios. The results in average pruning ratios of 35%, 65%, and 90%, respectively. Retained objects are marked in orange. Red boxes highlight key objects referenced in prompts.

Attention Map	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D
10% layers	9.24	14.57	25.74	11.82	51.47
30% layers	32.66	28.23	30.31	12.95	52.87
( $L-K$ ) layers	50.35	50.02	35.75	14.42	54.20
DVTIE	51.35	51.44	35.96	14.37	55.55

Table 3: Performance with attention maps from different layers. The visual token pruning ratio is set to 90% for all experiments. Predicting attention maps via our DVTIE compared with attention maps aggregated from (

L-K

) layers of the 3D MLLM where

L

= 32 and

K

= 2.

$K$ layers	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D
0	50.30	50.16	32.88	14.12	54.45
1	51.23	51.40	35.76	14.27	55.63
2	51.35	51.44	35.96	14.39	55.55
3	51.05	51.46	35.81	14.39	55.50

Table 4: Effect of the number of removed shallow initial layers

K

on the DVTIE training targets under an average pruning ratio of 90%. Here,

K

and

L

denote the number of removed shallow initial layers and the total number of layers in the 3D MLLM.

Methods	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D
cross-attention	50.88	51.04	35.81	13.92	54.89
high-rank	51.28	51.41	35.72	14.22	55.62
DHCT (Ours)	51.35	51.44	35.96	14.37	55.55

Table 5: Performance of using Dynamic Hybrid Compensation Attention Transformer (DHCT) layers in the DVTIE module under 90% average pruning ratio.

Low-Rank	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D
32	51.30	51.16	35.97	14.27	55.15
64	51.35	51.44	35.96	14.37	55.55
128	22.43	28.97	26.51	12.46	53.03

Table 6: Performance of using low-rank dimensions in the DHCT module under a 90% average pruning ratio.

Training target	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D
$a_{self}$	31.91	39.30	30.75	10.08	52.20
$a_{prompt}$	41.33	41.22	32.97	12.27	53.15
$a_{text}$	45.90	44.72	32.61	13.00	52.20
$a_{self}+a_{prompt}$	50.33	49.22	35.97	14.29	55.15
$a_{prompt}+a_{text}$	50.19	50.18	35.91	14.43	54.45
$a_{self}+a_{prompt}+a_{text}$	51.35	51.44	35.96	14.37	55.55

Table 7: Performance of aggregating different attention maps as the DVTIE’s training target.

Visual Token Pruning	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D
10% layers + ATR	10.04	14.03	25.75	11.72	52.49
30% layers + ATR	33.55	29.58	29.91	13.28	52.34
( $L-K$ ) layers + ATR	52.24	51.53	36.22	14.26	56.03
DVTIE + ATR	52.98	52.39	36.87	14.20	56.72

Table 8: Performance of ATR strategy with Different Attention Map Sources. Note that

L

= 32 and

K

= 2.

Scaling Factor $\lambda$	ScanRefer	Multi3DRefer	Scan2Cap	ScanQA	SQA3D
0	50.28	50.56	35.16	13.60	54.51
0.1	50.72	50.93	35.49	14.01	54.97
0.2	51.35	51.44	35.96	14.37	55.55
0.3	50.93	51.49	35.28	14.41	55.10

Table 9: Performance of the

\lambda

parameter in training the DVTIE network under a 90% average pruning ratio.

Efficient VLM Inference

Efficient vision-language model inference reduces the compute, memory, latency, and energy needed to process images, video, and text.

See all 482 papers

CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport
Aug 13, 2026Peng Ling, Yingda Yin, Lingting Zhu +53D Spatial Reasoning Visual Token Pruning
Shenzhen International Graduate School, Tsinghua University · LIGHTSPEED
Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answering
Jul 5, 2026Ruei-Chi Lai, Bolivar Solarte, Chin-Hsuan Wu +2Efficient VLM Inference Multimodal QA
National Tsing Hua University · Industrial Technology Research Institute ITRI · University of Toronto +1
HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding
Aug 5, 2026Jiuhe Qu, Yingping Liang, Ying Fu3D Spatial Reasoning Visual Token Pruning
Beijing Institute of Technology Beijing, China

Aug 13, 2026J/K move · Enter open · S save

CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport

Peng Ling, Yingda Yin, Lingting Zhu, Weikai Chen+4

Shenzhen International Graduate School, Tsinghua University · LIGHTSPEED

3D Spatial Reasoning Visual Token Pruning 3D VLMs

Open paper AskPDF arXiv

While 3D Vision-Language Models (3D VLMs) have demonstrated remarkable spatial reasoning capabilities, they suffer from massive visual token counts that create severe computational bottlenecks during inference. Existing token pruning methods primarily rely on diversity-based selection, discarding similar tokens to maximize dispersion. However, in 3D environments, this approach frequently drops representative prototype tokens in favor of outliers, breaking the multi-view consistencies and geometric structures essential for spatial reasoning. In this paper, we propose a paradigm shift for 3D VLM token pruning: from maximizing diversity to preserving visual evidence coverage. We introduce CoverPrune, a training-free framework that formulates inference-time token pruning as an Optimal Transport (OT) problem. To overcome the intractable combinatorial subset selection inherent in this formulation, we design the Feature-Spatial-Temporal (FST) transport cost and target capacity, along with an efficient Spatial-Guided Greedy Selection (SGS) algorithm to approximate the OT objective. Furthermore, we propose CoverPrune-Lite, an accelerated variant utilizing spatially structured local matching for minimal overhead. Extensive experiments across multiple 3D visual-spatial reasoning benchmarks demonstrate that our methods achieve state-of-the-art token efficiency, maintaining robust reasoning performance even under highly aggressive pruning budgets. Visit our project website at https://github.com/Brucess/CoverPrune.

Abstract

Efficient VLM Inference

Explore similar work

CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport

Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answering

HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding