Vision-language models (VLMs) have been widely adopted for 3D question answering (3D QA). In typical pipelines, visual tokens extracted from multiple viewpoints are concatenated with language tokens and jointly processed by a large language model (LLM) for inference. However, aggregating multi-view observations inevitably introduces severe token redundancy, leading to an overly large visual token set that significantly hinders inference efficiency under constrained token budgets. Visual token pruning has emerged as a prevalent strategy to address this issue. Nevertheless, most existing pruners are primarily tailored to 2D inputs or rely on indirect geometric cues, which limits their ability to explicitly retain semantically critical objects and maintain sufficient spatial coverage for robust 3D reasoning. In this paper, we propose SeGPruner, a semantic-aware and geometry-guided token reduction framework for efficient 3D QA with multi-view images. Specifically, SeGPruner first preserves semantically salient tokens through an attention-based importance module (Saliency-aware Token Selector), ensuring that object-critical evidence is retained. It then complements these tokens with spatially diverse ones via a geometry-guided selector (Geometry-aware Token Diversifier), which jointly considers semantic relevance and 3D geometric distance. This cooperation between saliency preservation and geometry-guided diversification balances object-level evidence and global scene coverage under aggressive token reduction. Extensive experiments on ScanQA and OpenEQA demonstrate that SeGPruner substantially improves inference efficiency, reducing the visual token budget by 91% and inference latency by 86%, while maintaining competitive performance in 3D reasoning tasks.
Figures & tables
Figure 1: Comparison between 2D visual token pruning and spatially-aware pruning for 3D QA. The visualization overlays retained visual tokens on the input image, where transparent regions indicate retained tokens and masked regions denote discarded ones. Conventional 2D pruning methods (top) allocate excessive tokens to background regions due to the lack of spatial awareness. In contrast, integrating spatial information (bottom) encourages more object-centric attention and more uniform sampling in 3D space, preserving richer and more comprehensive scene content for downstream reasoning.
Figure 2: Overall framework of our method. Our module is inserted between the visual encoder and the LLM and consists of three components: (1) 3D-Aware Feature Construction , (2) Salient Token Selection , and (3) Diverse Token Selection . Initially, the visual encoder extracts 2D visual tokens from multi-view inputs, where darker colors indicate higher attention scores. We select the top- k tokens as important tokens . To preserve spatial context, the remaining tokens are back-projected into 3D space using the corresponding depth map and processed by the Geometry-aware Token Diversifier to obtain diverse tokens . In the Geometry-aware Token Diversifier, tokens sharing the same shape originate from the same input view. Dashed outlines denote tokens discarded during selection. Finally, the important and diverse tokens are concatenated and fed into the LLM for cross-modal reasoning.
Figure 3: Illustration of Geometry-aware Token Diversifier in 3D space. The outer cube represents the 3D scene space, and geometric primitives denote visual tokens extracted from a single image. We first initialize the selected set D with the highest-attention token r0 . Subsequently, we iteratively calculate the minimum fusion distance from the remaining tokens to D and select the farthest candidate r1 to update the set. This strategy effectively reduces redundancy by discouraging tokens that are spatially or semantically close to those already selected.
Model
EM@1
Specialized Models
VoteNet+MCAN [ 54 ]
17.3
ScanQA [ 2 ]
21.1
3D-VisTA [ 55 ]
22.4
DSPNet [ 4 ]
23.5
BridgeQA [ 13 ]
27.0
Table 1: Comparison of our base model with representative 3DQA methods on ScanQA.
Model
LLM-Match
Proprietary Models
Gemini 1.0 Pro Vision
44.9
Claude-3.5 Sonnet
48.7
GPT4-V (15 frames)
54.6
GPT4-V (50 frames)
55.3
Open Models
Table 2: Comparison of our base model with representative 3DQA methods on OpenEQA
Table 3: Performance comparison of different pruning methods on ScanQA.
Table 4: Performance comparison of different pruning methods on OpenEQA.
Figure 4: Qualitative ablation of token selection strategies. (a) Full tokens without reduction. (b) Saliency-only selection (Saliency-aware Token Selector) preserves salient objects but may miss fine-grained details like cables. (c) Diversity-only selection (Geometry-aware Token Diversifier) improves spatial coverage but can overlook semantically important regions on the tabletop. (d) Our method combines saliency and spatial diversity, retaining salient objects while preserving fine-grained structures and producing more continuous and complete structures for large objects such as the table.
Method
Strategy
ScanQA
OpenEQA
Base Model (Full)
None (100% tokens)
27.6
56.2
Compressed Models w/ Retention:
23%
26%
+ Uniform
Uniform
27.1
53.5
+ STS
Sal-only
27.9
54.9
+ GTD
Div-only
27.8
54.5
+ Ours
Sal + Div
28.0
55.0
Table 5: Ablation Study of Components in Our Method.
Method
Retention Ratio (%)
54
40
23
14
9
Uniform
28.3
27.7
27.1
27.7
21.2
Semantic Similarity
28.2
28.0
27.4
27.0
25.8
GTD
28.4
28.3
27.8
26.7
26.3
Table 6: Comparison of Diversity-Aware Token Selection Strategies on ScanQA (Without Important Tokens).
Figure 5: Inference time comparison between VisPruner [ 1 ] and ours on the ScanQA [ 2 ] validation set. Our method achieves lower latency per example at all token retention ratios.
Figure 6: Qualitative comparison between our method and VisPruner on the ScanQA dataset. Left: retained tokens mapped onto the original image, with transparent areas denoting the selected tokens; our method achieves a more even spatial distribution of tokens in the image plane, thereby offering a more complete set of visual cues. Right: retained tokens projected into 3D space; compared with VisPruner, our approach produces a more continuous and complete point distribution, better capturing object structures with fewer holes and higher geometric completeness.
Recent Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on 2D question answering tasks. However, extending these models to the 3D question answering remains challenging, as they typically require multiple views of the scene, which incurs substantial computational cost at inference. To mitigate this issue, existing solutions rely on strategic frame selection or token-merging algorithms that require preprocessing in advance all frames of the scene, i.e., an offline fashion. In contrast, we propose the first online token-pruning method that can be integrated seamlessly with current MLLM models for 3D question answering tasks, without additional training and with lower memory usage.Our key insight is to project each input frame into a shared voxel space using depth information and camera pose, identifying spatially-overlapped regions across frames and selectively pruning redundant image tokens before they enter the language model. Our method enables efficient online processing while reducing up to 50% of token usage. We apply this approach to Qwen2.5-VL-7B and Qwen3-VL-8B, demonstrating improved performance on the ScanQA, SQA3D, and OpenEQA-HM3D benchmarks.
Ruei-Chi Lai, Bolivar Solarte, Chin-Hsuan Wu +2
National Tsing Hua University · Industrial Technology Research Institute ITRI · University of Toronto +1
Multimodal large language models have demonstrated remarkable capabilities in 2D vision, motivating their extension to 3D scene understanding. Recent studies represent 3D scenes as 3D spatial videos composed of image sequences with depth and camera pose information, enabling pre-trained video-language models to perform 3D reasoning tasks. However, the large number of visual tokens in spatial videos remains a major bottleneck for efficient inference and context management. Existing pruning methods overlook the view consistency of spatial videos and the spatial diversity of the remaining tokens, which prevents them from effectively removing inter-frame redundancy and preserving scene completeness. In this paper, we propose Geo3DPruner, a Geometry-Guided 3D Visual Token Pruning framework. Geo3DPruner first models cross-frame relevance through geometry-aware global attention, and then performs a two-stage pruning process. The intra-voxel stage selects representative multi-view features within each voxel, while the inter-voxel stage preserves spatial diversity by selecting a globally distributed subset of voxels. Extensive experiments on multiple 3D scene understanding benchmarks demonstrate that Geo3DPruner retains over 90% of the original performance while pruning 90% of visual tokens, significantly outperforming existing text-guided and vision-guided pruning methods.
While 3D Vision-Language Models (3D VLMs) have demonstrated remarkable spatial reasoning capabilities, they suffer from massive visual token counts that create severe computational bottlenecks during inference. Existing token pruning methods primarily rely on diversity-based selection, discarding similar tokens to maximize dispersion. However, in 3D environments, this approach frequently drops representative prototype tokens in favor of outliers, breaking the multi-view consistencies and geometric structures essential for spatial reasoning. In this paper, we propose a paradigm shift for 3D VLM token pruning: from maximizing diversity to preserving visual evidence coverage. We introduce CoverPrune, a training-free framework that formulates inference-time token pruning as an Optimal Transport (OT) problem. To overcome the intractable combinatorial subset selection inherent in this formulation, we design the Feature-Spatial-Temporal (FST) transport cost and target capacity, along with an efficient Spatial-Guided Greedy Selection (SGS) algorithm to approximate the OT objective. Furthermore, we propose CoverPrune-Lite, an accelerated variant utilizing spatially structured local matching for minimal overhead. Extensive experiments across multiple 3D visual-spatial reasoning benchmarks demonstrate that our methods achieve state-of-the-art token efficiency, maintaining robust reasoning performance even under highly aggressive pruning budgets. Visit our project website at https://github.com/Brucess/CoverPrune.
Peng Ling, Yingda Yin, Lingting Zhu +5
Shenzhen International Graduate School, Tsinghua University · LIGHTSPEED