Vision-language models (VLMs) have been widely adopted for 3D question answering (3D QA). In typical pipelines, visual tokens extracted from multiple viewpoints are concatenated with language tokens and jointly processed by a large language model (LLM) for inference. However, aggregating multi-view observations inevitably introduces severe token redundancy, leading to an overly large visual token set that significantly hinders inference efficiency under constrained token budgets. Visual token pruning has emerged as a prevalent strategy to address this issue. Nevertheless, most existing pruners are primarily tailored to 2D inputs or rely on indirect geometric cues, which limits their ability to explicitly retain semantically critical objects and maintain sufficient spatial coverage for robust 3D reasoning. In this paper, we propose SeGPruner, a semantic-aware and geometry-guided token reduction framework for efficient 3D QA with multi-view images. Specifically, SeGPruner first preserves semantically salient tokens through an attention-based importance module (Saliency-aware Token Selector), ensuring that object-critical evidence is retained. It then complements these tokens with spatially diverse ones via a geometry-guided selector (Geometry-aware Token Diversifier), which jointly considers semantic relevance and 3D geometric distance. This cooperation between saliency preservation and geometry-guided diversification balances object-level evidence and global scene coverage under aggressive token reduction. Extensive experiments on ScanQA and OpenEQA demonstrate that SeGPruner substantially improves inference efficiency, reducing the visual token budget by 91% and inference latency by 86%, while maintaining competitive performance in 3D reasoning tasks.
Figures & tables
Figure 1: Comparison between 2D visual token pruning and spatially-aware pruning for 3D QA. The visualization overlays retained visual tokens on the input image, where transparent regions indicate retained tokens and masked regions denote discarded ones. Conventional 2D pruning methods (top) allocate excessive tokens to background regions due to the lack of spatial awareness. In contrast, integrating spatial information (bottom) encourages more object-centric attention and more uniform sampling in 3D space, preserving richer and more comprehensive scene content for downstream reasoning.
Figure 2: Overall framework of our method. Our module is inserted between the visual encoder and the LLM and consists of three components: (1) 3D-Aware Feature Construction , (2) Salient Token Selection , and (3) Diverse Token Selection . Initially, the visual encoder extracts 2D visual tokens from multi-view inputs, where darker colors indicate higher attention scores. We select the top- k tokens as important tokens . To preserve spatial context, the remaining tokens are back-projected into 3D space using the corresponding depth map and processed by the Geometry-aware Token Diversifier to obtain diverse tokens . In the Geometry-aware Token Diversifier, tokens sharing the same shape originate from the same input view. Dashed outlines denote tokens discarded during selection. Finally, the important and diverse tokens are concatenated and fed into the LLM for cross-modal reasoning.
Figure 3: Illustration of Geometry-aware Token Diversifier in 3D space. The outer cube represents the 3D scene space, and geometric primitives denote visual tokens extracted from a single image. We first initialize the selected set D with the highest-attention token r0 . Subsequently, we iteratively calculate the minimum fusion distance from the remaining tokens to D and select the farthest candidate r1 to update the set. This strategy effectively reduces redundancy by discouraging tokens that are spatially or semantically close to those already selected.
Model
EM@1
Specialized Models
VoteNet+MCAN [ 54 ]
17.3
ScanQA [ 2 ]
21.1
3D-VisTA [ 55 ]
22.4
DSPNet [ 4 ]
23.5
BridgeQA [ 13 ]
27.0
Table 1: Comparison of our base model with representative 3DQA methods on ScanQA.
Model
LLM-Match
Proprietary Models
Gemini 1.0 Pro Vision
44.9
Claude-3.5 Sonnet
48.7
GPT4-V (15 frames)
54.6
GPT4-V (50 frames)
55.3
Open Models
Table 2: Comparison of our base model with representative 3DQA methods on OpenEQA
Table 3: Performance comparison of different pruning methods on ScanQA.
Table 4: Performance comparison of different pruning methods on OpenEQA.
Figure 4: Qualitative ablation of token selection strategies. (a) Full tokens without reduction. (b) Saliency-only selection (Saliency-aware Token Selector) preserves salient objects but may miss fine-grained details like cables. (c) Diversity-only selection (Geometry-aware Token Diversifier) improves spatial coverage but can overlook semantically important regions on the tabletop. (d) Our method combines saliency and spatial diversity, retaining salient objects while preserving fine-grained structures and producing more continuous and complete structures for large objects such as the table.
Method
Strategy
ScanQA
OpenEQA
Base Model (Full)
None (100% tokens)
27.6
56.2
Compressed Models w/ Retention:
23%
26%
+ Uniform
Uniform
27.1
53.5
+ STS
Sal-only
27.9
54.9
+ GTD
Div-only
27.8
54.5
+ Ours
Sal + Div
28.0
55.0
Table 5: Ablation Study of Components in Our Method.
Method
Retention Ratio (%)
54
40
23
14
9
Uniform
28.3
27.7
27.1
27.7
21.2
Semantic Similarity
28.2
28.0
27.4
27.0
25.8
GTD
28.4
28.3
27.8
26.7
26.3
Table 6: Comparison of Diversity-Aware Token Selection Strategies on ScanQA (Without Important Tokens).
Figure 5: Inference time comparison between VisPruner [ 1 ] and ours on the ScanQA [ 2 ] validation set. Our method achieves lower latency per example at all token retention ratios.
Figure 6: Qualitative comparison between our method and VisPruner on the ScanQA dataset. Left: retained tokens mapped onto the original image, with transparent areas denoting the selected tokens; our method achieves a more even spatial distribution of tokens in the image plane, thereby offering a more complete set of visual cues. Right: retained tokens projected into 3D space; compared with VisPruner, our approach produces a more continuous and complete point distribution, better capturing object structures with fewer holes and higher geometric completeness.