GeoGR: Spatial-Semantic Token Compression and Group Routing for Efficient 3D VLMs

Yongxin Zheng; Hao Wu; Fang Li; Hu Fu; Peiran Yin; Xudong Wang; Haozhe Hu; Xinghao Chen

cs.CVOct 9, 2026

GeoGR: Spatial-Semantic Token Compression and Group Routing for Efficient 3D VLMs

Authors: Yongxin Zheng, Hao Wu, Fang Li, Hu Fu, Peiran Yin, Xudong Wang, Haozhe Hu, Xinghao Chen,

Organizations: EIT-NLP Lab, Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University · The Hong Kong Polytechnic University · Munich Center for Machine Learning, LMU Munich

Abstract

Multi-view representations enable pretrained image and video vision-language models to understand 3D scenes, but their long visual token sequences make inference costly. Redundancy arises both from repeated observations of the same scene regions across views and from processing all visual tokens at every LLM layer, regardless of their relevance to the current computation. We propose GeoGR, a two-stage framework that jointly compresses scene representations and visual computation. At the projector level, GeoSemZip consolidates cross-view observations into 3D voxel tokens and combines semantic saliency with spatial coverage to construct a compact scene representation. Unselected voxel features are aggregated into contextual anchors to retain scene information that would otherwise be discarded. At the LLM level, we identify an effective visual computation window and observe that query-conditioned token relevance remains locally stable but changes over longer layer intervals. Guided by these findings, GroupRoute combines late entry and early exit with recoverable group-wise skipping. Low-relevance tokens bypass complete LLM layers while retaining their hidden states and can rejoin the active group at subsequent anchor layers selected under a computation budget. For compression-aware post-training, we construct a compact dataset of 67K samples by retaining 30% of the original training pool. With this protocol, GeoGR retains 95.2% and 96.8% of the dense post-trained models' performance on LLaVA-OneVision-7B and Video-3D LLM-7B, respectively, while accelerating LLM prefill by $6.8\times$ and $6.1\times$ . These results demonstrate the consistent effectiveness of GeoGR across 3D VLMs with and without explicit 3D spatial encoding, highlighting a favorable performance-efficiency trade-off. The code is available at https://github.com/EIT-NLP/GeoGR.

Figures & tables

Figure 1: Redundancy in multi-view 3D VLMs. (a) Overlapping views repeatedly observe the same 3D region (blue boxes). (b) For the same question, visual token importance remains locally stable but shifts across more distant layers. See Section 3 for quantitative analysis.

Figure 2: Cross-layer stability of visual token rankings. Ranking stability decreases with layer distance, while initially low-ranked tokens increasingly enter the top half and account for a growing share of its attention.

Source	Full Set	Selected
ScanRefer	36,665	11,000
Multi3DRefer	43,838	13,151
Scan2Cap	36,665	11,000
ScanQA	26,515	7,955
SQA3D	79,445	23,834
Total	223,128	66,940

Table 1: Our post-training dataset distilled by selecting 30% of samples from each source using PRISM (top), and performance retained relative to full-data post-training (bottom).

Figure 3: Overview of GeoGR . (1) Projector-level Compression: GeoSemZip consolidates cross-view observations into 3D voxel tokens using depth and camera poses. Saliency-based selection preserves dominant tokens, while Morton-order sampling selects contextual anchors that aggregate unselected voxel features by cosine similarity. The compressed visual tokens are combined with text tokens to form the LLM input. (2) LLM-level Compression: GroupRoute confines visual computation to an intermediate-layer window through Late Entry and Early Exit. Within this window, Recoverable Group-wise Skipping routes query-conditioned token groups through or around complete LLM layers. Skipped tokens retain their hidden states and can resume computation after regrouping at subsequent anchor layers. Text tokens remain active throughout the LLM.

Figure 4: Visual-window diagnosis for LLaVA-OneVision on 256 ScanQA examples: Late Entry (left) and Early Exit (right). Lines show mean next-token KL divergence relative to full visual computation; shading indicates

\pm 1

standard deviation, with the lower bound clipped at zero.

Method	Budget		ScanQA		SQA3D	Scan2Cap		ScanRefer		Multi3DRefer		Avg.
Method	Tokens	TFLOPs	CIDEr	EM	EM	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	(%)
LLaVA-OneVision-7B
Dense	6272	97.64	94.84	28.09	57.06							100.00
VisionZip [ yang2025visionzip ]	640	8.52	78.54	23.14	51.41	-	-	-	-	-	-	85.10
VisPruner [ zhang2025beyond ]	640	8.52	80.37	23.81	52.09	-	-	-	-	-	-	86.93
SeGPruner ∗ [ li2026segpruner ]	640	8.52	85.15	25.43	53.88	-	-	-	-	-	-	91.59

Table 2: Performance on LLaVA-OneVision-7B and Video-3D LLM-7B at approximately 10% retained LLM-side visual FLOPs. LLaVA-OneVision is evaluated only on ScanQA and SQA3D, as it does not support the 3D localization outputs required by the other benchmarks. “Avg.” measures performance retention relative to the dense model. “Tokens” denotes the average number of active visual tokens per LLM layer. Unshaded baseline rows apply compression without recovery, while light-red rows use compression-aware post-training. Light-blue rows denote GeoGR. † marks baselines originally proposed without recovery and evaluated with it here. ∗ marks compression baselines originally designed for 3D inputs, while the remaining baselines were designed for 2D inputs. Bold and underlined values indicate the best and second-best results among compressed methods, respectively. GeoGR achieves the highest average performance retention on both 3D VLMs at comparable computational budgets.

Method	KV Cache (MB)	Latency (ms)			Avg. (%)
Method	KV Cache (MB)	Projector	LLM Prefill	TTFT	Avg. (%)
LLaVA-OneVision-7B
Dense	359.7	53.3	1021.4 ( $1.0\times$ )	1426.7 ( $1.0\times$ )	100.0
SeGPruner †	36.7	100.8	103.0 ( $9.9\times$ )	548.5 ( $2.6\times$ )	91.7
DTC †	40.3	2025.0	147.2 ( $6.9\times$ )	2522.2 ( $0.6\times$ )	92.6
GeoGR	36.6	99.0	151.2 ( $6.8\times$ )	605.0 ( $2.4\times$ )	95.2

Table 3: Efficiency comparison on two 3D VLMs. Parentheses indicate speedups relative to the dense model. “Projector” covers all operations between the vision encoder and the LLM, including token compression. Encoder latency is similar across methods and is omitted from the breakdown but included in TTFT. GeoGR offers a favorable trade-off between performance retention and inference efficiency on both 3D VLMs.

Semantic	Spatial	Residual	ScanQA		SQA3D	Avg.
Saliency	Coverage	Merge	CIDEr	EM	EM	Avg.
	✓		85.97	26.16	55.24	93.53%
✓			92.04	26.50	56.81	96.99%
✓	✓		92.67	26.95	57.57	98.19%
✓	✓	✓	93.95	27.51	57.63	99.33%

Table 4: Component ablation of GeoSemZip. All variants retain 30% of the visual tokens and are post-trained independently from the same initialization. The semantic-only variant omits cross-view voxel consolidation. Residual merging requires contextual anchors from the spatial coverage branch.

Grouping	ScanQA		SQA3D	Avg.
Signal	CIDEr	EM	EM	Avg.
Projector-derived	85.34	25.30	55.98	92.73%
Query-conditioned	88.25	26.20	56.61	95.18%

Table 5: Grouping signal with anchor layers 9 and 13 and

\kappa=0.5

. Both variants share the same checkpoint, schedule, and decoding protocol and differ only in the ranking used to form the groups.

Method	Anchors	ScanQA		SQA3D	Avg.
Method	Anchors	CIDEr	EM	EM	Avg.
Pruning	9	87.66	25.97	56.52	94.65%
	13	87.20	26.07	57.23	95.03%
	9, 13	87.82	25.88	56.75	94.73%
Skipping	9, 13	88.25	26.20	56.61	95.18%

Table 6: Skipping vs. pruning within the window

[8,23]

. All variants use the same window-adapted checkpoint without further training and average 638 active visual tokens per LLM layer.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Blocks	Active sequence	Event
1–7	Text only	Visual tokens not yet injected
8	All	Visual tokens enter
9	All	Score and group at anchor 9
10–12	Text, $\mathcal{H}_{9}$	$\mathcal{L}_{9}$ bypasses
13	All	Anchor 13; score and regroup
14–23	Text, $\mathcal{H}_{13}$	$\mathcal{L}_{13}$ bypasses

Appendix

Table 7: GroupRoute execution schedule on LLaVA-OneVision.

Component	LLaVA-OneVision-7B	Video-3D LLM-7B
Pretrained VLM family	LLaVA-OneVision	LLaVA-Video
Vision encoder	SigLIP SO400M/14	SigLIP SO400M/14
Encoder input resolution	$384\times 384$	$384\times 384$
Visual feature dimension	1,152	1,152
Multimodal projector	Two-layer MLP, GELU	Two-layer MLP, GELU
LLM	Qwen2-7B	Qwen2-7B

Appendix

Table 8: Architectures and scene inputs of the evaluated 3D VLMs. Token counts use the 32-view setting and exclude formatting tokens. Q/KV denotes query and key/value attention heads.

Benchmark	Train	Eval split	Eval	Scenes	Reported metrics
ScanQA [ azuma2022scanqa ]	26,515	Validation	4,675	71	CIDEr, normalized EM
SQA3D [ ma2023sqa3d ]	79,445	Test	3,519	67	Normalized EM
Scan2Cap [ chen2021scan2cap ]	36,665	Validation	2,068	141	BLEU-4, CIDEr (IoU 0.5)
ScanRefer [ chen2020scanrefer ]	36,665	Validation	9,508	141	[email protected], [email protected]
Multi3DRefer [ zhang2023multi3drefer ]	43,838	Validation	11,120	141	[email protected], [email protected]

Appendix

Table 9: Training sources and evaluation splits for five ScanNet-based benchmarks. Train counts examples before PRISM selection; Eval counts held-out examples. Evaluation and scene counts follow the Video-3D LLM annotations [ zheng2024video3dllm ] .

Training data	ScanQA CIDEr	ScanQA EM	SQA3D EM	Retention (%)
Full set (223,128)	100.27	29.07	59.53	100.00
PRISM 30% (66,940)	94.84	28.09	57.06	95.69

Appendix

Table 10: Dense post-training on full data and PRISM-selected data. PRISM retains approximately 30% from each source; retention averages the metric-wise ratios to the full-data post-trained checkpoint.

Budget	Vis. tokens	Avg. (%)
$\approx 20\%$	1255.00	97.26
$\approx 30\%$	1881.73	99.33
$\approx 40\%$	2475.47	98.38

Appendix

Table 11: GeoSemZip budget selection on LLaVA-OneVision. Each budget uses a separately post-trained checkpoint. Vis. tokens is the ScanQA mean entering the LLM; Avg. is retention relative to Dense.

Voxel size $s$	$K$	Keep ratio	ScanQA		SQA3D	Avg.
Voxel size $s$	$K$	Keep ratio	CIDEr	EM	EM	Avg.
0.05	4778.6	30%	92.10	27.40	57.26	98.34%
0.10	3574.0	30%	93.95	27.51	57.63	99.33%
0.15	2520.8	30%	91.31	27.42	57.63	98.30%

Appendix

Table 13: GeoSemZip voxel-size ablation at a 30% token budget. LLaVA-OneVision variants are post-trained independently;

K

is the mean number of occupied voxels per scene.

Aggregation	ScanQA		SQA3D	Avg. (%)
Aggregation	CIDEr	EM	EM	Avg. (%)
Mean	93.15	27.76	57.12	99.06
Max (default)	93.95	27.51	57.63	99.33

Appendix

Table 14: Voxel-score aggregation for GeoSemZip at a 30% token budget. LLaVA-OneVision variants are post-trained independently and differ only in the within-voxel score-pooling rule.

Dominant ratio $\rho$	ScanQA		SQA3D	Avg. (%)
Dominant ratio $\rho$	CIDEr	EM	EM	Avg. (%)
0.65	91.46	27.40	57.46	98.23
0.75	92.89	27.66	57.37	98.99
0.85 (default)	93.95	27.51	57.63	99.33
0.95	92.36	27.61	57.00	98.52

Appendix

Table 15: GeoSemZip budget-split ablation at a 30% token budget. On LLaVA-OneVision, independently post-trained variants differ in

\rho

, the fraction assigned to dominant tokens; residual merging remains enabled.

Window	ScanQA		SQA3D	Avg.
Window	CIDEr	EM	EM	Avg.
Step 1: late entry (visual tokens through block 28)
$[5,28]$	87.60	25.56	56.27	93.99%
$[6,28]$	87.87	25.13	56.15	93.51%
$[7,28]$	85.37	25.41	55.61	92.65%
$[8,28]$	86.61	25.35	56.49	93.53%

Appendix

Table 16: Visual-computation window selection on LLaVA-OneVision. We first vary late entry with visual tokens retained through block 28, then vary early exit with entry fixed at block 8. Avg. is relative to the dense reference in Table 2 .

Keep ratios (9 / 13)	Visual tokens	ScanQA		SQA3D	Avg.
Keep ratios (9 / 13)	Visual tokens	CIDEr	EM	EM	Avg.
0.75 / 0.25	520.79	87.28	25.99	56.38	94.46%
0.50 / 0.50 (default)	638.38	88.25	26.20	56.61	95.18%
0.25 / 0.75	755.98	87.77	26.07	56.07	94.55%

Appendix

Table 17: Keep-ratio allocation at GroupRoute anchors 9 and 13. On LLaVA-OneVision with window

[8,23]

, “Visual tokens” is the mean active count per LLM layer; Avg. is relative to the dense model.

Panel of Figure 5 — Figure 5: Layer-pair stability of visual-token relevance on ScanQA. For LLaVA-OneVision decoder layers 8–23, (a) gives Spearman correlation between complete visual-token rankings, and (b) gives the fraction shared by the two layers’ top-200 sets.

Configuration	Proj. keep	Vis. tokens	ScanQA		SQA3D	Avg. (%)
Configuration	Proj. keep	Vis. tokens	CIDEr	EM	EM	Avg. (%)
GeoSemZip only	10%	628	87.44	25.41	56.69	94.00
GroupRoute only	100%	2128	88.48	26.07	55.84	94.65
GeoGR (both)	30%	638	88.25	26.20	56.61	95.18

Appendix

Table 18: Projector-only, LLM-only, and two-stage compression on LLaVA-OneVision. Vis. tokens is the mean active count per decoder block; Avg. is retention relative to the dense model.

Layer Skipping in Language Model Decoding

See all 11 papers

HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding
Aug 5, 2026Jiuhe Qu, Yingping Liang, Ying Fu3D Spatial Reasoning Visual Token Pruning
Beijing Institute of Technology Beijing, China
Geometry-Guided 3D Visual Token Pruning for Video-Language Models
Apr 20, 2026Han Li, Zehao Huang, Jiahui Fu +2Visual Token Pruning Visual Token Compression
Beihang University · Zhongguancun Academy
3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
Aug 2, 2026Changwoo Baek, Kyeongbo KongMultimodal Token Compression 3D VQA
Department of Electrical and Electronics Engineering, Pusan National University, Busan, Republic of Korea

Aug 5, 2026J/K move · Enter open · S save

HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding

Jiuhe Qu, Yingping Liang, Ying Fu

Beijing Institute of Technology Beijing, China

3D Spatial Reasoning Visual Token Pruning Visual Token Compression 3D Scene Understanding 3D VLMs

Open paper AskPDF arXiv

3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy due to duplicated observations and large uninformative regions, leading to high computational cost. Although visual token compression has shown promise in accelerating 2D VLMs, it fails to capture the structured nature of 3D scenes and leads to incomplete spatial coverage and loss of fine-grained details. In this paper, we propose HiSC, a training-free framework for hierarchical spatial clustering token compression in 3D VLMs. HiSC lifts token compression from token-level selection to cluster-level processing by organizing tokens into spatially grounded clusters using joint geometric and semantic cues. Specifically, we first introduce a spatial graph-based merging (SGraM) strategy that models cross-view redundancy as spatial connectivity and consolidates physically consistent regions, effectively merging extremely similar redundant tokens prior to LLM inference. We then propose a spatial clustering-based pruning (SCluP) paradigm within LLM inference, which performs hierarchical compression across clusters and within clusters, preserving object instance completeness while retaining fine-grained details for important regions. Extensive experiments on diverse 3D reasoning benchmarks show validate the effectiveness of HiSC, particularly under high visual token pruning ratios. Besides, HiSC achieves over 90% token reduction with minimal performance degradation. Code is accessible at https://github.com/elecreak/HiSC.

Abstract

Layer Skipping in Language Model Decoding

Explore similar work

HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding

Geometry-Guided 3D Visual Token Pruning for Video-Language Models

3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering