SA-GEM: Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning for Efficient Remote Sensing Large Vision-Language Models

Kexin Ma; Jing Xiao; Bowen Xing; Liang Liao; Chia-Wen Lin

cs.CVAug 15, 2026

SA-GEM: Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning for Efficient Remote Sensing Large Vision-Language Models

Authors: Kexin Ma, Jing Xiao, Bowen Xing, Liang Liao, Chia-Wen Lin

Organizations: School of Artificial Intelligence, Wuhan University · Hangzhou Institute of Technology, Xidian University · Department of Electrical Engineering, National Tsing Hua University

Abstract

RS-LVLMs have advanced multimodal understanding of Earth observation imagery, yet their performance is fundamentally constrained by high-resolution processing, as visual token counts grow quadratically with linear input resolution while important visual evidence is inherently sparse and increasingly diluted across the expanded sequence. Existing token pruning methods largely rely on scale-agnostic resolution policies and isolated importance cues, limiting task-aligned granularity adaptation and holistic evidence preservation. To address this, we present Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning (SA-GEM), a plug-and-play framework that unifies task-adaptive token granularity allocation with holistic geospatial token importance modulation. Specifically, a lightweight router selects the resolution based on query-dependent token granularity, while a token importance modulator jointly models task relevance, spatial structure, and local redundancy to preserve holistic geospatial evidence. We show that higher resolution is not universally beneficial and, once sufficient granularity is reached, token quality matters more than token quantity. Experiments across various benchmarks demonstrate that SA-GEM achieves consistent gains in both accuracy and efficiency over existing pruning methods. On XLRS-Bench, it surpasses GeoLLaVA-8K by 2.3% in accuracy with a 2.4 times total inference speedup.

Figures & tables

Figure 1: Overview of task-specific semantic granularity and token-pruning paradigms for efficient RS-LVLMs . Existing methods emphasize fine-grained perception at high or ultra-high resolutions and rely on single-criterion importance estimates, whereas ours adapts resolution to task-specific granularity and modulates token importance with complementary evidence.

Figure 2: Overview of SA-GEM. A lightweight router predicts the required visual granularity and selects the corresponding input resolution. Wavelet cues, feature affinity, and decoupled attention model spatial structure, local redundancy, and task relevance, respectively. Their fusion yields holistic token-importance scores for top - $k$ selection before LLM decoding.

Method	Size	Avg Tok.	Perception								Reasoning					Overall
Sub-tasks (L-3 Capability)			OC	RC	OLUC	RLUC	OCC	OCL	OMS	OSR	AD	ECR	RP	RCCD	CCR	Avg.
Closed-source MLLMs
GPT-4o	-	-	25.0	32.0	15.0	66.0	9.5	11.3	11.7	24.6	73.0	73.0	35.0	20.0	25.0	32.4
Claude 3.7 Sonnet	-	-	27.6	22.7	17.4	68.4	30.5	29.9	63.6	27.6	64.8	78.4	34.5	27.8	32.6	40.5
GPT-4o-mini	-	-	23.3	25.0	19.0	59.5	40.9	31.0	65.0	23.6	71.0	71.0	29.0	6.7	30.0	38.1
Open-source MLLMs

Table 1: Results on XLRS-Bench. ‘Avg.’ denotes average sub-task accuracy. Best and second-best results are bolded and underlined . “ pX ” denotes at most X predefined grids. Full sub-task names are provided in the appendix.

Method	Max Size	Color	Count.	Pos.	Avg.
GPT-4o	–	29.83	18.90	33.52	27.42
LLaVA-1.5-7B	336 $\times$ 336	22.95	16.31	21.48	20.28
LLaVA-Next-7B	672 $\times$ 672	24.06	20.47	26.49	23.70
LLaVA-OV-7B	2,304 $\times$ 2,304	26.14	27.57	26.81	26.83
Qwen2.5-VL-7B	3,584 $\times$ 3,584	15.54	14.93	22.12	17.55
GeoChat	504 $\times$ 504	23.11	15.66	25.06	21.28

Table 2: Results on MMERealWorld-RS. We report accuracy on Count, Color, Position, and their mean (Avg.).

Method	Max Size	FAIR	Bridge	STAR	Avg.
GPT-4o	–	22.15	31.84	27.40	27.13
GPT-4o-mini	–	18.67	31.99	25.85	25.50
Claude-3.5-Sonnet	–	12.95	26.69	13.29	17.64
LLaVA-OV-7B	2,304 $\times$ 2,304	20.61	35.11	26.08	27.27
LLaVA-1.5-7B	336 $\times$ 336	18.76	30.70	22.63	24.03
GeoChat	504 $\times$ 504	20.18	24.54	13.75	19.49

Table 3: Results on LRS-VQA. We report accuracy for the FAIR, Bridge, and STAR subsets and their mean (Avg.).

Token Pruning Rate = 75.0% $\downarrow$
Task	Structure	Redundancy	Color	Count	Pos.	Acc.
$\times$	$\times$	$\times$	41.56	31.05	46.14	39.58
$\checkmark$	$\times$	$\times$	40.32	33.71	45.65	39.89
$\times$	$\checkmark$	$\times$	40.16	32.97	48.23	40.45
$\times$	$\times$	$\checkmark$	39.73	32.89	48.07	40.23
$\checkmark$	$\times$	$\checkmark$	40.60	33.63	46.10	40.11

Table 4: Ablation study of geospatial evidence guidance under a 75% pruning ratio on MMERealWorld-RS.

Figure 3: Visualization analysis on MMERealWorld-RS comparing GeoLLaVA- 8K , UHR-BAT, and SA-GEM (Ours). The original image and the corresponding pruning guidance with pruned results at pruning ratios of 75% and 87.5% are shown. Gray areas indicate discarded tokens, while red regions highlight the primary important area.

Figure 4: Ablations on wavelet settings and retained ratios. (left) Wavelet configurations across input resolutions. Red boxes and bold values denote the selected best-performing configurations. (right) Sensitivity to visual-token retention ratios, with circled markers indicating the selected ratios.

Method	V-Tokens $\downarrow$	$\boldsymbol{T_{\mathrm{total}}}\downarrow$	$\boldsymbol{T_{\mathrm{enc.}}}\downarrow$	TFLOPs $\downarrow$	Acc. $\uparrow$
GeoLLaVA- 8K	13.8k (24.0 $\times$ )	3.13	2.19	418.5	51.5
+ VisionZip	13.8k (24.0 $\times$ )	3.32	2.61	419.1	42.8
+ SparseVLM	13.8k (24.0 $\times$ )	3.09	2.38	417.0	41.2
UHR-BAT	11.1k (29.8 $\times$ )	2.70	2.13	359.9	48.6
SA-GEM (Ours)	6.8k (49.3 $\times$ )	1.29	0.76	164.0	53.8

Table 5: Efficiency comparisons on the XLRS-Bench. We report averaged visual token volume, measured total inference time (s), encoding time (s), total TFLOPs, and accuracy.

Remote Sensing VLMs

See all 41 papers

Hierarchical and Query-Aware Visual Token Pruning for High-Resolution LVLMs
Oct 16, 2026Luther Wei, Ling Zou, Kang-Yang Huang +2Visual Token Pruning Efficient VLM Inference
Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan · MediaTek Inc., Hsinchu, Taiwan
S $^2$ Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models
Sep 1, 2026Yuanyuan Jia, Shunpu Tang, Qianqian YangEfficient VLM Inference Multimodal Large Language Models
College of Information Science and Electronic Engineering, Zhejiang University Hangzhou, China
OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
May 28, 2026Geng Li, Guohao Chen, Ting Chen +6Efficient VLM Inference Large Vision-Language Models
1Nanyang Technological University (NTU)

Oct 16, 2026J/K move · Enter open · S save

Hierarchical and Query-Aware Visual Token Pruning for High-Resolution LVLMs

Luther Wei, Ling Zou, Kang-Yang Huang, Yu-Syuan Xu+1

Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan · MediaTek Inc., Hsinchu, Taiwan

Visual Token Pruning Efficient VLM Inference LVLM Pruning

Open paper AskPDF arXiv

High-resolution Large Vision-Language Models (LVLMs) preserve critical, fine-grained visual details by encoding a global thumbnail alongside multiple local crops. However, the resulting thousands of vision tokens introduce severe computational bottlenecks during Large Language Model (LLM) inference. Existing token pruning methods rely primarily on query-agnostic visual saliency, while current query-aware approaches either operate at a coarse crop-level resolution or compute token relevance too late, after vision tokens have already entered the heavy LLM layers. To address this, we propose a training-free, hierarchical token pruning framework that utilizes query guidance prior to executing any LLM blocks. Our top-down hierarchy first operates at the instance-level, dynamically scaling the vision token retention ratio of each sample based on its visual complexity. Next, at the crop-level, we distribute this sample budget across local crops using sentence-level cross-modal similarity. Finally, at the token-level, we exploit projected vision tokens alongside textual query token embeddings in the shared space to measure fine-grained alignment, applying a text-debiasing normalization to produce high-fidelity query scores. These query-guided budgets and scores are fused with local and global visual saliency metrics to select the most critical vision tokens. Extensive experiments on LLaVA-NeXT-7B demonstrate that our plug-and-play framework retains 98.0% and 94.8% of unpruned performance at 25% and 10% vision token retention, respectively. Notably, at 25% retention, our method delivers a 2.0x inference throughput speedup and even surpasses the unpruned baseline on POPE by 0.7 points.

Abstract

Remote Sensing VLMs

Explore similar work

Hierarchical and Query-Aware Visual Token Pruning for High-Resolution LVLMs

S2^22Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models

OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning

S $^2$ Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models