Authors: Yueying Li, Fengxiang Wang, Yan Li, Mingshuo Chen, Mengying Zhao, Long Lan
Organizations: College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China
Multimodal Large Language Models (MLLMs) have demonstrated immense potential in Earth observation. However, the massive visual tokens generated when processing Ultra-High-Resolution (UHR) imagery introduce prohibitive computational overhead, severely bottlenecking their inference efficiency. Existing visual token compression methods predominantly adopt static and uniform compression strategies, neglecting the inherent "Semantic-Geometric Duality" in remote sensing interpretation tasks. Specifically, object semantic tasks focus on the abstract semantics of objects and benefit from aggressive background pruning, whereas scene geometric tasks critically rely on the integrity of spatial topology. To address this challenge, we propose DualComp, a task-adaptive dual-stream token compression framework. Dynamically guided by a lightweight pre-trained router, DualComp decouples feature processing into two dedicated pathways. In the object semantic stream, the Spatially-Contiguous Semantic Aggregator (SCSA) utilizes size-adaptive clustering to aggregates redundant background while protecting small object. In the scene geometric stream, the Instruction-Guided Structure Recoverer (IGSR) introduces a greedy path-tracing topology completion mechanism to reconstruct spatial skeletons. Experiments on the UHR remote sensing benchmark XLRS-Bench demonstrate that DualComp accomplishes high-fidelity remote sensing interpretation at an exceptionally low computational cost, achieving simultaneous improvements in both efficiency and accuracy.
Figures & tables
Figure 1: Pilot study reveals semantic–geometric duality under token compression. We evaluate a UHR remote-sensing MLLM across multiple compression ratios and report performance changes for three representative groups of subtasks: (a) semantic-dominant improves as compression increases; (b) balanced peaks at a moderate ratio; (c) geometric-dominant drops under strong compression. This highlights that background tokens can be either redundant noise or indispensable geometric evidence, depending on task intent.
Figure 2: DualComp achieves performance improvements under high compression ratios through semantic–geometric dual-stream adaptive compression. We reveal a pervasive semantic–geometric duality in UHR remote sensing MLLMs.
Figure 3: Overview of DualComp. Text-conditioned routing outputs (λ,ρ) , steering SCSA (semantic) and IGSR (geometric) token reduction and fusion.
Figure 4: Two compression modules in DualComp. (a) SCSA aggregates redundant background while preserving semantic cues. (b) IGSR recovers topological cues via instruction-guided completion.
Method
Compression
Perception
Reasoning
Avg.
Sub-tasks (L-3 Capability)
OC
RC
OSR
OLUC
RLUC
OCC
OCL
OMS
AD
ECR
RP
RCCD
CCR
Closed-source MLLMs
Claude 3.7 Sonnet [ 3 ]
-
27.6
22.7
27.6
17.4
68.4
30.5
29.9
63.6
64.8
78.4
34.5
27.8
32.6
40.5
Gemini 2.5 Pro [ 12 ]
-
-
-
-
-
-
-
-
-
-
-
-
-
-
45.2
GPT-5.2 [ 27 ]
-
30.0
37.0
34.0
17.0
70.5
43.0
41.4
68.3
74.0
76.0
52.0
36.7
38.0
47.5
Open-source MLLMs
Table 1: Experimental results on the perception and reasoning dimensions on XLRS-Bench. The sub-tasks include Perception : Overall/Regional Counting (OC/RC), Overall/Regional Land Use Classification (OLUC/RLUC), Object Classification (OCC), Object Color (OCL), Object Motion State (OMS), and Object Spatial Relationship (OSR); and Reasoning : Anomaly Detection and Interpretation (AD), Environmental Condition Reasoning (ECR), Route Planning (RP), Regional Counting with Change Detection (RCCD), and Counting with Complex Reasoning (CCR). ‘Avg.’ represents the macro average accuracy across sub-tasks. Rankings are calculated column-wise: green (1st, bold), light green (2nd), and yellow (3rd). Tie scores are treated identically.
Metrics
GeoLLaVA-8K
w/ VisionZip
w/ FastV
w/ SparseVLMs
w/ Ours
Compression Ratio
24×
24×
24×
24×
42.4×
Tokens per Grid
24
24
24
24
14.2
Avg. Visual Token Volume
13.8k
13.8k
13.8k
13.8k
6.4k
TFLOPs of LLM
198.1
198.7
198.1
186.1
99.8
Inference Speed (s/image)
8.15
8.41
19.56
7.84
3.87
Visual Encoding + Compression
4.28
5.08
8.59
4.72
1.52
Table 2: Comparison of computational efficiency with different token reduction methods based on GeoLLaVA-8K.
Method
Perception
Reasoning
Avg.
Sub-tasks
OC
RC
OSR
OLUC
RLUC
OCC
OCL
OMS
AD
ECR
RP
RCCD
CCR
SCSA-only
25.0
44.0
36.0
50.0
69.0
40.0
26.5
65.0
62.0
77.0
67.0
45.0
47.0
50.3
IGSR-only
25.0
42.0
34.8
50.0
68.0
43.4
28.6
65.0
65.0
78.0
71.0
45.0
48.0
51.1
Top-K
25.0
41.0
34.4
52.0
65.0
38.9
31.3
65.0
65.0
76.0
65.0
45.0
50.0
50.5
TASM-off
25.0
44.0
35.8
48.0
66.5
42.8
31.8
65.0
67.0
77.0
71.0
45.0
47.0
51.2
Index-Reorder
25.0
42.0
35.4
51.0
66.0
43.9
32.1
65.0
67.0
77.0
72.0
45.0
48.0
51.5
Table 3: Ablation results of DualComp on XLRS-Bench. We compare the full model with variants that remove or modify key components, including SCSA-only (w/o geometric stream), IGSR-only (w/o semantic stream), Top-K (w/o topology), TASM-off (w/o text-aware modulation), and Index-Reorder (w/o topological unrolling). The subtask abbreviations follow Table 1 .
Method
Compression
Perception
Reasoning
Avg.
Sub-tasks
OC
RC
OSR
OLUC
RLUC
OCC
OCL
OMS
AD
ECR
RP
RCCD
CCR
Qwen2.5-VL-7B
−
33.3
40.0
36.2
31.0
77.0
40.6
40.5
66.7
68.0
72.0
27.0
38.3
45.0
47.4
+ VisionZip
1.43×
35.0
38.0
36.8
33.0
78.5
39.0
40.6
66.7
68.0
73.0
25.0
38.3
46.0
47.5
+ VisionZip
10×
36.7
42.0
34.0
33.0
78.5
38.6
40.3
66.7
60.0
72.0
24.0
38.3
46.0
46.9
+ Ours
10.24×
38.3
36.0
35.4
36.0
72.5
41.9
40.6
66.7
71.0
74.0
29.0
38.3
43.0
47.9
Table 4: Performance comparison on XLRS-Bench based on Qwen2.5-VL-7B.
Visual token compression reduces image and video representations to lower inference, memory, storage, or communication costs while retaining task-relevant information.
Ultra-high-resolution (UHR) remote sensing imagery couples kilometer-scale context with query-critical evidence that may occupy only a few pixels. Such vast spatial scale leads to a quadratic explosion of visual tokens and hinders the extraction of information from small objects. Previous works utilize direct downsampling, dense tiling, or global top-k pruning, which either compromise query-critical image details or incur unpredictable compute. In this paper, we propose UHR-BAT, a query-guided and region-faithful token compression framework to efficiently select visual tokens under a strict context budget. Specifically, we leverage text-guided, multi-scale importance estimation for visual tokens, effectively tackling the challenge of achieving precise yet low-cost feature extraction. Furthermore, by introducing region-wise preserve and merge strategies, we mitigate visual token redundancy, further driving down the computational budget. Experimental results show that UHR-BAT achieves state-of-the-art performance across various benchmarks. Code will be available at https://github.com/Yunkaidang/UHR.
Yunkai Dang, Minxin Dai, Yuekun Yang +4
School of Artificial Intelligence Science and Technology, Nanjing University · School of Electronic Science and Engineering, Nanjing University · School of Physics, Nanjing University
RS-LVLMs have advanced multimodal understanding of Earth observation imagery, yet their performance is fundamentally constrained by high-resolution processing, as visual token counts grow quadratically with linear input resolution while important visual evidence is inherently sparse and increasingly diluted across the expanded sequence. Existing token pruning methods largely rely on scale-agnostic resolution policies and isolated importance cues, limiting task-aligned granularity adaptation and holistic evidence preservation. To address this, we present Scale-Adaptive and Geospatial Evidence-Modulated Token Pruning (SA-GEM), a plug-and-play framework that unifies task-adaptive token granularity allocation with holistic geospatial token importance modulation. Specifically, a lightweight router selects the resolution based on query-dependent token granularity, while a token importance modulator jointly models task relevance, spatial structure, and local redundancy to preserve holistic geospatial evidence. We show that higher resolution is not universally beneficial and, once sufficient granularity is reached, token quality matters more than token quantity. Experiments across various benchmarks demonstrate that SA-GEM achieves consistent gains in both accuracy and efficiency over existing pruning methods. On XLRS-Bench, it surpasses GeoLLaVA-8K by 2.3% in accuracy with a 2.4 times total inference speedup.
Kexin Ma, Jing Xiao, Bowen Xing +2
School of Artificial Intelligence, Wuhan University · Hangzhou Institute of Technology, Xidian University · Department of Electrical Engineering, National Tsing Hua University
Current visual tokenizers in Multimodal Large Language Models (MLLMs) predominantly rely on patch-based partitioning, which causes severe semantic mixture and object fragmentation in remote sensing imagery due to the irregular contours of geo-objects. Moreover, existing adaptive methods struggle to extract precise object-level tokens and lack dedicated geometric positional encodings for irregular regions. In this paper, we propose HeatTok, a semantic-aware tokenizer driven by thermodiffusion aggregation. Inspired by the physical principles of heat conduction, HeatTok adaptively merges adjacent homogeneous regions to generate semantically independent, object-aligned irregular tokens. To enable MLLMs to perceive these irregular shapes, we design the Gaussian Multimodal Rotary Positional Embedding (G-MRoPE), which models token spatial distributions via 2D Gaussians and explicitly injects center, scale, and orientation cues. Extensive evaluations on the VRSBench and EarthVQA datasets demonstrate that HeatTok effectively preserves object-level semantic integrity and achieves state-of-the-art performance under a reasonable token budget. The code is available: https://github.com/YingyingYan1/HeatTok.
Yingying Yan, Jiaqi Tang, Wei Wei +6
Hong Kong University of Science and Technology Hong Kong, China