cs.CVApr 15, 2026
SaveUHR-BAT: Budget-Aware Token Compression Vision-Language model for Ultra-High-Resolution Remote Sensing
Organizations: School of Artificial Intelligence Science and Technology, Nanjing University · School of Electronic Science and Engineering, Nanjing University · School of Physics, Nanjing University
Abstract
Ultra-high-resolution (UHR) remote sensing imagery couples kilometer-scale context with query-critical evidence that may occupy only a few pixels. Such vast spatial scale leads to a quadratic explosion of visual tokens and hinders the extraction of information from small objects. Previous works utilize direct downsampling, dense tiling, or global top-k pruning, which either compromise query-critical image details or incur unpredictable compute. In this paper, we propose UHR-BAT, a query-guided and region-faithful token compression framework to efficiently select visual tokens under a strict context budget. Specifically, we leverage text-guided, multi-scale importance estimation for visual tokens, effectively tackling the challenge of achieving precise yet low-cost feature extraction. Furthermore, by introducing region-wise preserve and merge strategies, we mitigate visual token redundancy, further driving down the computational budget. Experimental results show that UHR-BAT achieves state-of-the-art performance across various benchmarks. Code will be available at https://github.com/Yunkaidang/UHR.
Figures & tables
Figure 1: Accuracy–efficiency frontier for Ultra-High-Resolution Remote Sensing MLLMs on XLRS-Bench benchmark. We report overall weighted average accuracy versus the number of visual tokens. The results show that our method achieves superior performance using fewer tokens.
Figure 2: Overview of our method. We encode a high-resolution remote sensing image at multiple scales using a frozen ViT, applying Scale-Specific Positional Embeddings to distinguish visual tokens across scales, and then obtain an anchor-scale query-to-vision attention map from the MLLM interface. A region partition (e.g., induced by SAM or feature+coordinate clustering) enables region-wise preserve-and-merge, while remaining tokens are merged via average pooling to retain coarse context. A final top- step enforces the per-scale token budget. Finally, the processed sequence is fed into the LLM to generate the final answer.
| Method | Size | Perception | Reasoning | Overall | |||||||||||
| Sub-tasks (L-3 Capability) | OC | RC | OLUC | RLUC | OCC | OCL | OMS | OSR | AD | ECR | RP | RCCD | CCR | w.Avg. | |
| number of samples | 60 | 100 | 100 | 200 | 800 | 800 | 60 | 500 | 100 | 100 | 100 | 60 | 100 | 3080 | |
| Open-source MLLMs | |||||||||||||||
| InternLM-XComposer-2.5 ( Zhang et al., 2024a ) | 7B | 21.7 | 42.0 | 7.0 | 68.0 | 31.8 | 27.8 | 6.7 | 26.0 | 72.0 | 81.0 | 41.0 | 36.7 | 47.0 | 34.8 |
| LLaVA-Next ( Liu et al., 2024 ) | 7B | 26.7 | 40.0 | 5.0 | 67.0 | 28.8 | 32.8 | 66.7 | 30.0 | 69.0 | 78.0 | 27.0 | 35.0 | 36.0 | 36.0 |
| LLaVA-OneVision-7B ( Li et al., 2024a ) | 7B | 25.0 | 38.0 | 2.0 | 69.5 | 35.9 | 35.3 | 65.0 | 25.2 | 76.0 | 83.0 | 24.0 | 43.3 | 36.0 | 38.1 |
Table 1: Results on XLRS-Bench across perception and reasoning sub-tasks. ‘w.Avg.’ indicates the sample-count-weighted average. The best results are bolded , and the second-best results are underlined . Full sub-task names are provided in the appendix.
| Model | Size | Perception | Reasoning | Average | ||||||||||||||
| COL | SHP | DET | OC | REL | OGD | RG | OCN | RCN | AD | FP | MRJC | MRJCS | OSJ | P.Avg. | R.Avg. | Avg. | ||
| Open-source VLMs | ||||||||||||||||||
| InternVL2.5-8B ( Chen et al., 2024b ) | 8B | 25.5 | 22.0 | 26.0 | 26.0 | 22.5 | 24.5 | 30.0 | 22.5 | 20.0 | 26.0 | 20.0 | 22.5 | 34.0 | 20.0 | 24.3 | 24.5 | 24.4 |
| InternVL3.5-8B ( Wang et al., 2025c ) | 8B | 21.5 | 28.0 | 18.0 | 21.5 | 29.0 | 28.5 | 30.0 | 26.5 | 25.0 | 20.0 | 16.0 | 29.0 | 34.0 | 26.0 | 25.3 | 25.0 | 25.2 |
| MiniCPM2_6 ( Yao et al., 2024 ) | 7B | 21.5 | 28.0 | 30.0 | 24.0 | 19.5 | 29.5 | 34.3 | 22.0 | 29.0 | 26.0 | 30.0 | 35.0 | 32.0 | 30.0 | 27.4 | 30.6 | 28.5 |
| Phi-3.5-Vision ( Abdin et al., 2024 ) | 7B | 25.0 | 24.0 | 25.0 | 25.0 | 23.5 | 25.0 | 22.9 | 25.0 | 25.0 | 24.0 | 22.0 | 23.5 | 30.0 | 22.0 | 24.5 | 24.3 | 24.4 |
Table 2: RSHR-Bench results across Perception and Reasoning tasks. We report accuracy (%) for each subtask. ‘P.Avg.’ and ‘R.Avg.’ denote the mean accuracy over subtasks within Perception and Reasoning, respectively. ‘All Avg.’ denotes the mean accuracy over all Perception and Reasoning subtasks. Subtask abbreviations are defined in the appendix.
| Method | Params | Position | Color | Count | Avg. |
| GPT-4o ( Hurst et al., 2024 ) | – | 33.52 | 29.83 | 18.90 | 27.42 |
| LLaVA-v1.5-7B ( Liu et al., 2023 ) | 7B | 21.48 | 22.95 | 16.31 | 20.28 |
| LLaVA-v1.6-7B ( Liu et al., 2024 ) | 7B | 26.49 | 24.06 | 20.47 | 23.70 |
| LLaVA-ov-7B ( Li et al., 2024a ) | 7B | 26.81 | 26.14 | 27.57 | 26.83 |
| Qwen2.5-VL-7B ( Bai et al., 2025 ) | 7B | 22.12 | 15.54 | 14.93 | 17.55 |
| LLaVA-HR ( Luo et al., 2024a ) | 7B | 35.56 | 44.30 | 7.91 | 29.26 |
Table 3: Results on the MMERealworld-RS dataset, with methods ranked by Avg. We report accuracy (%) on Position, Color, and Count, and their mean (Avg.) for all methods. ‘Params’ denotes the number of model parameters.
Figure 3: Ablation on clustering centers and retained tokens. Left: accuracy versus the number of cluster centers . Right: accuracy versus the number of retained visual tokens.
| Compression ( ) | Kept Tokens | TFLOPs | Avg. Total (s) | Avg. Gen. (s) |
| 32.83 | 4k | 267.68 | 20.93 | 4.34 |
| 26.27 | 5k | 274.91 | 21.58 | 4.56 |
| 21.89 | 6k | 289.10 | 21.83 | 4.74 |
| 16.42 | 8k | 317.48 | 21.99 | 5.11 |
| 13.13 | 10k | 345.75 | 25.50 | 5.65 |
| 11.94 | 11k | 359.94 | 25.79 | 5.86 |
Table 4: Efficiency analysis: Latency and FLOPs per sample under different token budgets (Total input tokens = 131,328).
Figure 4: Multi-scale token selection accuracy on XLRS-Bench. We evaluate four input resolutions (672 672, 1344 1344, 2688 2688, 4032 4032) and vary the number of retained visual tokens at each scale (x-axis).
Figure 5: Qualitative comparisons of our method and GeoLLaVA-8K under 95% pruning ratio.
| Kept Tokens | Image Resolution | Perception Mean | Reasoning Mean | w.Avg. |
| 3k | 8k | 40.38 | 57.83 | 43.00 |
| 4k | 8k | 40.61 | 57.52 | 43.12 |
| 3k | 4k | 40.50 | 58.04 | 43.12 |
| 4k | 4k | 40.73 | 57.83 | 43.28 |
Table 5: Performance analysis on XLRS-Bench under varying resolutions and token budgets.
| Resolution | Other scales | Perception | Reasoning | w.Avg. | |
| 672 672 | 320,600,2400 | 0 | 40.76 | 57.83 | 43.31 |
| 672 672 | 320,600,2400 | 40 | 40.88 | 58.04 | 43.44 |
| 672 672 | 320,600,2400 | 80 | 40.95 | 57.83 | 43.47 |
| 672 672 | 320,600,2400 | 120 | 41.15 | 58.04 | 43.67 |
| 1344 1344 | 120,600,2400 | 80 | 40.69 | 57.61 | 43.21 |
| 1344 1344 | 120,600,2400 | 160 | 40.88 | 57.17 | 43.33 |
Table 6: Performance on XLRS-Bench across multiple resolutions ( , , and ) under varying scale-specific token budgets .
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | 10 | 50 | 120 | 240 | 360 | 480 | 600 | 720 | 1000 |
| Perception | 37.2 | 37.6 | 37.7 | 37.3 | 37.4 | 37.3 | 37.5 | 37.1 | 37.2 |
| Reasoning | 62.4 | 64.1 | 63.5 | 62.4 | 63.3 | 63.7 | 63.9 | 63.3 | 63.0 |
| Average | 40.9 | 41.6 | 41.6 | 41.1 | 41.3 | 41.2 | 41.4 | 41.0 | 41.1 |
Table 7: Ablation study: Accuracy (%) across Perception and Reasoning tasks under different cluster counts ( ).
Figure 6: Qualitative examples of coarse segmentation on remote-sensing images produced by our segmentation module, used to form semantically coherent regions for region-aware processing.
Figure 7: t-SNE visualization of visual tokens. Each color represents a cluster assigned by our region-based grouping module. Although the high-dimensional nature of tokens leads to some spatial overlap in the 2D projection, tokens with the same semantic labels exhibit clear clustering behavior, validating the redundancy of visual information and the feasibility of our token merging strategy.
Figure 8: Qualitative visualization of region-aware attention on remote-sensing images. We display original images and enlarged views alongside their corresponding attention maps. The heatmaps demonstrate that the model accurately focuses on semantically relevant regions—such as specific vehicles or infrastructure—aligned with the textual query, while assigning lower importance to the background.
Figure 9: Visualization of attention-guided token pruning. We display the original image and the retained visual tokens at increasing pruning ratios. The method effectively focuses on high-attention areas, preserving query-relevant details (red boxes) like specific colors or text, while discarding the non-informative background.
Figure 10: Visualization of clustering-guided token pruning. This example demonstrates how region-based partitioning helps retain structural objects. Even at a 97.5% pruning ratio, small targets like boats or roundabouts are preserved within their clusters, ensuring the model retains sufficient context for spatial reasoning.
Figure 11: Qualitative comparison of model responses across InternVL3.5-8B, Qwen2.5-VL-72B, and our method. The examples demonstrate that our model correctly handles fine grained tasks such as counting dense oil tanks or identifying specific land types.
Figure 12: Qualitative comparison of model responses. Our method consistently provides accurate answers for challenging counting tasks within localized regions, effectively distinguishing targets like swimming pools or houses from complex backgrounds.
Figure 13: Qualitative comparison of model responses. The visualization highlights the ability of our model to ground the correct visual evidence for fine grained queries, specifically including color identification of small objects and reasoning about spatial directions.
Figure 14: Qualitative examples of responses generated by our model. These samples illustrate the versatility of our method in handling diverse tasks, including spatial reasoning about road orientation and fine-grained attribute recognition of small objects.
Figure 15: Qualitative examples of responses generated by our model. The visualization demonstrates the ability of the model to interpret geometric shapes and complex spatial relationships, such as identifying the curvature of a bridge or determining the relative position.
Figure 16: Qualitative examples of responses generated by our model. These results show that our model effectively captures dynamic scene details, such as assessing water conditions, identifying construction activities, detecting minute objects, and recognizing colors.
More in this topic
Visual Token Compression
Visual token compression reduces image and video representations to lower inference, memory, storage, or communication costs while retaining task-relevant information.
See all 261 papers