Observe Less, Understand More: Cost-aware Cross-scale Observation for Remote Sensing Understanding

Zhenghao Xie; Jing Xiao; Zhenqi Wang; Kexin Ma; Liang Liao; Gui-song Xia; Mi Wang

cs.CVApr 13, 2026

Observe Less, Understand More: Cost-aware Cross-scale Observation for Remote Sensing Understanding

Authors: Zhenghao Xie, Jing Xiao, Zhenqi Wang, Kexin Ma, Liang Liao, Gui-song Xia, Mi Wang

Organizations: School of Artificial Intelligence, Wuhan University, Wuhan, China · Hangzhou Institute of Technology, Xidian University, Hangzhou, China · Hubei Luojia Laboratory, Wuhan University, Wuhan, China

Abstract

Remote sensing understanding inherently requires multi-resolution observation, since different targets and application tasks demand different levels of spatial detail. While low-resolution (LR) imagery enables efficient global observation, high-resolution (HR) imagery provides critical local details at a much higher acquisition cost and with limited coverage. This motivates a cross-scale sensing strategy that selectively acquires HR imagery guided by LR-based global perception to improve task performance under constrained cost. Existing HR sampling methods typically make selection decisions from isolated LR patches, thereby ignoring fine-grained intra-patch importance and cross-patch contextual interactions, leading to fragmented feature representation and suboptimal scene reasoning under sparse HR observations. To address this issue, we formulate cross-scale remote sensing understanding as a unified cost-aware problem that couples fine-grained HR sampling with cross-patch representation prediction, enabling more effective task reasoning with fewer HR observations. Furthermore, we present GL-10M, a high- and low-resolution dataset with nearly 100,000 scene pairs and 10 million images for large-scale cross-resolution pretraining. Extensive experiments on recognition and retrieval tasks show that our method consistently achieves a superior performance-cost trade-off. The code is publicly available at https://github.com/xzhacc/CrossSO.

Figures & tables

Panel of Figure 1 — Figure 1. Cost-aware cross-scale understanding with sparse HR observations. (a) Dense HR preserves details at high cost, whereas sparse HR leaves incomplete scene evidence. (b) From an LR overview, the cost-aware sampler selects HR tiles, and the observation-guided latent predictor completes the representation for recognition and retrieval. (c) Relative to full HR, our method reduces the observed HR area by 86% on average across both tasks while maintaining comparable or better performance. A three-panel overview contrasts dense and sparse high-resolution observation, illustrates the proposed low-resolution-guided selection and completion framework, and summarizes its performance and cost advantages.

Figure 2. Cost-aware sampling and latent completion for cross-scale understanding. Given an LR overview $I^{g}$ , the sampler selects informative HR tiles. The observation-guided latent predictor combines the selected HR evidence with global LR context to produce a completed representation $\tilde{H}$ . Supervised tasks use dense visual targets. For zero-shot tasks, $\tilde{H}$ can be aligned directly with text features or with visual features already aligned to the same text space. The lower panels show the sampler and predictor in detail. The framework first selects high-resolution regions from a low-resolution global overview, then completes a scene representation from sparse local evidence and global context, and finally aligns it with task-specific target features for supervised and zero-shot tasks.

Method	Input	Seen Concepts		Unseen Concepts		HR Usage (OBR) $\downarrow$	Time (s) $\downarrow$
		mAP@100 $\uparrow$	mAP@20 $\uparrow$	mAP@100 $\uparrow$	mAP@20 $\uparrow$
Full observation
GRAFT (HR) ( Mall et al., 2024 ; Revankar et al., 2025 )	HR	0.501	0.513	0.541	0.574	1.000	962
GRAFT (LR+LLM) ( Mall et al., 2024 ; Revankar et al., 2025 )	LR+HR	0.530	0.561	0.549	0.573	1.000	984
Ours-Full	LR+HR	0.589	0.602	0.539	0.592	1.000	968
LR only

Table 1. Main comparison of retrieval-based recognition for concepts seen and unseen during training on Sentinel-2/NAIP. Results include retrieval performance, HR observation usage, and inference efficiency. N.L. denotes nightlight sampling; bold and underlined values mark the best and second-best results per setting, respectively; light blue rows mark our methods.

Method	Input	mAP@100 $\uparrow$	mAP@20 $\uparrow$	OBR $\downarrow$
Full observation
Scale-aware (HR) ( Revankar et al., 2025 )	HR	0.695	0.735	1.000
Scale-aware (LR+LLM) ( Revankar et al., 2025 )	LR+HR	0.689	0.731	1.000
Ours w/o LR	HR	0.718	0.769	1.000
Ours-Full	LR+HR	0.722	0.767	1.000
LR only

Table 2. Fully supervised retrieval-based recognition on Sentinel-2/NAIP under controlled HR observation settings.

Method	EuroSAT	BigEarthNet	GL-10M
	Acc. $\uparrow$	mAP $\uparrow$	mAP ${}^{\ast}\uparrow$
CLIP ( Radford et al., 2021 )	0.536	0.231	0.401
GRAFT ( Mall et al., 2024 )	0.638	0.325	0.441
RemoteCLIP ( Liu et al., 2024a )	0.386	0.228	0.362
Ours	0.652	0.346	0.487

Table 3. Zero-shot recognition performance on EuroSAT ( Helber et al., 2019 ) , BigEarthNet ( Sumbul et al., 2019 ) , and GL-10M, where

\ast

denotes support-weighted mAP for the long-tailed class distribution.

Figure 3. Performance–cost trade-off, ablations, and retrieval robustness. (a)

\mathbf{G}^{*}

sampling provides the offline-supervision reference; patch-level supervision remains closer to it than image-level supervision. (b) Budget ablations isolate module effects. (c) Gains persist across depths at

\mathrm{OBR}=0.12

. Three plots compare the cost–performance frontier, matched high-resolution observation budgets, and retrieval performance across depths at a fixed observation budget.

Settings		Metrics
HR Sampling	Fusion	mAP@100 $\uparrow$	OBR $\downarrow$
LR only	None	0.468	0.000
Random	Predictor	0.495	0.123
$s_{\mathrm{gain}}$ only	Predictor	0.480	0.123
$s_{\mathrm{vis}}$ only	Predictor	0.502	0.123
$s_{\mathrm{vis}}\odot s_{\mathrm{gain}}$	Max ( Revankar et al., 2025 )	0.475	0.123

Table 4. Ablation of HR sampling and fusion on the unseen-concept split, reporting mAP@100 and OBR for each setting.

Figure 4. Qualitative comparison of cost-aware sampling and latent completion under sparse HR observation. The second column shows predicted sampling maps, while the latent feature visualizations share the same PCA basis and color scale. Using LR context produces more coherent scene-level predictions closer to the full-HR target. Three rows compare low-resolution inputs, predicted sampling maps, sparse high-resolution observations, latent completion variants, and full-resolution targets. Low-resolution context produces more coherent predictions.

Property	Description
LR imagery	Sentinel-2, 10 m spatial resolution
HR imagery	NAIP, 1 m spatial resolution
Pair structure	One LR overview and 100 ordered HR tiles on a $10\times 10$ grid
Semantic source	OSM-derived mapping to 66 geographic concepts
Geographic coverage	11 U.S. states with geographically separated data partitions
Dataset scale	Nearly 100,000 scene pairs and approximately 10 million images

Table 5. Summary of the construction and released structure of GL-10M.

Panel of Figure 5 — Figure 5. Retrieval-based diagnostic of latent completion. The left column shows target HR tiles; the remaining groups show the top-three neighbors retrieved by features completed without LR context, with LR context, and with GL-10M pretraining plus LR context. Red boxes mark representative local mismatches. Five target high-resolution tiles and their top-three nearest-neighbor retrievals under completion without low-resolution context, with low-resolution context, and with both GL-10M pretraining and low-resolution context. Red boxes mark visually inconsistent regions in selected retrievals.

Figure 6. Qualitative threshold sweep under controlled HR acquisition. The threshold increases from 0.00 (full HR observation) to 1.00 (no HR observation); 0.55 is the shared operating point used in the main experiments. Five representative scenes under thresholds from 0.00 to 1.00. Each scene shows the retained high-resolution tiles and the corresponding completed latent visualization as the observation budget decreases.

Panel of Figure 7 — Figure 7. Representative failure cases. (a) missed informative HR regions, (b) drift under sparse evidence, and (c) complex scene structures. Three examples containing an input scene, a latent feature visualization, sparse high-resolution evidence, and the resulting completion. Red dashed boxes mark representative regions associated with each failure pattern.

Remote Sensing Image Understanding

Remote sensing image understanding uses AI to interpret satellite and aerial imagery for objects, land cover, change, hazards, and spatial relationships.

See all 208 papers

More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe
Jul 17, 2026Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi +2Remote Sensing Image Understanding Remote Sensing VQA
INSAIT, Sofia University "St. Kliment Ohridski"
Cross-Scale Sparse Attention for Referring Segmentation in Remote Sensing
Oct 24, 2026Yuxi Li, Lu Si, Xia Liao +4Referring Image Segmentation Remote Sensing Image Segmentation
Westlake Institute for Advanced Study, Hangzhou, Zhejiang, China · School of Engineering, Westlake University, Hangzhou, Zhejiang, China · Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, China +2
The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understanding
Sep 23, 2026Yao Zhang, Pengyu Dai, Wei Guo +5Remote Sensing Image Understanding Remote Sensing VLMs
Wuhan University

Jul 17, 2026J/K move · Enter open · S save

More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe

Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi, Luc Van Gool+1

INSAIT, Sofia University "St. Kliment Ohridski"

Remote Sensing Image Understanding Remote Sensing VQA Remote Sensing VLMs Large Vision-Language Models

Open paper AskPDF arXiv

Remote sensing vision-language models are increasingly expected to support open-ended reasoning over Earth Observation data and a variety of tasks. Most recent progress in this area has been driven by remote-sensing-specific architectural designs, often introducing new encoders, alignment modules, or task-specific fusion mechanisms. In this work, we challenge the necessity of such architectural specialization. We show that a generally capable vision-language model can achieve competitive or state-of-the-art performance at challenging remote sensing benchmarks, provided that it is trained at sufficient scale across diverse data and tasks. Our model uses a single language policy that can either answer directly in text or invoke a localization tool for segmentation and grounding. To train this heterogeneous behaviour, we employ a multi-task reinforcement learning framework with adaptive task rewards covering multiple-choice VQA, free-form VQA, captioning, detection, and segmentation across a large variety of input types. Our approach achieves competitive results across a broad set of benchmarks, including high-resolution, multi-temporal, multi-modal and multi-view tasks. Further, as training data scales, our experiments show consistent improvements across most tasks both in and out of distribution, which correlate with per-task data diversity. These findings suggest that, for remote sensing VLMs, data scale is sufficient even without architectural novelty.

Abstract

Remote Sensing Image Understanding

Explore similar work

More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe

Cross-Scale Sparse Attention for Referring Segmentation in Remote Sensing

The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understanding