Organizations: RIKEN Center for Advanced Intelligence Project (AIP), RIKEN, Tokyo, Japan. · The University of Tokyo, Japan. · the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China.
Ultra-high-resolution (UHR) remote sensing visual question answering (VQA) requires locating fine-grained evidence in large scenes under limited visual input budgets. Active observation addresses this challenge through selective exploration, but repeated generative judgments can make region search costly. Its decisions also concern distinct prediction targets: spatial pointer validity, evidence presence, and reasoning correctness. We propose EarthJev, a framework that separates region scoring from answer generation and uses target-specific decision signals while keeping the answering model frozen. A spatial pointer verifier uses visual and geometric cues to guide exploration or bypass search. An evidence-aware decision mechanism scores compact candidate views in batches using a lightweight head on frozen representations, without generating textual region judgments. Format-specific confidence calibration then guides selection between candidate answers generated from memory and overview presentations of the acquired evidence. Across five benchmarks, EarthJev achieves a mean accuracy of 53.06% with Qwen3.5-9B, outperforming ZoomSearch with the same answering model by 2.33 to 14.73 percentage points per benchmark while reducing mean latency from 11.30 - 1.67 seconds per question. With four additional frozen answering models, model-specific fitting of the decision modules and calibration mappings yields mean accuracy gains of 3.41 - 6.53 percentage points over the corresponding single-pass baselines. Code is available at https://github.com/Da1daidaidai/EarthJev
Figures & tables
Fig. 1: Challenges in active observation: ambiguous spatial cues, misleading search history, and uncertain answers despite relevant evidence. Cyan marks target-relevant regions and evidence, red indicates distractors and off-target search, and yellow boxes delineate crop regions.
Fig. 2: Preliminary diagnostics of active observation: (a) Spatial cues: valid cues aid perception whereas wrong cues degrade it; (b) Evidence coverage: heuristic search (ZS) and Yes/No probing (YN) frequently miss target evidence; and (c) Answer scoring: answer confidence predicts answer correctness far better than evidence scores.
Fig. 3: Comparison of visual evidence acquisition paradigms. ZoomSearch employs crop-based search, while GazeEarth uses non-uniform resizing. Both may focus on incomplete or misleading evidence, resulting in incorrect answers in this example. In contrast, EarthJev verifies spatial cues, reassesses candidate views during search, and selects among candidate answers using calibrated confidence, enabling more accurate identification.
Fig. 4: The EarthJev workflow consists of three stages: (1) Spatial pointer verification, which integrates visual and geometric information to evaluate the localization cues provided in a question, determining whether the perspective presented by these cues is sufficient to answer the question directly or if further searching is required. (2) An EDM, which utilizes vision-language representations to score candidate regions, thereby guiding hierarchical expansion and the retention of evidence. (3) The selected evidence is organized into a “memory” and a “canvas”, which are then accessed by a shared, frozen answering model. Finally, the system evaluates candidate answers based on calibrated confidence scores and selects the most reliable one as the definitive answer.
Method
Backbone
LRS-VQA
MME-RS
LRS-GRO
RSHR-Bench
XLRS-lite
Overall
FAIR
Bridge
STAR
Avg.
Color
Count
Position
Avg.
Global
Region
Object
Avg.
Perception
Reasoning
Avg.
Avg.
Avg.
ZoomSearch [ 40 ]
Qwen3.5-9B
29.75
31.26
34.31
32.46
70.20
39.31
71.84
60.62
67.14
48.52
50.31
53.91
35.74
55.97
37.52
44.48
45.80
GazeEarth [ 41 ]
Qwen3-VL-8B
29.18
26.74
29.38
28.94
63.67
21.78
71.12
52.43
77.96
50.39
58.04
61.00
44.49
59.27
46.45
52.95
48.36
ViCrop [ 36 ]
Qwen3-VL-8B
27.33
29.57
30.51
29.39
46.85
16.97
47.02
37.11
75.56
51.48
54.03
58.58
34.58
61.67
36.05
43.51
40.93
AttWarp [ 38 ]
Qwen3-VL-8B
26.76
31.54
30.96
29.74
54.58
19.00
62.13
45.45
77.75
49.00
53.39
58.18
33.80
60.34
36.22
45.36
42.99
WeaveEarth [ 39 ]
Qwen3-VL-8B
27.64
24.95
34.78
31.15
54.02
23.49
65.71
47.94
75.34
47.74
56.50
58.94
32.74
63.76
36.14
41.36
43.10
TABLE I: Comparison of EarthJev with existing methods across five remote sensing VQA benchmarks. Results are reported as accuracy (%). The top three distinct scores per column are highlighted in coral, amber, and blue; best results are boldfaced, and ties share the same color. Benchmark Avg. denotes question-level accuracy, while Overall Avg. is the mean across five benchmarks. RSHR-Bench Perception/Reasoning scores are averaged over task categories.
TABLE II: Performance comparison of Earth with existing methods on XLRS-Bench-lite, covering 13 fine-grained Level-3 (L3) task categories. Training settings are indicated by ✓ (trained), ✓ ✓ ✓ (answering model frozen, with only lightweight decision modules trained), and ✗ (training-free).
Fig. 5: Comparison of overall accuracy across five remote sensing benchmarks.
Group
EDM
SPV
CAS
MME-RS
LRS-VQA
XLRS-lite
LRS-GRO
RSHR-Bench
Avg.
1
✗
✗
✗
60.62
32.46
44.48
53.91
37.52
45.80
2
✓
✗
✗
62.97
34.52
44.29
55.42
38.99
47.24
3
✓
✓
✗
62.81
35.20
55.00
56.04
54.16
52.64
4
✓
✓
✓
63.16
34.79
55.81
59.27
52.25
53.06
TABLE III: Component ablation of EarthJev with Qwen3.5-9B. Results are reported as accuracy (%). The full configuration is shaded in light coral; the best result in each column is boldfaced.
Fig. 6: Visual memory examples combining a global overview with local views indexed by their source locations. Memory and canvas answers are shown with calibrated confidence in parentheses.
Answerer
Setting
MME-RS
LRS-VQA
XLRS-lite
LRS-GRO
RSHR-Bench
Avg.
Qwen3.5-4B
Base
51.95
29.74
41.95
58.11
37.52
43.85
+ EarthJev
57.76
30.32
53.96
59.01
50.87
50.38
Gain (pp)
+5.81
+0.57
+12.01
+0.90
+13.34
+6.53
Qwen3-VL-8B
Base
55.97
32.27
46.10
59.27
36.05
45.93
+ EarthJev
56.39
34.97
54.38
59.40
49.48
50.92
Gain (pp)
+0.43
+2.70
+8.28
+0.13
+13.43
+4.99
TABLE IV: Performance of EarthJev across answering models (accuracy, %). Gray and coral cells show the base and EarthJev-augmented results, respectively; blue cells report gains in percentage points.
Selection
MME-RS
LRS-VQA
XLRS-lite
LRS-GRO
RSHR-Bench
Avg.
Memory only
62.92
33.15
55.00
58.53
54.16
52.75
Canvas only
56.18
35.20
53.64
56.04
45.84
49.38
Raw confidence
62.17
34.53
56.01
58.88
51.82
52.68
CAS (EarthJev)
63.16
34.79
55.81
59.27
52.25
53.06
TABLE V: Answer selection over the same retrieved evidence with Qwen3.5-9B (accuracy, %). Raw confidence and CAS choose between the memory and canvas answers by raw or calibrated confidence.
Fig. 7: Design evidence for the three modules (Qwen3.5-9B). (a) EDM vs. other decision sources: mean accuracy over the five benchmarks against seconds per question. (b) SPV vs. blind trust: accuracy change on the LRS-VQA pointer test with a correct or a wrong stated box, relative to no box. (c) Held-out calibration error of each module’s decision score (evidence presence, pointer trust, answer confidence) before and after Platt scaling.
Method
s/q
Rel. time
Avg.
Qwen3.5-9B
Direct
3.30
1.0 ×
46.93
ZoomSearch
11.30
3.4 ×
45.80
Zoom-Refine
11.34
3.4 ×
46.94
EarthJev
1.67
0.5 ×
53.06
Qwen3-VL-8B
TABLE VI: Inference cost under the same answering model. s/q: seconds per question on one NVIDIA H100 GPU
Fig. 8: Qualitative comparison on spatial reasoning and object counting. EarthJev selects the correct answer by evaluating calibrated confidence across memory and canvas presentations of the retrieved evidence.
Remote sensing visual question answering (VQA) studies how AI answers natural-language questions about satellite, aerial, and other Earth-observation imagery.
Ultra-high-resolution (UHR) remote sensing imagery provides rich spatial details but poses challenges for accurate and efficient multimodal understanding. Existing evidence acquisition methods selectively explore relevant image regions, yet largely rely on generative multimodal large language models (MLLMs) for intermediate search control. However, reinforcement learning from human feedback (RLHF) primarily aligns these models with human-preferred responses rather than explicitly optimizing region-level relevance judgments, while repeated autoregressive decoding introduces additional computational overhead. To address these limitations, we propose GeoJeV, a training-free method that formulates evidence acquisition as spatial probability-driven decision making. At each evaluated region, a frozen JeV model produces a probability distribution over candidate regions. Specifically, Region Rank prioritizes candidate regions, Sparse Branch adaptively determines branching width through sparse probability support, and Depth Gate controls refinement depth based on branch sparsity and distribution concentration. By coordinating search priority, branching width, and refinement depth through a shared distribution, GeoJeV reduces reliance on RLHF-aligned generation and eliminates intermediate decoding. Experiments on LRS-GRO, MME-RealWorld-RS, and LRS-VQA achieve accuracies of 60.59%, 52.79%, and 34.32%, respectively. Across the three benchmarks, GeoJeV improves the average accuracy of Qwen3-VL-8B by +5.32pp with only 1.12 additional seconds per question, while achieving a 7.8-fold inference speedup over the recent method.
Xinyi Niu, Lingfeng Chen, Jiehua Zhang +4
College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, Hunan, China.
Multimodal large language models (MLLMs) must balance local detail against scene context when interpreting ultra-high-resolution (UHR) remote sensing (RS) imagery within a limited visual-input budget. Existing selection-based methods either prune tokens and select patches through relevance scoring, or crop actively through repeated inspection. Neither strategy directly redistributes pixels within a continuous full-scene view: the first retains selected tokens or patches, and the second re-encodes a crop detached from its surroundings. Our pilot study finds that a frozen MLLM already produces useful question-guided spatial requests, yet crop-based inspection of the selected regions does not consistently improve its answers. We therefore formulate UHR understanding as a question of where to spend a fixed pixel budget. Based on this, we introduce GazeEarth, a simple-yet-effective training-free framework that couples question-guided region selection with full-scene foveated observation. The MLLM selects evidence cells from an indexed overview; a deterministic, topology-preserving warp resamples the original image onto a fixed-size canvas, enlarging their shared neighborhood while compressing the periphery; the same frozen model answers from this focused view, using at most two MLLM calls and no external selector or iterative search. Across three UHR remote sensing benchmarks and four frozen backbones, GazeEarth improves benchmark-averaged accuracy by 4.6 to 9.4 percentage points over direct answering and 3.4 to 4.3 over overview answering, outperforming task-trained methods. Our analyses show that existing MLLMs can guide where to look in UHR images on their own, and that what they can infer from the selected evidence depends on how that evidence is presented.
Ultra-high-resolution (UHR) remote sensing visual question answering (VQA) requires models to resolve small visual evidence within extremely large images. Existing approaches typically rely on token pruning, visual search, or tool-augmented reasoning at inference time. We instead investigate whether the benefit of zoom-in visual privilege can be internalized into the model. We introduce RS-OPSD, a reliable privileged on-policy self-distillation (OPSD) framework for UHR remote sensing VQA. To provide high-quality privileged information with explicit question-relevant evidence, we construct GeoEvidence-6K, containing 6,750 VQA samples across seven task categories with evidence-region annotations, and develop Human Feedback-Guided Skill Refinement (HF-SR) for scalable annotation. To address context loss from tight crops and conflicting signals from imperfect teachers, RS-OPSD introduces Context-Preserving Visual Privilege (CPVP) and Correctness-Aligned Distillation (CAD). Without any additional visual search or tool calls at inference time, RS-OPSD achieves state-of-the-art (SOTA) performance on XLRS-Bench, MME-RealWorld-RS, and LRS-VQA, outperforming previous SOTA models of comparable scale by an average of 4.0 percentage points. Moreover, our 2B variant, RS-OPD-Lite, surpasses most 8B-scale models while achieving the fastest measured inference speed. Our Code, GeoEvidence-6K, and the model weights for RS-OPSD and RS-OPD-Lite are publicly available.
Chengjie Jiang, Yunqi Zhou, Jiafeng Yan +3
Tsinghua University · Zhejiang University · Central University of Finance and Economics +2