The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understanding
Organizations: Wuhan University
Abstract
Multimodal large language models (MLLMs) must balance local detail against scene context when interpreting ultra-high-resolution (UHR) remote sensing (RS) imagery within a limited visual-input budget. Existing selection-based methods either prune tokens and select patches through relevance scoring, or crop actively through repeated inspection. Neither strategy directly redistributes pixels within a continuous full-scene view: the first retains selected tokens or patches, and the second re-encodes a crop detached from its surroundings. Our pilot study finds that a frozen MLLM already produces useful question-guided spatial requests, yet crop-based inspection of the selected regions does not consistently improve its answers. We therefore formulate UHR understanding as a question of where to spend a fixed pixel budget. Based on this, we introduce GazeEarth, a simple-yet-effective training-free framework that couples question-guided region selection with full-scene foveated observation. The MLLM selects evidence cells from an indexed overview; a deterministic, topology-preserving warp resamples the original image onto a fixed-size canvas, enlarging their shared neighborhood while compressing the periphery; the same frozen model answers from this focused view, using at most two MLLM calls and no external selector or iterative search. Across three UHR remote sensing benchmarks and four frozen backbones, GazeEarth improves benchmark-averaged accuracy by 4.6 to 9.4 percentage points over direct answering and 3.4 to 4.3 over overview answering, outperforming task-trained methods. Our analyses show that existing MLLMs can guide where to look in UHR images on their own, and that what they can infer from the selected evidence depends on how that evidence is presented.
Figures & tables
| Method | Param. | TF | LRS-GRO (%) | MME-RealWorld-RS (%) | ||||||
| Global | Region | Object | Avg. | Color | Count | Position | Avg. | |||
| Commercial MLLMs | ||||||||||
| GPT-4o ( OpenAI, 2024 ) | – | ✗ | 31.10 | 12.37 | 21.86 | 21.83 | 34.18 | 15.17 | 37.07 | 28.92 |
| Gemini-2.5-Flash ( Gemini Team, 2025 ) | – | ✗ | 57.95 | 23.39 | 30.76 | 35.52 | 55.86 | 33.28 | 51.63 | 47.03 |
| General-purpose MLLMs | ||||||||||
| LLaVA-OV ( Li et al., 2025a ) | 7B | ✗ | 62.43 | 36.18 | 39.92 | 44.42 | 26.81 | 26.14 | 27.57 | 26.85 |
| Method | Param. | TF | XLRS-Bench (%) | ||||||||
| Cnt | SC | OSR | OP | Plan | AR | CR | SR | Avg. | |||
| Commercial MLLMs | |||||||||||
| GPT-4o ( OpenAI, 2024 ) | – | ✗ | 32.50 | 47.67 | 35.20 | 37.47 | 36.00 | 72.00 | 58.50 | 31.67 | 40.16 |
| Gemini-2.5-Flash ( Gemini Team, 2025 ) | – | ✗ | 38.75 | 51.67 | 30.40 | 39.10 | 42.00 | 78.00 | 65.50 | 23.33 | 41.66 |
| General-purpose MLLMs | |||||||||||
| LLaVA-OV ( Li et al., 2025a ) | 7B | ✗ | 31.25 | 35.67 | 25.20 | 45.42 | 24.00 | 76.00 | 59.50 | 43.33 | 41.62 |
| Variant | Gaze | Q-guided gaze | Detail | Full scene | Continuous Binding | Performance (%) | |||
| XLRS- Bench | LRS- GRO | MME- RealWorld-RS | Avg. | ||||||
| Direct | 0 | ✗ | ✗ | ✓ | – | 42.92 | 56.90 | 42.11 | 47.31 |
| Overview | 0 | ✗ | ✗ | ✓ | – | 45.13 | 59.99 | 49.01 | 51.38 |
| Gaze + Crop | 1 | ✓ | ✓ | ✗ | ✗ | 50.55 | 58.93 | 39.81 | 49.76 |
| Think with Image | 1 | ✓ | ✓ | ✓ | ✗ | 51.43 | 60.54 | 42.24 | 51.40 |
| Linked-Global + Crop | 1 | ✓ | ✓ | ✓ | ✓ | 52.56 | 60.25 | 43.39 | 52.07 |
Appendix figures & tables36 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Qwen3-VL 8B | LLaVA-v1.6 Mistral-7B | Intern-S1 mini | Commercial MLLMs API |
| Execution interface | Local | Local | Local | OpenRouter |
| Vision patch size | Not disclosed | |||
| Vision input unit | Variable-size image | tile | tile | PNG/JPEG data URL |
| Resolution handling | Dynamic resolution | AnyRes tiling | Dynamic tiling | Provider-managed |
| Default processor limit | 16,777,216 pixels | 4 local tiles | 12 local tiles | Not disclosed |
| Additional global view | – | 1 | 1 when tiled | Not disclosed |
| Benchmark | Questions | Answer format | Reporting categories |
| LRS-GRO | 9,734 | Open-ended | Global, Region, Object |
| MME-RealWorld-RS | 3,738 | Multiple-choice | Color, Count, Position |
| XLRS-Bench | 3,080 | Multiple-choice | Cnt, SC, OSR, OP, Plan, AR, CR, SR |
| Observer | XLRS-Bench (%) | LRS-GRO (%) | MME-RealWorld-RS (%) | Avg. (%) |
| Random Focus | 14.65 | 21.83 | 12.56 | 16.35 |
| Question-only MLLM | 41.50 | 44.56 | 42.40 | 42.82 |
| SigLIP2 | 39.49 | 54.18 | 35.59 | 43.09 |
| RemoteCLIP | 33.44 | 45.82 | 32.33 | 37.20 |
| Frozen MLLM | 73.89 | 77.60 | 81.74 | 77.74 |
| Observer | Center hit (%) | Box coverage (%) | Answer accuracy (%) |
| SigLIP2 | 52.33 | 47.56 | 64.43 |
| RemoteCLIP | 37.99 | 34.33 | 62.94 |
| Self-MLLM | 64.99 | 55.65 | 66.29 |
| Oracle Gaze | – | – | 70.20 |
| Observer | Align. (%) | Grid Dist. | Acc.@ Aligned (%) | Acc.@ Off-target (%) | Spatial Acc. (%) |
| Random Focus | 14.65 | 1.951 | 39.13 | 38.99 | 39.01 |
| SigLIP2 | 39.49 | 1.326 | 40.32 | 37.37 | 38.54 |
| RemoteCLIP | 33.44 | 1.435 | 41.90 | 35.89 | 37.90 |
| Frozen MLLM | 73.89 | 0.408 | 44.40 | 37.20 | 42.52 |
| Observer | Align. (%) | Grid Dist. | Acc.@ Aligned (%) | Acc.@ Off-target (%) | Spatial Acc. (%) |
| Random Focus | 12.56 | 2.170 | 41.98 | 36.72 | 37.38 |
| SigLIP2 | 35.59 | 1.548 | 49.27 | 36.22 | 40.87 |
| RemoteCLIP | 32.33 | 1.599 | 48.64 | 36.16 | 40.20 |
| Frozen MLLM | 81.74 | 0.280 | 49.43 | 38.81 | 47.49 |
| Observer | Align. (%) | Grid Dist. | Acc.@ Aligned (%) | Acc.@ Off-target (%) | Spatial Acc. (%) |
| Random Focus | 21.83 | 1.516 | 53.63 | 53.07 | 53.19 |
| SigLIP2 | 54.18 | 0.825 | 57.26 | 51.39 | 54.57 |
| RemoteCLIP | 45.82 | 0.996 | 55.86 | 52.36 | 53.96 |
| Frozen MLLM | 77.60 | 0.314 | 57.65 | 50.00 | 55.93 |
| Variant | Gaze | Q-guided gaze | Detail | Full scene | Binding | XLRS-Bench (%) | ||||||||
| Cnt | SC | OSR | OP | Plan | AR | CR | SR | Avg. | ||||||
| Visual Evidence Composition | ||||||||||||||
| Direct | 0 | ✗ | ✗ | ✓ | – | 32.50 | 41.67 | 31.40 | 43.92 | 34.00 | 70.00 | 63.50 | 46.33 | 42.92 |
| Overview | 0 | ✗ | ✗ | ✓ | – | 35.00 | 60.67 | 28.80 | 44.52 | 42.00 | 71.00 | 64.50 | 45.00 | 45.13 |
| Random Focus | 1 | ✗ | ✓ | ✓ | ✓implicit | 35.00 | 61.33 | 30.60 | 44.10 | 39.00 | 78.00 | 65.50 | 43.33 | 45.42 |
| Gaze + Crop | 1 | ✓ | ✓ | ✗ | ✗ | 41.88 | 66.33 | 29.20 | 52.29 | 41.00 | 72.00 | 69.50 | 41.67 | 50.55 |
| Variant | Gaze | Q-guided gaze | Detail | Full scene | Binding | LRS-GRO (%) | |||
| Global | Region | Object | Avg. | ||||||
| Visual Evidence Composition | |||||||||
| Direct | 0 | ✗ | ✗ | ✓ | – | 76.25 | 42.03 | 54.77 | 56.90 |
| Overview | 0 | ✗ | ✗ | ✓ | – | 77.88 | 50.22 | 56.22 | 59.99 |
| Random Focus | 1 | ✗ | ✓ | ✓ | ✓implicit | 77.84 | 48.78 | 54.95 | 58.97 |
| Gaze + Crop | 1 | ✓ | ✓ | ✗ | ✗ | 74.66 | 47.39 | 56.95 | 58.93 |
| Variant | Gaze | Q-guided gaze | Detail | Full scene | Binding | MME-RealWorld-RS (%) | |||
| Color | Count | Position | Avg. | ||||||
| Visual Evidence Composition | |||||||||
| Direct | 0 | ✗ | ✗ | ✓ | – | 48.07 | 21.35 | 56.41 | 42.11 |
| Overview | 0 | ✗ | ✗ | ✓ | – | 55.54 | 18.35 | 72.39 | 49.01 |
| Random Focus | 1 | ✗ | ✓ | ✓ | ✓implicit | 52.43 | 16.80 | 66.98 | 45.64 |
| Gaze + Crop | 1 | ✓ | ✓ | ✗ | ✗ | 62.47 | 22.10 | 34.45 | 39.81 |
| Observer | Color | Count | Position | Avg. | vs. Random |
| Random | 52.43 | 16.80 | 66.98 | 45.64 | 0.00 |
| SigLIP2 | 56.10 | 19.33 | 66.35 | 47.49 | +1.85 |
| RemoteCLIP | 55.46 | 18.11 | 66.98 | 47.08 | +1.44 |
| GazeEarth | 63.67 | 21.78 | 71.12 | 52.43 | +6.79 |
| Observer | Global | Region | Object | Avg. | vs. Random |
| Random | 77.84 | 48.78 | 54.95 | 58.97 | – |
| SigLIP2 | 77.58 | 49.52 | 55.83 | 59.54 | +0.58 |
| RemoteCLIP | 77.75 | 50.78 | 56.24 | 60.10 | +1.13 |
| GazeEarth | 77.96 | 50.39 | 58.04 | 61.00 | +2.03 |
| Observer | Cnt | SC | OSR | OP | Plan | AR | CR | SR | Avg. | vs. Random |
| Random | 35.00 | 61.33 | 30.60 | 44.10 | 39.00 | 78.00 | 65.50 | 43.33 | 45.42 | 0.00 |
| SigLIP2 | 36.25 | 62.67 | 31.40 | 45.12 | 39.00 | 73.00 | 65.50 | 41.67 | 46.10 | +0.68 |
| RemoteCLIP | 37.50 | 61.67 | 29.80 | 45.12 | 38.00 | 77.00 | 65.50 | 43.33 | 45.94 | +0.52 |
| GazeEarth | 44.38 | 66.67 | 37.80 | 53.07 | 49.00 | 76.00 | 74.50 | 55.00 | 53.51 | +8.09 |
| Benchmark | Comparison | 95% CI | ||
| XLRS-Bench | Overview Direct | |||
| GazeEarth Direct | ||||
| GazeEarth Overview | ||||
| LRS-GRO | Overview Direct | |||
| GazeEarth Direct | ||||
| GazeEarth Overview |
| Variant | XLRS- Bench (s) | LRS- GRO (s) | MME- RealWorld- RS (s) | Avg. (s) | Speedup |
| Visual Evidence Composition | |||||
| Direct | 8.051 | 6.417 | 7.563 | 7.344 | – |
| Overview | 4.096 | 3.967 | 3.998 | 4.020 | 1.827 |
| Random Focus | 4.687 | 5.124 | 5.019 | 4.943 | 1.486 |
| Gaze + Crop | 3.945 | 4.689 | 4.661 | 4.432 | 1.657 |
| Think with Image | 4.690 | 5.323 | 5.088 | 5.034 | 1.459 |
| Variant | MLLM calls | Selection (s) | Visual construction (s) | Answer (s) | Total (s) | Speedup vs. Direct |
| Visual Evidence Composition | ||||||
| Direct | 1 | – | – | 7.563 | 7.563 | – |
| Overview | 1 | – | 0.147 | 3.851 | 3.998 | 1.892 |
| Random Focus | 2 | 0.889 | 0.325 | 3.805 | 5.019 | 1.507 |
| Gaze + Crop | 2 | 0.905 | 0.176 | 3.580 | 4.661 | 1.623 |
| Think with Image | 2 | 0.884 | 0.252 | 3.952 | 5.088 | 1.486 |
| Variant | MLLM calls | Selection (s) | Visual construction (s) | Answer (s) | Total (s) | Speedup vs. Direct |
| Visual Evidence Composition | ||||||
| Direct | 1 | – | – | 6.417 | 6.417 | – |
| Overview | 1 | – | 0.169 | 3.798 | 3.967 | 1.618 |
| Random Focus | 2 | 1.118 | 0.212 | 3.794 | 5.124 | 1.252 |
| Gaze + Crop | 2 | 1.141 | 0.060 | 3.488 | 4.689 | 1.369 |
| Think with Image | 2 | 1.155 | 0.192 | 3.976 | 5.323 | 1.206 |
| Variant | MLLM calls | Selection (s) | Visual construction (s) | Answer (s) | Total (s) | Speedup vs. Direct |
| Visual Evidence Composition | ||||||
| Direct | 1 | – | – | 8.051 | 8.051 | – |
| Overview | 1 | – | 0.331 | 3.765 | 4.096 | 1.966 |
| Random Focus | 2 | 0.459 | 0.481 | 3.747 | 4.687 | 1.718 |
| Gaze + Crop | 2 | 0.465 | 0.193 | 3.287 | 3.945 | 2.041 |
| Unlinked Global + Crop | 2 | 0.474 | 0.422 | 3.794 | 4.690 | 1.717 |
| Setting | Color | Count | Position | Avg. |
| Grid-size Sensitivity | ||||
| Grid | 61.75 | 20.88 | 67.78 | 50.37 |
| Grid | 63.67 | 21.78 | 71.12 | 52.43 |
| Grid | 62.15 | 21.78 | 69.29 | 51.31 |
| Region-budget Sensitivity | ||||
| 63.75 | 22.02 | 70.56 | 52.35 | |
| Setting | Global | Region | Object | Avg. |
| Grid-size Sensitivity | ||||
| Grid | 77.23 | 50.44 | 56.99 | 60.28 |
| Grid | 77.96 | 50.39 | 58.04 | 61.00 |
| Grid | 77.45 | 49.78 | 56.85 | 60.11 |
| Region-budget Sensitivity | ||||
| 77.84 | 50.74 | 56.69 | 60.35 | |
| Setting | Cnt | SC | OSR | OP | Plan | AR | CR | SR | Avg. |
| Grid-size Sensitivity | |||||||||
| Grid | 43.13 | 67.67 | 34.00 | 50.42 | 45.00 | 77.00 | 70.50 | 46.67 | 50.97 |
| Grid | 44.38 | 66.67 | 37.80 | 53.07 | 49.00 | 76.00 | 74.50 | 55.00 | 53.51 |
| Grid | 41.25 | 65.33 | 34.20 | 49.88 | 47.00 | 78.00 | 70.50 | 46.67 | 50.49 |
| Region-budget Sensitivity | |||||||||
| 40.63 | 67.00 | 32.80 | 50.06 | 45.00 | 79.00 | 71.00 | 48.33 | 50.52 | |
Remote Sensing Image Understanding
Remote sensing image understanding uses AI to interpret satellite and aerial imagery for objects, land cover, change, hazards, and spatial relationships.
See all 207 papers