Benchmarking Reasoning Reliability,Prior_Evidence Contrast,and Spectral Discrimination in Remote-Sensing Geological Image Interpretation
Organizations: School of Computer Science and Artificial Intelligence, China University of Geosciences · PRADA Lab, King Abdullah University of Science and Technology · Department of Computer Science, University of Exeter · School of Mathematics and Computer Science, Wuhan Polytechnic University ·
Abstract
Reliable lithology interpretation requires accurate geological recognition and reasoning grounded in visual evidence. Although Multimodal Large Language Models show promise in remote sensing, their reliability in geological interpretation remains unclear. We introduce HyGeoBench, a diagnostic benchmark of 10,000 questions based on hyperspectral imagery across three dimensions: reasoning reliability, prior_evidence contrast, and spectral discrimination. It covers ten task types, with both multiple-choice and open-ended response formats. Integrating hyperspectral satellite imagery with geological annotations and verified evidence, HyGeoBench examines reasoning errors, conflicts between geological priors and visual evidence, and cross-band inconsistencies. Evaluations reveal limitations in fine-grained recognition and evidence-grounded interpretation. To address these challenges, we propose HyGeoAgent, a tool-augmented framework integrating region observation, text knowledge retrieval, same-band image retrieval, and native PCA analysis through adaptive tool use. Experiments demonstrate consistent improvements over the corresponding base models, improving geological image interpretation across diagnostic dimensions.
Figures & tables
| R (nm) | G (nm) | B (nm) | Reference |
| Single-sensor samples | |||
| 1642 | 1600 | 1196.5 | WV-3 SWIR 3–2–1 [ 30 ] |
| 1757.5 | 1600 | 1196.5 | SWIR spectral basis [ 49 , 2 ] |
| 2147 | 1757.5 | 1600 | HI SWIR band set [ 21 ] |
| 2213.5 | 2147 | 1600 | ASTER 6–5–4 (SWIR) [ 50 ] |
| 2265 | 2213.5 | 2147 | ASTER 7–6–5 (DCS) [ 18 ] |
| Model | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | MCQ | Open | RQ1 | RQ2 | RQ3 | OA | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MCQ | Open | MCQ | Open | MCQ | Open | MCQ | Open | MCQ | Open | MCQ | Open | MCQ | Open | MCQ | Open | MCQ | Open | MCQ | Open | |||||||
| Only Text as Input | ||||||||||||||||||||||||||
| DeepSeek V4.1 Flash [ 14 ] | 54.60 | 26.37 | 44.75 | 17.18 | 48.32 | 3.35 | 57.56 | 3.23 | 37.82 | 7.37 | 91.63 | 87.09 | 48.80 | 4.35 | 45.80 | 4.15 | 45.42 | 3.67 | 41.86 | 6.03 | 52.30 | 17.31 | 9.19 | 22.61 | 37.15 | 31.31 |
| Open-source MLLMs | ||||||||||||||||||||||||||
| EarthDial-4B-RGB [ 62 ] | 28.10 | 8.58 | 26.82 | 8.34 | 28.85 | 1.66 | 33.61 | 0.89 | 26.96 | 3.18 | 59.09 | 18.84 | 32.38 | 1.37 | 25.06 | 1.65 | 24.81 | 1.66 | 23.13 | 1.29 | 31.51 | 5.11 | 4.24 | 22.12 | 11.56 | 15.67 |
| SkyEyeGPT [ 81 ] | 25.24 | 1.26 | 20.52 | 0.92 | 25.98 | 1.27 | 28.36 | 0.76 | 29.20 | 1.53 | 39.71 | 16.78 | 23.60 | 0.78 | 21.50 | 1.19 | 20.48 | 0.85 | 23.26 | 0.89 | 26.27 | 2.71 | 2.33 | 14.74 | 6.50 | 12.13 |
| Configuration | MCQ | Open | Average Score |
|---|---|---|---|
| Full Agent | 62.50 | 43.42 | 51.05 |
| Original Model | 55.00 | 39.75 | 45.85 |
| w/o Text Retrieval | 52.50 | 43.04 | 46.83 |
| w/o Reference Image Retrieval | 57.50 | 41.38 | 47.83 |
| w/o PCA Tool | 60.00 | 40.29 | 48.18 |
| w/o Local Image Observation | 55.00 | 40.75 | 46.45 |
Remote Sensing Image Understanding
Remote sensing image understanding uses AI to interpret satellite and aerial imagery for objects, land cover, change, hazards, and spatial relationships.
See all 217 papers