cs.CVApr 10, 2026
SaveGeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensing
Organizations: RIKEN AIP · Wuhan University · Linköping University · Nanjing University of Information Science and Technology · University of Tokyo
Abstract
Recent advances in multimodal large language models (MLLMs) have accelerated progress in domain-oriented AI, yet their development in geoscience and remote sensing (RS) remains constrained by distinctive challenges: wide-ranging disciplinary knowledge, heterogeneous sensor modalities, and a fragmented spectrum of tasks. To bridge these gaps, we introduce GeoMMBench, a comprehensive multimodal question-answering benchmark covering diverse RS disciplines, sensors, and tasks, enabling broader and more rigorous evaluation than prior benchmarks. Using GeoMMBench, we assess 36 open-source and proprietary large language models, uncovering systematic deficiencies in domain knowledge, perceptual grounding, and reasoning--capabilities essential for expert-level geospatial interpretation. Beyond evaluation, we propose GeoMMAgent, a multi-agent framework that strategically integrates retrieval, perception, and reasoning through domain-specific RS models and tools. Extensive experimental results demonstrate that GeoMMAgent significantly outperforms standalone LLMs, underscoring the importance of tool-augmented agents for dynamically tackling complex geoscience and RS challenges.
Figures & tables
Figure 1: Expert-level knowledge dimensions in geoscience and RS covered in GeoMMBench.
Figure 2: Examples from GeoMMBench, covering multiple disciplines, diverse sensor modalities, and a wide range of task types. Answering these questions requires advanced geoscience and RS expertise, along with specialized reasoning and interpretation skills.
Figure 3: Overview of GeoMMAgent, a multi-agent framework that plans, executes, and self-evaluates multimodal tasks for expert-level geospatial understanding.
Figure 4: Representative error cases from advanced MLLMs specific to geospatial tasks. : GPT-4o; : Gemini-1.5 pro.
Figure A1 : Performance of MLLMs on different types of images in GeoMMExpert.
Figure A2 : A sample error case of remote sensing principle understanding. Subfield: Spectral curve interpretation.
Figure A3 : A sample error case of remote sensing principle understanding. Subfield: Spectral curve interpretation.
Figure A4 : A sample error case in remote sensing principles. Subfield: Atmospheric window recognition.
Figure A5 : A sample error case in remote sensing principles. Subfield: Remote sensing process.
Figure A6 : A sample error case in remote sensing platforms. Subfield: Remote sensing satellite identification.
Figure A7 : A sample error case in remote sensing principles. Subfield: Resolution type recognition.
Figure A8 : A sample error case in time series analysis. Subfield: Environmental monitoring.
Figure A9 : A sample error case in time series analysis. Subfield: Pollution analysis.
Figure A10 : A sample error case in remote sensing principles. Subfield: LiDAR platforms.
Figure A11 : A sample error case in remote sensing perception. Subfield: SAR imagery perception.
Figure A12 : A sample error case in hyperspectral theoretical understanding. Subfield: Spectral index understanding.
Figure A13 : A sample error case in GIS interpretation. Subfield: GIS map recognition.
Figure A14 : A sample error case in data quality examination. Subfield: Image quality inspection.
Figure A15 : A sample error case in spatial relation analysis. Subfield: Geospatial location identification.
Figure A16 : A sample error case in remote sensing image recognition. Subfield: GSD recognition.
| Agent Role | System Prompt Content |
|---|---|
| Coordinate Agent | You are an Intelligent Orchestration Expert in the field of Remote Sensing that analyzes images and queries, creating execution plans through the coordination of multiple specialized agents. You provide a detailed description of the inputs and deliver professional task decomposition informed by a meticulous review of the agents’ and toolkits’ documentation. |
| Perception Agent | You are a Specialized Perception Expert in the field of Remote Sensing, responsible for extracting reliable visual evidence from multi-sensor imagery. You perform scene classification, object detection, and semantic segmentation, and provide calibrated predictions for downstream reasoning. |
| Knowledge Agent | You are a Geospatial Knowledge Retrieval Expert in the field of Remote Sensing, specialized in querying external knowledge bases and web resources. You retrieve, filter, and summarize factual information about remote sensing, geophysics, and geographic entities. |
| Reasoning Agent | You are an Expert Reasoning Agent in the field of Remote Sensing, specialized in multimodal geospatial reasoning. You integrate visual features, retrieved knowledge, and task context to perform step-by-step analysis and produce logically consistent answers. |
| Self-Evaluation Agent | You are a Professional Assessment Expert in the field of Remote Sensing, specialized in evaluating the correctness of image analysis results. You assess logic, consistency, completeness. |
Table A1 : Representative system prompts assigned to different agents within the GeoMMAgent framework.
| Tool | System Prompt Content |
|---|---|
| General Toolkit | |
| Format conversion | Use this tool to convert inputs between different image and geospatial data formats so that downstream models and tools can directly consume the data. |
| Patch tiling and merging | Use this tool to split large images into tiles for efficient processing and then merge tile-level predictions back into a spatially consistent full-scene result. |
| Filtering | Use this tool to denoise, smooth, or sharpen imagery in order to improve data quality before perception or reasoning. |
| Cropping | Use this tool to crop user-specified or automatically selected regions of interest, removing irrelevant areas and reducing computational cost. |
| Scaling | Use this tool to resize imagery to the resolution or aspect ratio required by subsequent models, supporting both upsampling and downsampling. |
Table A2: Representative system prompts assigned to different tools within the GeoMMAgent framework.
Figure A17 : The multimodal query for our case study. The question is: ”Which of the following statements about Band ① is accurate?” The options are: (A) It is suitable for thermal remote sensing, (B) It is used in passive remote sensing, (C) It operates only during the day and is affected by weather conditions, and (D) It can penetrate vegetation to detect subsurface targets. The answer is D.
Figure A18 : The multimodal query for aircraft counting. The question is: ”How many aircraft are there in the image?” The options are: (A) 13, (B) 10, (C) 12, (D) 9. The answer is C.
| val | test | |
|---|---|---|
| w/o Knowledge | 83.8 | 87.4 |
| w/o Perception | 83.8 | 80.3 |
| w/o Reasoning | 59.5 | 67.3 |
| w/o Self-evaluation | 81.1 | 80.1 |
| full GeoMMAgent | 86.5 | 88.4 |
Table A3 : Component-wise ablation study of GeoMMAgent on the val and test sets.