DisasterAgent: An Evidence-Driven Agent for Verifiable Disaster Report Generation From Remote Sensing Imagery
Authors: Zhijian Li, Jiawei Song, Chao Ren, Yidan Zhang, Peijin Wang, Xian Sun, Kun Fu
Organizations: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China · the School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China · the University of Chinese Academy of Sciences, Beijing 100190, China · the National Key Laboratory of Target Cognition and Application Technology (TCAT), Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China
Remote sensing disaster assessment is moving from simple damage classification toward comprehensive report-level analysis. In this setting, responders need not only damage descriptions but also verifiable evidence to guide response and recovery. Recent vision-language models (VLMs) and tool-augmented agents offer strong multimodal reasoning and perception, but they struggle to provide the evidence required for high-stakes verification. To address these limitations, we propose DisasterAgent, an evidence-driven agent that uses a Perception-Augmentation-Generation workflow to generate verifiable disaster reports from bitemporal optical imagery. DisasterAgent first invokes a specialized vision expert toolkit in the perception phase to build a Spatiotemporal Perception Graph (STPG), serving as a structured memory to store and organize disaster-induced changes across space and time. Evidence-based Augmentation (EA) then extracts evidence from the STPG, attaching inline citations that form a closed evidence loop from each statement to the supporting visual evidence. Extensive experiments on the DisasterM3 and RSCC benchmarks demonstrate that DisasterAgent consistently outperforms general-domain VLMs, remote sensing-specialized models, and existing remote sensing agents. Notably, compared with the original model, it improves the average score for disaster report generation on DisasterM3 by 13.5% and ST5-SCS on RSCC by 15.34%, providing a more accurate and verifiable solution for disaster emergency response. The code will be available at https://github.com/ZhijianLi2003/DisasterAgent.
Figures & tables
Fig. 1: Comparison between standard VLM inference and the evidence-grounded reasoning of DisasterAgent. (a) A user query for a disaster impact caption from a bitemporal optical image pair. (b) An existing VLM generates an unverified report and misidentifies the disaster type. (c) DisasterAgent invokes perception tools and attaches inline citations to pixel-level visual evidence, mitigating hallucinations and producing more accurate captions.
Fig. 2: The overall architecture of DisasterAgent. The framework processes multimodal inputs through two primary stages. (1) Skill Routing: Driven by the VLM’s interpretation of the query, the system dynamically routes to appropriate skills within a pre-defined repository and seamlessly registers the required functional tools. (2) Skill-Guided Workflow: Guided by the registered skills, the framework sequentially executes a three-step pipeline comprising Perception, Evidence-based Augmentation, and Generation.
Fig. 3: Illustration of the STPG. (a) The Spatial Perception Graph constructs the spatial relationships between diverse geographical targets (e.g., buildings, roads, forests, farmland, and water) within a single phase. (b) The Temporal Evolution Edges establish the cross-phase mappings, explicitly capturing state transitions such as “maintained”, “shrunk”, “expanded”, “transitioned to”, “disappeared”, and “appeared”, while integrating domain-specific damage level assessments (e.g., building damage, road damage).
Vision Expert Tool
Function Description
Disaster Type Recognition [ 36 ]
Built upon DINOv3, this tool identifies the disaster type from bitemporal remote sensing images and supports 10 representative disaster categories, including earthquakes, floods, hurricanes, volcanoes, explosions, and conflicts.
Building Damage Analysis (4-class) [ 7 ]
Implemented with ChangeOS, this tool performs pixel-level building damage assessment based on the Joint Damage Scale, which integrates FEMA and HAZUS standards, and categorizes building conditions into four severity levels: no-damage, minor-damage, major-damage, and destroyed.
Building Damage Analysis (3-class) [ 8 ]
Based on ChangeMamba, this tool provides pixel-level structural damage perception following FEMA-oriented damage definitions, classifying buildings into three levels: intact, damaged, and destroyed.
Road Damage Analysis [ 8 ]
Powered by ChangeMamba, this tool extracts pixel-level road damage information from pre- and post-disaster imagery, distinguishing undamaged roads, flood-covered roads, and roads damaged by debris or rubble.
General Concept Segmentation [ 37 ]
Integrated with SAM3, this tool enables open-vocabulary object detection and segmentation for general disaster-related concepts, such as farmland, forest, water bodies, and other scene elements, supporting zero-shot generalization across diverse disaster scenarios.
TABLE I: Vision Expert Tools for Spatiotemporal Change Perception
Fig. 4: Illustration of the SCE extraction pipeline. The VLM first reasons over the STPG to generate atomic factual claims. Each claim is then independently verified against the raw bitemporal images to obtain a confidence score. Finally, claims with scores above the threshold are filtered to form the robust SCE set.
Fig. 5: Illustration of the Evidence-Fused Verifiable Generation. The framework employs a cascaded two-stage generation process: Stage 1 generates the disaster description grounded in all three evidence streams ( ESCE , EVGE , EEEEdesc ), while Stage 2 provides restoration advice conditioned exclusively on the generated description and the retrieved historical advice ( EEEEres ). Through explicit inline citations (e.g., [1] , [2] ) mapped to a reference list, the system establishes a closed, traceable evidence loop connecting the final report back to the raw pixel regions.
RSCC [ 16 ]
DisasterM3 [ 17 ]
Tasks
Change Captioning
Disaster Type Recognition Building Damage Counting Disaster Region Estimation Disaster Captioning Restoration Advice
TABLE II: Comparison of Benchmark Datasets Used for Evaluation
Task
Evaluation Dimension
Score
Abbr.
Disaster Captioning
Damage Assessment Precision
0∼5
DAP
Damage Detail Recall
0∼5
DDR
Factual Correctness
0∼5
FC
Restoration Advice
Recovery Necessity
0∼5
RN
Action Priority Precision
0∼5
APP
Strategic Completeness
0∼5
SC
TABLE III: Evaluation Dimensions for Comprehensive Report Generation
Method
N-Gram
Contextual Similarity
ROUGE-L (%)
METEOR (%)
ST5-SCS (%)
∙ General VLM
Kimi K2.6 [ 52 ]
11.14
10.98
52.29
GLM-4.5V [ 53 ]
13.24
10.64
50.57
InternVL3.5-8B [ 27 ]
12.05
10.84
47.98
Qwen3.5-122B-A10B [ 51 ]
12.27
10.59
50.20
TABLE IV: Performance Comparison on the RSCC Benchmark
Method
DTR
BDC
DRE
Disaster Report Generation
Disaster Captioning
Restoration Advice
ACC ↑
MAE ↓
MAE ↓
AVG ↑
DAP ↑
DDR ↑
FC ↑
AVG ↑
RN ↑
APP ↑
SC ↑
∙ General VLM
Kimi K2.6 [ 52 ]
74.22
32.63
4.46
1.37
1.61
1.21
1.30
2.96
3.52
2.78
2.57
MiniMax-M3 [ 56 ]
68.84
26.50
3.81
2.03
2.31
1.80
1.97
2.79
3.37
2.58
2.42
Qwen3.5-27B [ 51 ]
70.25
30.05
4.81
1.88
2.16
1.65
1.81
2.86
3.47
2.67
2.44
TABLE V: Performance Comparison on the DisasterM3 Benchmark
Fig. 6: Disaster captioning scores on the DisasterM3 benchmark across eight disaster types. The VLM, EarthAgent, and DisasterAgent all use Qwen3.5-27B as the base VLM, and EarthAgent and DisasterAgent both use the same expert toolkit.
Group
STPG
EA
Disaster Captioning
Restoration Advice
AVG
w/o Toolkit
×
×
1.88
2.86
2.37
w/ Toolkit
×
×
2.36
3.15
2.76
✓
×
2.59
2.96
2.78
✓
✓
2.87
3.22
3.05
TABLE VI: Ablation Study on the Structural Components
Fig. 7: Detailed ablation results across individual evaluation metrics on the DisasterM3 benchmark. The figure illustrates the incremental performance improvements in (a) disaster captioning and (b) restoration advice when integrating the expert toolkit, STPG, and EA into the base model.
Fig. 8: Qualitative results on a DisasterM3 report generation case. (a) The input query with the corresponding pre- and post-disaster images. (b) Caption produced by Qwen3.5-27B, which attributes the scene to a tsunami and introduces unsupported coastal changes (scores: DAP 2, DDR 1, FC 0). (c) Caption produced by DisasterAgent, which identifies a localized explosion, notes that surrounding roads and water bodies remain intact, and links each claim to a numbered visual evidence reference (scores: DAP 4, DDR 4, FC 5). (d) Ground truth caption.
Task
Metric
Full
w/o (Remove)
Base
EEE
VGE
SCE
Disaster Captioning
AVG
2.87
2.56
2.85
2.58
1.88
DAP
3.20
2.94
3.22
2.87
2.16
DDR
2.52
2.14
2.46
2.30
1.65
FC
2.88
2.60
2.89
2.56
1.81
Restoration Advice
AVG
3.22
3.08
-
-
2.86
TABLE VII: Contribution of Multi-Source Evidence
Metric
DisasterM3
RSCC
Number of Samples
2,004
988
Samples with SCE Citations (%)
94.86
85.32
Mean STPG Entities per Sample
182.37
35.84
Mean SCE Citations per Sample
3.40
1.93
Mean Linked STPG Entities per SCE Citation
5.23
8.02
TABLE VIII: SCE Citation Coverage and Entity Associations
Fig. 9: An SCE citation and its linked STPG entities. The citation in the report excerpt is traced to the corresponding SCE claim and its associated STPG entities, including region nodes and an evolution edge, which are rendered on the pre- and post-disaster images.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 12: Qualitative DisasterM3 captioning case for an earthquake scene. (a) The input query with the corresponding pre- and post-disaster images. (b) Caption produced by Qwen3.5-27B, which exaggerates minor structural disturbances as widespread building collapse and severe destruction (scores: DAP 2, DDR 1, FC 2). (c) Caption produced by DisasterAgent, which describes buildings as largely intact, acknowledges limited structural disturbances, and identifies intact roads, with selected claims linked to numbered evidence references (scores: DAP 5, DDR 4, FC 5). (d) Ground truth caption.
Fig. 13: Qualitative DisasterM3 captioning case for a flood scene. (a) The input query with the corresponding pre- and post-disaster images. (b) Caption produced by Qwen3.5-27B, which identifies the flood but overstates its impact on buildings, including unsupported claims that some buildings are submerged with only their rooftops visible (scores: DAP 3, DDR 2, FC 2). (c) Caption produced by DisasterAgent, which distinguishes inundated roads and farmland from buildings assessed as intact and retains uncertainty for some structures, with selected claims linked to numbered evidence references (scores: DAP 5, DDR 4, FC 5). (d) Ground truth caption.
Fig. 14: Qualitative DisasterM3 captioning case for a wildfire scene. (a) The input query with the corresponding pre- and post-disaster images. (b) Caption produced by Qwen3.5-27B, which misclassifies the event as a mudslide or debris flow and introduces unsupported claims of sediment deposition, altered drainage channels, and restricted road access (scores: DAP 1, DDR 0, FC 1). (c) Caption produced by DisasterAgent, which identifies the fire, describes extensive vegetation loss, and recognizes that the road network remains intact, with selected claims linked to numbered evidence references (scores: DAP 5, DDR 5, FC 5). (d) Ground truth caption.
Texas A&M University College Station, Texas, USA · National University of Singapore Singapore, Singapore · The University of Alabama Tuscaloosa, Alabama, USA