Hallucination-aware Faithful Reasoning in Remote Sensing
Organizations: School of Computer Science, Wuhan University · Zhongguancun Academy · School of Computer Science, Chongqing University
Abstract
Multimodal large language models (MLLMs) have demonstrated promising capabilities for remote sensing reasoning, yet they frequently generate visually unsupported reasoning that contradicts the visual evidence. Existing post-training methods optimize primarily for final-answer correctness, providing limited supervision for identifying and correcting hallucinations that arise during the reasoning process. To address this limitation, we propose FaithRS, a hallucination-aware self-reflection framework for faithful reasoning in remote sensing. We first construct SFHD-43K, a dataset of 43K structured, fine-grained hallucination annotations, and train a detector to identify hallucination types, underlying causes, and supporting visual evidence. FaithRS then incorporates this diagnostic feedback into a Safe Detector-Guided Corrective Rollout framework, where a safety-aware gate selectively determines whether and how to revise the response. Training is driven by a composite reward that jointly optimizes answer accuracy, reasoning faithfulness, hallucination reduction, format compliance, and revision safety. Experiments with Qwen3.5 backbones (0.8B, 4B, and 9B) show consistent improvements across four remote sensing benchmarks, with gains of up to 11.64 points.
Figures & tables
| Model | Publication | MME-RealWorld-RS | RSVQA-HR | CHOICE | LHRS | Avg |
|---|---|---|---|---|---|---|
| General Domain MLLMs | ||||||
| GPT-4o ( Hurst et al., 2024 ) | arXiv 2024 | 21.36 | 60.82 | 65.66 | 66.15 | 53.50 |
| Gemini-2.5-Pro ( Comanici et al., 2025 ) | arXiv 2025 | 38.06 | 67.49 | 72.48 | 66.41 | 61.11 |
| Claude-Haiku-4.5 | – | 21.83 | 38.79 | 59.82 | 54.03 | 43.62 |
| Qwen3.5-0.8B ( Qwen Team, 2026 ) | – | 38.36 | 53.48 | 51.90 | 45.40 | 47.29 |
| Qwen3.5-4B ( Qwen Team, 2026 ) | – | 53.22 | 65.92 | 70.84 | 68.54 | 64.63 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Parse (%) | Acc. | F1 | MCC |
|---|---|---|---|---|
| SFT-Short | 22.30 | 0.5331 | 0.3497 | 0.2386 |
| SFT-Full | 56.90 | 0.6737 | 0.6126 | 0.4690 |
| RL (Best) | 98.40 | 0.8865 | 0.9006 | 0.7685 |
| Setting | Parse (%) | Acc. | Prec. | Recall | F1 | MCC | Factual-F1 | Logical-F1 |
|---|---|---|---|---|---|---|---|---|
| Baseline | 98.40 | 0.8865 | 0.9014 | 0.8998 | 0.9006 | 0.7685 | 0.9033 | 0.7221 |
| w/ | 91.50 | 0.8534 | 0.8784 | 0.8629 | 0.8706 | 0.7018 | 0.8672 | 0.7322 |
| w/ | 97.70 | 0.8464 | 0.8050 | 0.9649 | 0.8777 | 0.6959 | 0.8794 | 0.7385 |
| w/ | 95.30 | 0.8484 | 0.8472 | 0.8963 | 0.8711 | 0.6891 | 0.8338 | 0.6391 |
| w/ | 96.60 | 0.8685 | 0.8911 | 0.8770 | 0.8840 | 0.7323 | 0.8623 | 0.7060 |
| w/ | 92.30 | 0.8404 | 0.8024 | 0.9561 | 0.8725 | 0.6818 | 0.8760 | 0.7348 |
VLM Hallucination Mitigation
Vision-language model hallucination mitigation keeps generated claims grounded in visual evidence.
See all 227 papers