Hallucination-aware Faithful Reasoning in Remote Sensing

Yi Liu; Jing Zhang; Xiang Feng; Xiaoyu Tian; Haonan Guo; Di Wang; Zengmao Wang; Bo Du

cs.CVOct 13, 2026

Hallucination-aware Faithful Reasoning in Remote Sensing

Authors: Yi Liu, Jing Zhang, Xiang Feng, Xiaoyu Tian, Haonan Guo, Di Wang, Zengmao Wang, Bo Du

Organizations: School of Computer Science, Wuhan University · Zhongguancun Academy · School of Computer Science, Chongqing University

Abstract

Multimodal large language models (MLLMs) have demonstrated promising capabilities for remote sensing reasoning, yet they frequently generate visually unsupported reasoning that contradicts the visual evidence. Existing post-training methods optimize primarily for final-answer correctness, providing limited supervision for identifying and correcting hallucinations that arise during the reasoning process. To address this limitation, we propose FaithRS, a hallucination-aware self-reflection framework for faithful reasoning in remote sensing. We first construct SFHD-43K, a dataset of 43K structured, fine-grained hallucination annotations, and train a detector to identify hallucination types, underlying causes, and supporting visual evidence. FaithRS then incorporates this diagnostic feedback into a Safe Detector-Guided Corrective Rollout framework, where a safety-aware gate selectively determines whether and how to revise the response. Training is driven by a composite reward that jointly optimizes answer accuracy, reasoning faithfulness, hallucination reduction, format compliance, and revision safety. Experiments with Qwen3.5 backbones (0.8B, 4B, and 9B) show consistent improvements across four remote sensing benchmarks, with gains of up to 11.64 points.

Figures & tables

Figure 2: Overview of the structured fine-grained hallucination diagnosis pipeline.

Figure 3: Overview of the hallucination-aware self-reflection training framework.

General Domain MLLMs
Model	Publication	MME-RealWorld-RS	RSVQA-HR	CHOICE	LHRS	Avg
GPT-4o ( Hurst et al., 2024 )	arXiv 2024	21.36	60.82	65.66	66.15	53.50
Gemini-2.5-Pro ( Comanici et al., 2025 )	arXiv 2025	38.06	67.49	72.48	66.41	61.11
Claude-Haiku-4.5	–	21.83	38.79	59.82	54.03	43.62
Qwen3.5-0.8B ( Qwen Team, 2026 )	–	38.36	53.48	51.90	45.40	47.29
Qwen3.5-4B ( Qwen Team, 2026 )	–	53.22	65.92	70.84	68.54	64.63

Table 1: Quantitative comparison of general-purpose and remote sensing MLLMs on four remote sensing benchmarks. Avg denotes the arithmetic mean across MME-RealWorld-RS, RSVQA-HR, CHOICE, and LHRS. Dark, medium, and light green indicate the best, second-best, and third-best results among models with reported scores in each column, respectively.

Figure 4: Ablation study of FaithRS at the 0.8B and 4B scales.

Figure 5: Training dynamics of FaithRS-0.8B during detector-guided corrective rollout.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Parse (%)	Acc.	F1	MCC
SFT-Short	22.30	0.5331	0.3497	0.2386
SFT-Full	56.90	0.6737	0.6126	0.4690
RL (Best)	98.40	0.8865	0.9006	0.7685

Appendix

Table 2: Comparison across hallucination detector training stages. Bold values indicate the best result in each column.

Setting	Parse (%)	Acc.	Prec.	Recall	F1	MCC	Factual-F1	Logical-F1
Baseline	98.40	0.8865	0.9014	0.8998	0.9006	0.7685	0.9033	0.7221
w/ $\rho$	91.50	0.8534	0.8784	0.8629	0.8706	0.7018	0.8672	0.7322
w/ $e$	97.70	0.8464	0.8050	0.9649	0.8777	0.6959	0.8794	0.7385
w/ $j$	95.30	0.8484	0.8472	0.8963	0.8711	0.6891	0.8338	0.6391
w/ $\rho{+}e$	96.60	0.8685	0.8911	0.8770	0.8840	0.7323	0.8623	0.7060
w/ $\rho{+}j$	92.30	0.8404	0.8024	0.9561	0.8725	0.6818	0.8760	0.7348

Appendix

Table 3: Ablation of structured diagnostic fields during detector RL. Bold values indicate the best result in each column.

Figure 6: Evolution of the composite reward and its constituent terms. Solid curves show means and shaded regions show standard deviations.

Panel of Figure 7 — Figure 7: Core detector-guided training dynamics. Left: initial hallucination, final hallucination, and fix rates. Middle: Safe-Gate routing rates. Right: improved, preserved, and degraded answer-correctness outcomes.

Panel of Figure 8 — Figure 8: Representative color-recognition examples from MME-RealWorld-RS. Case 1 shows a scale-dependent correction of a prior-driven attribute error. Case 2 is answered correctly by all three FaithRS scales, demonstrating stable color recognition. Case 3 remains a shared multicolor-recognition failure across all three models.

Panel of Figure 9 — Figure 9: Representative counting examples from MME-RealWorld-RS. Case 4 exposes an inconsistency between the absence judgment in the reasoning and the final option selected by FaithRS-0.8B. Case 5 shows a scale-dependent small-target omission, with only FaithRS-9B correctly identifying the two red cars. Case 6 is answered correctly by all three models and demonstrates stable target-absence recognition.

Panel of Figure 10 — Figure 10: Representative spatial-localization examples from MME-RealWorld-RS. Case 7 is answered correctly by all three FaithRS scales and provides a stable evidence-grounded localization example. Case 8 shows scale-dependent recovery from spatial misgrounding by FaithRS-0.8B. In Case 9, all three models select the correct location, although some explanations introduce semantic interpretations that are not directly verifiable from the image.

VLM Hallucination Mitigation

Vision-language model hallucination mitigation keeps generated claims grounded in visual evidence.

See all 227 papers

Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing
Mar 3, 2026Yi Liu, Jing Zhang, Di Wang +3Remote Sensing VQA VLM Hallucination Mitigation
School of Computer Science, Wuhan University, China · Zhongguancun Academy, China · School of Computer Science, Chongqing University, China
ReDiS: Reasoning-Diagnosis Self-Evolution for Multimodal Reasoning in Remote Sensing
Oct 13, 2026Yi Liu, Wentao Jiang, Shuo Ni +5Remote Sensing VLMs Multimodal Reasoning
School of Computer Science, Wuhan University · Zhongguancun Academy · National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology
GeoSolver: Scaling Test-Time Reasoning in Remote Sensing with Fine-Grained Process Supervision
Mar 10, 2026Lang Sun, Ronghao Fu, Zhuoran Duan +3VLM Reasoning Process Reward Models
College of Computer Science and Technology, Jilin University, Changchun 130012, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University

Mar 3, 2026J/K move · Enter open · S save

Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing

Yi Liu, Jing Zhang, Di Wang, Xiaoyu Tian+2

School of Computer Science, Wuhan University, China · Zhongguancun Academy, China · School of Computer Science, Chongqing University, China

Remote Sensing VQA VLM Hallucination Mitigation Multimodal Large Language Models Vision-Language Grounding Hallucination Detection in VLMs Remote Sensing Benchmarks

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) suffer from pronounced hallucinations in remote sensing visual question-answering (RS-VQA), primarily caused by visual grounding failures in large-scale scenes or misinterpretation of fine-grained small targets. To systematically analyze these issues, we introduce RSHBench, a protocol-based benchmark for fine-grained diagnosis of factual and logical hallucinations. To mitigate grounding-induced factual hallucinations, we further propose Relative Attention-Driven Actively Reasoning (RADAR), a training-free inference method that leverages intrinsic attention in MLLMs to guide progressive localization and fine-grained local reasoning at test time. Extensive experiments across diverse MLLMs demonstrate that RADAR consistently improves RS-VQA performance and reduces both factual and logical hallucinations. Code and data will be publicly available at: https://github.com/MiliLab/RADAR

Abstract

VLM Hallucination Mitigation

Explore similar work

Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing

ReDiS: Reasoning-Diagnosis Self-Evolution for Multimodal Reasoning in Remote Sensing

GeoSolver: Scaling Test-Time Reasoning in Remote Sensing with Fine-Grained Process Supervision