VLM Hallucination Mitigation

VLM: Vision-Language Model

Overview

This field studies how to prevent vision-language models from inventing objects, attributes, relationships, or events that images and videos do not support. Common approaches include training-free changes to attention, logits, or hidden representations during decoding; contrastive and counterfactual visual checks; and preference or reinforcement learning that rewards grounded reasoning. Detection, verification, and abstention methods also check claims against regions, objects, or structured evidence, with evaluations spanning benchmarks such as POPE, CHAIR, AMBER, and medical and video tasks. A central challenge is reducing unsupported claims without making answers less informative or harming broader capabilities, and gains can vary across models and evaluation settings.

Latest papers

CardsList

243 papers11 papers in the last four weeks, down 54% on the four weeks before. 0.1% of all new papers.

  1. Beyond Seeing: Fine-Grained Preference Validation and Severity-Aware Alignment for Mitigating Visual Hallucinations in Vision-Language Models

    Nov 16, 2026Abdulaziz Alqahtani, Alwaleed Alharthi, Muzammil BehzadVLM Hallucination MitigationObject Hallucination in VLMs

  2. GEM-MM: Reducing VLM Hallucination with Entropy-Guided Preference Alignment

    Nov 15, 2026Yuanshan Chua, Xuejiao ZhaoVision-Language ModelsVLM Hallucination Mitigation

  3. SARGO: Agreement-Gated Reinforcement and Suppression via On-Policy Self-Guidance for Visual Hallucination Mitigation

    Nov 14, 2026Jinyu Li, Zhiming Zhang, Hao Fang +4VLM Hallucination MitigationRL for VLMs

  4. VEPO: Mitigating Multilingual Hallucinations in LVLMs via Visual Evidence-guided Preference Optimization

    Nov 13, 2026Chengpeng Fu, Xiaocheng Feng, Wenshuai Huo +4VLM Hallucination MitigationVision-Language Models

  5. Fine-Grained Generative Hallucination Detection and Correction for Aerial Image Captioning

    Nov 11, 2026Debolena Basak, Somsubhra De, Maunendra Sankar DesarkarHallucination Detection in VLMsVLM Hallucination Mitigation

  6. Towards Robust Vision-Language Reasoning: A Multi-Stage Optimization Pipeline with Masked Diffusion Model

    Nov 11, 2026Shunming Liu, Pei Huang, Yajing Sun +1VLMs for Autonomous DrivingAutonomous Driving Planning

  7. Entropogram-guided Visual Realignment Decoding: Capturing the Rhythm of Multimodal Reasoning

    Nov 4, 2026Jiaqi Deng, Zhe Fu, Yakun Chen +5Entropy-Based DecodingMultimodal Hallucination

  8. VIGOR: Visual Gain Ordering for Hallucination Mitigation in Multimodal Discrete Diffusion Language Models

    Nov 3, 2026Junzhe Chen, Tian Qin, Eugenie Shi +3VLM Hallucination MitigationDiffusion Language Models

  9. SPOD: Recovering Suppressed Temporal Evidence in Video-LLMs via Single-Pass Orthogonal Decomposition

    Nov 3, 2026Yangguang Lin, Quan Fang, Xiaoyan Liu +1Video-Language ModelsTemporal Reasoning in VLMs

  10. Entropy-Gated Dynamic Rectification for Object Hallucination in Large Vision-Language Models

    Nov 3, 2026Riasad Alvi, Md. Asaduzzaman Shuvo, Md. Ahashan Habib +1activation steering in vision-language modelsObject Hallucination in VLMs

  11. CheXPoT: Post-Hoc Visual Verification and Correction of VLM-Generated Radiology Reports

    Oct 31, 2026Joey Guo, Rayan Ansari, Alaa YoussefVLM Hallucination MitigationRadiology VLMs

  12. PLANE: Plug-and-Play Fine-Grained Visual Experts for Hallucination Mitigation in Large Vision-Language Models

    Oct 18, 2026Jianghao Yin, Lang Yu, Qianxi Yu +4VLM Hallucination MitigationObject Hallucination in VLMs

  13. MedPAO-VL: Interpretable and Protocol-Compliant Agent for Structured Radiology Report Generation

    Oct 15, 2026Shrish Shrinath Vaidya, Gowthamaan Palani, Minmini Selvam +2Radiology Report GenerationVLM Hallucination Mitigation

  14. Mitigating Video Hallucinations via Attention-Guided Representation Perturbation

    Oct 14, 2026Zhixin Cai, Ruoqi Yuan, Xinchen Yang +4VLM Hallucination MitigationHallucination Mitigation

  15. Hallucination-aware Faithful Reasoning in Remote Sensing

    Oct 13, 2026Yi Liu, Jing Zhang, Xiang Feng +5VLM Hallucination MitigationFaithfulness in VLMs

  16. From Token Correction to Claim Admission: Training-Free Hallucination Mitigation for Large Vision-Language Models

    Oct 13, 2026Hyeonggeun Han, Taehyun Cho, Sangwoo HongVLM Hallucination MitigationObject Hallucination in VLMs

  17. Teaching Vision-Language Models to Decline: Support Completion Against Mirage

    Oct 12, 2026Alfaj Uddin Ahmed, Md. Shaown Miah, Md Arid Hasan +2Abstention in VLMsVLM Hallucination Mitigation

  18. RSAJEV: Training-Free Abstaining Prototype Memory for Eliminating Silent Hallucination in JEV-Based Remote Sensing Classification

    Oct 11, 2026Man Chen, Antai Chen, Shuai Xu +5Remote Sensing Scene ClassificationVLM Hallucination Mitigation

  19. Metric-Driven Reinforcement Learning for Object Hallucination Reduction in LVLM-Based Image Captioning

    Oct 10, 2026Thanh Nguyen, Hee Suk Yoon, Tung Minh Luu +2Object Hallucination in VLMsVLM Hallucination Mitigation

  20. The Evidence Anchoring Boundary: Where Visual Interventions Matter and When They Help in Multimodal LLMs

    Oct 9, 2026Qiji Zhou, Junyi Hu, Peiyu Wang +1VLM Hallucination MitigationMultimodal Large Language Models

  21. Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations

    Oct 8, 2026Shuran Ma, JiaLe Li, Yuxin Dong +9VLM Hallucination MitigationLarge Vision-Language Models

  22. From Suppression to Repair: Mitigating Object Hallucination in Large Vision-Language Models via Localized Distribution Alignment

    Oct 8, 2026Chen Zhao, Xingping Dong, Jiachun Shi +5Object Hallucination in VLMsVLM Hallucination Mitigation

  23. SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders

    Oct 8, 2026Jeonghyo Song, YoungJoon YooAttention Intervention in VLMsVision-Language Grounding

  24. Confidence-Gated Cloud-Edge Cascade Triage via Variational Risk Minimization for Medical Imaging

    Oct 1, 2026Xinye Yang, Zhusi Zhong, Scott Collins +7Knowledge DistillationChest X-Ray Classification

  25. Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splitting

    Sep 30, 2026Chang Liu, Yu Tian, Rui XieFDR ControlLarge Vision-Language Models

  26. Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery

    Sep 29, 2026Siqi Lu, Suo Wei, Yongbin Zheng +3Vision-Language ModelsEfficient VLM Inference

  27. DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing

    Sep 29, 2026Jae-Ho Lee, Jeong-Eun Lee, Gyeong-Moon ParkVision-Language ModelsLarge Vision-Language Models