VLM Hallucination Mitigation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, down 15% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

Grounding Agentic VLMs with Dedicated Segmentation for Fine-Grained Vehicle Damage Assessment
Aug 3, 2026Vishwajeet Shivaji Hogale, Anjali Pai, Nitya RaviImage Segmentation Vision-Language Grounding
Northeastern University
When Model Priors Conflict with Visual Evidence: Mitigating Commonsense-Driven Hallucinations by Selective Prior Calibration
Jul 31, 2026Kesheng Chen, Yamin Hu, Wenjian LuoVLM Hallucination VLM Calibration
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies Institute of Cyberspace Security School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China
CAER: Conflict-Aware Evidence Routing with Dual Prefix Experts for Multimodal Large Language Models
Jul 31, 2026Zixuan Liu, Juntao Cai, Xiaoxu Cai +2Mixture of Experts Multimodal Large Language Models
Zhejiang Key Lab of Accessible Perception and Intelligent Systems, Zhejiang University, Hangzhou 310027, China
Hallucinations Leave a Grounding Signature:Verifier-Guided Decoding for Selective Object Correction
Jul 30, 2026Lei Yang, Xinze Liu, Dayan Wu +7Hallucination Detection in VLMs LLM Hallucination Mitigation
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA
Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications
Jul 29, 2026Piyush Jain, Kousik Dasgupta, Rajarshi Roy +1Visual Spatial Reasoning 3D Spatial Reasoning
Heritage Institute of Technology, Kolkata, India · Kalyani Government Engineering College, Kalyani, India · IAIRO, India +1
TPCD: Tone-Pressure Contrastive Decoding and the Label-Free Gating Bottleneck in Vision-Language Models
Jul 29, 2026Jinkun Zhao, Kui Zhang, Wenjun WuVLM Robustness Adversarial Attacks on VLMs
Beihang University
Towards Faithful Sentimental Image Captioning via Evidence-Aware Multi-Agent Reasoning
Jul 28, 2026Tiecheng Cai, Zexian Yang, Chao Chen +2Image Captioning Vision-Language Grounding
College of Computer and Data Science, Fuzhou University Fuzhou, China · School of Computer Science and Technology, Harbin Institute of Technology Shenzhen, China
Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs
Jul 28, 2026Arnav Bendre, Guneesh Gupta, Kavish Grover +2Object Hallucination in VLMs Multimodal Large Language Models
Department of Hydro and Renewable Energy Indian Institute of Technology, Roorkee · Department of Electrical Engineering Indian Institute of Technology, Roorkee · Department of Mechanical Engineering Indian Institute of Technology, Roorkee +2
Disentangling Semantic Attention from Structural Bias in the Attention Manifold
Jul 27, 2026Pengkun Jiao, Bin Zhu, Jingjing Chen +1Object Hallucination in VLMs Multimodal Large Language Models
Fudan University · Singapore Management University
DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models
Jul 27, 2026Hao Yang, Jin Wang, Xuejie ZhangLLM Alignment Multimodal Large Language Models
School of Information Science and Engineering Yunnan University Kunming, China
HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models
Jul 23, 2026Weilin Jin, Mingyu Wang, Wenbo Li +5Hallucination Detection in VLMs Multimodal Hallucination
Peking University Beijing, China · Joy Future Academy Beijing, China
Geo3R: Mitigating Spatial Reasoning Hallucination in Multimodal Large Language Models
Jul 23, 2026Mingyu Wang, Weilin Jin, Wenbo Li +33D Spatial Reasoning Hallucination in Language Models
Peking University Beijing, China · Joy Future Academy Beijing, China
ProCap: Prominence-guided Object Rectification for Faithful and Comprehensive Video Captioning
Jul 23, 2026Debjyoti Das Adhikary, Aritra Hazra, Partha Pratim ChakrabartiVideo Captioning Vision-Language Grounding
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur 721302, India
Visual Token Compression Enhances Robustness of MLLMs
Jul 21, 2026Shishen Gu, Jiequan Cui, Wenbo Hu +3VLM Robustness Adversarial Attacks on VLMs
Hefei University of Technology · Hefei, China
MissingBench-Verified: Probing Vision-Language Models' Inability to Detect Missing Object Parts
Jul 21, 2026Wenqi Marshall Guo, Qingyun Qian, Shiyu Zhou +2VLM Evaluation Object Hallucination in VLMs
Department of CMPS, University of British Columbia, Canada · Weathon Software, Canada · Department of Computer Science, University of British Columbia, Canada
Look Clearly Before Answering: Mitigating Hallucinations in LVLMs via Saliency-Driven Perceptual Realignment
Jul 18, 2026Pengxu Chen, Yao Zhu, Guangming Zhu +4Large Vision-Language Models VLM Hallucination Mitigation
Xidian University Xi’an, China · Tsinghua University Beijing, China · Shanghai Road Transport Development Center Shanghai, China +1
Groc-PO: Grounded Context Preference Optimization for Truthful Multimodal LLMs
Jul 15, 2026Zhixiao Zheng, Zheren Fu, Zhiyuan Yao +3Multimodal Grounding Multimodal Large Language Models
University of Science and Technology of China Hefei, China · Xiaomi Corporation Beijing, China · State Key Laboratory of Communication Content Cognition, People’s Daily Online Beijing, China
Mixture of Cognitive Experts in Large Vision-Language Models
Jul 12, 2026Robert Wijaya, Ngai-Man CheungVision-Language Models Large Vision-Language Models
Singapore University of Technology and Design (SUTD), Singapore
VaseMuseum: Digital Intelligent Museum for Ancient Greek Pottery
Jul 7, 2026Jiazi Wang, Nonghai Zhang, Qiushi Xie +5Multimodal RAG Cultural Heritage
School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China · School of Computer Science, Peking University, Beijing 100871, China · Department of Computer Science and Information Technology, La Trobe University, Melbourne VIC 3000, Australia +1
Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention
Jul 7, 2026Daniel Shalam, Emanuel Ben Baruch, Avi Ben Cohen +1Multimodal Grounding LLM Uncertainty Estimation
Amazon
SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering
Jul 5, 2026Kai Tang, Jinhao You, Bohua Zhang +6Large Vision-Language Models Object Hallucination in VLMs
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Nanyang Technological University · University of Pennsylvania +4
ESC: Emotional Self-Correction for Reliable Vision-Language Models
Jul 1, 2026Tien-Huy Nguyen, Minh-Nhat Nguyen, Nguyen Nhat Huy +9LLM Self-Correction VLM Reasoning
GenAI4E Lab · University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam +10
M2Note: Continual Evolution of Vision Language Models via Mistake Notebook Learning
Jul 1, 2026Haiwen Li, Jing Tang, Rui Chen +2Memory-Augmented VLMs VLM Reasoning
AMAP, Alibaba Group
Information-Regularized Attention for Visual-Centric Reasoning
Jul 1, 2026Guohao Sun, Xiaofang Wang, Yash Patel +3VLM Robustness Vision-Language Grounding
1FAIR at Meta, 2Rochester Institute of Technology · *Work done at Meta
Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence
Jun 30, 2026Rui Hao, Qiankun Li, Junyuan Mao +4Radiology Report Generation Medical VQA
Huazhong University of Science and Technology, Wuhan, China · Imperial Global Singapore, Imperial College London, London, United Kingdom · Nanyang Technological University, Singapore, Singapore +2
ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs
Jun 30, 2026Zhiyuan Yao, Zheren Fu, Zhixiao Zheng +3Cross-Attention Direct Preference Optimization
University of Science and Technology of China, Hefei, China · State Key Laboratory of Communication Content Cognition, People’s Daily Online, China · Huawei Technologies Ltd., China
The Label Imitation Game: Turing Test Network for Zero-Shot Pseudo-Label Pruning
Jun 29, 2026Brent A. Griffin, Jason J. CorsoPseudo-Labeling VLM Hallucination Mitigation
Voxel51 · University of Michigan
Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning
Jun 29, 2026Peng, Lee, Yin Zhang +9Vision-Language Grounding VLM Reasoning
GMLab · Hong Kong Polytechnic University · XPENG Robotics
See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
Jun 29, 2026Yuqing Lei, Wenbo Lyu, Yingjun Du +3Visual Attention Large Vision-Language Models
University of Chinese Academy of Sciences · University of Amsterdam · United Imaging Healthcare Co., Ltd.
Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation
Jun 29, 2026Xin Zou, Haolin Deng, Yibo Yan +5Multimodal Large Language Models Multimodal Hallucination
HKUST (GZ) · HKUST · OPPO AI Center

Aug 3, 2026, cs.CVJ/K move · Enter open · S save

Grounding Agentic VLMs with Dedicated Segmentation for Fine-Grained Vehicle Damage Assessment

Vishwajeet Shivaji Hogale, Anjali Pai, Nitya Ravi

Northeastern University

Image Segmentation Vision-Language Grounding VLM Hallucination Mitigation

Open paper AskPDF arXiv

Vision-language models (VLMs) are increasingly deployed as reasoning agents in real-world visual assessment pipelines, yet their spatial grounding remains unreliable for fine-grained, visually ambiguous targets. We study this gap in the context of automated vehicle damage assessment, where fine-grained defects such as scratches and hairline cracks occupy few pixels, produce weak gradient signal, and are easily confused with reflections and surface texture. We show that a state-of-the-art VLM (Qwen-VL) achieves strong semantic classification accuracy (87.3%) on this task but is systematically ungrounded at the spatial level: it hallucinates damage in reflective regions, misses elongated scratches entirely, and produces spatially inconsistent outputs when prompted for localization. We propose TinyDamage, a hybrid architecture that delegates spatial grounding to a dedicated multi-task segmentation model while reserving the VLM for semantic reasoning and report generation. On the segmentation side, we find that the choice of loss function has an outsized and underexplored effect on tiny-object grounding: focal loss, widely used for class imbalance, collapses tiny-damage detection to zero, while a supervised contrastive objective measurably improves damage/background separability. We integrate the segmentation model into a 7-node LangGraph agent pipeline that grounds every VLM generation step in the segmentation output, and show that this grounding reduces the report hallucination rate from 92% (text-only) and 78% (image-only) to 31% in a controlled evaluation on 100 human-verified reports. We introduce DET_l, a permissive per-category detection metric for evaluating tiny-object grounding under class imbalance, and report latency and reliability characteristics of the deployed pipeline.

Momentum

Related topics

Latest papers 131