VLM Hallucination Mitigation

VLM: Vision-Language Model

Overview

This field studies how to prevent vision-language models from inventing objects, attributes, relationships, or events that images and videos do not support. Common approaches include training-free changes to attention, logits, or hidden representations during decoding; contrastive and counterfactual visual checks; and preference or reinforcement learning that rewards grounded reasoning. Detection, verification, and abstention methods also check claims against regions, objects, or structured evidence, with evaluations spanning benchmarks such as POPE, CHAIR, AMBER, and medical and video tasks. A central challenge is reducing unsupported claims without making answers less informative or harming broader capabilities, and gains can vary across models and evaluation settings.

Latest papers

CardsList

243 papers11 papers in the last four weeks, down 54% on the four weeks before. 0.1% of all new papers.

  1. IKOD: Image-Attention-Guided Key-Value Merging against Object Hallucination in the Decoding of Large Vision-Language Models

    Aug 5, 2025Jiabing Yang, Chenhang Cui, Yixiang Chen +3Object Hallucination in VLMsVLM Hallucination Mitigation

  2. SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision

    Aug 5, 2025Zhaoxu Li, Chenqi Kong, Yi Yu +6VLM EvaluationLLM Hallucination Mitigation

  3. Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models

    May 18, 2025Kai Tang, Jinhao You, Yichen Guo +8Vision-Language ModelsHallucination in Language Models