Object Hallucination in VLMs

This topic studies when vision-language models mention objects that are absent, miss visible objects, or misread relationships in images and videos. Researchers evaluate these failures with captioning, question-answering, and grounding benchmarks such as CHAIR, POPE, and AMBER, and use controlled tests to separate weak perception from language-prior effects. Mitigation approaches include training-free contrastive decoding and attention or representation steering, as well as region-based verification, preference tuning, and reinforcement learning. A central challenge is reducing unsupported claims without suppressing correct details or harming fluency, while ensuring improvements hold across models, tasks, and evaluation methods.