VLM Hallucination Mitigation

This field studies how to prevent vision-language models from inventing objects, attributes, relationships, or events that images and videos do not support. Common approaches include training-free changes to attention, logits, or hidden representations during decoding; contrastive and counterfactual visual checks; and preference or reinforcement learning that rewards grounded reasoning. Detection, verification, and abstention methods also check claims against regions, objects, or structured evidence, with evaluations spanning benchmarks such as POPE, CHAIR, AMBER, and medical and video tasks. A central challenge is reducing unsupported claims without making answers less informative or harming broader capabilities, and gains can vary across models and evaluation settings.