Multimodal Hallucination
Overview
Multimodal hallucination research studies when vision-language and audio-visual models invent details, misread evidence, or let one modality override another. Researchers diagnose these failures with benchmarks such as HallusionBench and AVHBench, including tests for adversarial inputs, temporal reasoning, false premises, and error propagation across dialogue. Mitigation approaches include training-free contrastive or relevance-aware decoding, causal interventions on internal representations, and preference optimization that rewards grounded reasoning. A central challenge is improving evidence use without sacrificing general capabilities: strong multimodal reasoning does not guarantee reliable grounding, and uncertainty or disagreement between modalities does not always indicate when a model should abstain or intervene.
Latest papers
51 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.