Multimodal Hallucination

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 37

All topics

Self-Captioning Multimodal Interaction Tuning: Amplifying Exploitable Redundancies for Robust Vision Language Models
May 3, 2026Yuriel Ryan, Hei Man Ip, Adriel Kuek +2VLM Robustness Multimodal Hallucination
Singapore University of Technology and Design · DSO National Laboratories · Massachusetts Institute of Technology
DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding
Apr 28, 2026Cennet Oguz, Yasser Hamidullah, Josef van Genabith +1Multimodal Grounding Video Understanding
German Research Center for Artificial Intelligence (DFKI) · Saarland Informatics Campus
Exploring Audio Hallucination in Egocentric Video Understanding
Apr 26, 2026Ashish Seth, Xinhao Mei, Changsheng Zhao +9Audio-Visual Understanding Hallucination in Language Models
Meta · University Of Maryland, College Park
HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models
Apr 21, 2026Feiyu Zhao, Yiming Chen, Wenhuan Lu +3Audio-Language Model Evaluation Audio-Language Model Hallucination Detection
College of Intelligence and Computing, Tianjin University, China · ASUS Intelligent Cloud Services, Singapore
Mitigating Multimodal Hallucination via Phase-wise Self-reward
Apr 20, 2026Yu Zhang, Chuyang Sun, Kehai Chen +3Vision-Language Models Hallucination in Language Models
Harbin Institute of Technology · Peng Cheng Laboratory · Shenzhen, China
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
Date pendingSangyun Chung, Se Yeon Kim, Youngchae Chee +1Multimodal Robustness Hallucination in Language Models
Integrated Vision Language Lab, KAIST, South Korea
Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy
Date pendingYi-Cheng Lai, Hen-Hsen HuangLLM Sycophancy Evidence-Grounded Reasoning
Institute of Information Science, Academia Sinica Taipei, Taiwan

May 3, 2026, cs.CVJ/K move · Enter open · S save

Self-Captioning Multimodal Interaction Tuning: Amplifying Exploitable Redundancies for Robust Vision Language Models

Yuriel Ryan, Hei Man Ip, Adriel Kuek, Paul Pu Liang+1

Singapore University of Technology and Design · DSO National Laboratories · Massachusetts Institute of Technology

VLM Robustness Multimodal Hallucination Multimodal Instruction Tuning Multimodal Understanding

Open paper AskPDF arXiv

Current vision language models face hallucination and robustness issues against ambiguous or corrupted modalities. We hypothesize that these issues can be addressed by exploiting the shared information between modalities to compensate for the impaired one. To this end, we analyze multimodal interactions -- redundant (shared), unique (exclusive), and synergistic (emergent) task-relevant information provided by the modalities -- to determine their impacts on model reliability. Specifically, amplifying redundant interactions would increase this exploitable shared information to resolve these issues; yet, modern instruction datasets often eliminate redundancies to prioritize visual grounding. We bridge this gap through a self-captioning workflow featuring a \textsc{Multimodal Interaction Gate}: a mechanism to convert unique interactions into redundant interactions. Our findings suggest that increasing redundancy can reduce visual induced errors by 38.3% and improve consistency by 16.8%.

Momentum

Related topics

Latest papers 37