VLM Hallucination Mitigation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, down 15% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 131

All topics

ROT: Rotating Hidden States towards Contextual Vectors for Hallucination Mitigation in LVLMs
Oct 5, 2026Yijing Du, Xiangcheng Zhan, Shuo YangObject Hallucination in VLMs VLM Hallucination Mitigation
Harbin Institute of Technology, Shenzhen
Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splitting
Sep 30, 2026Chang Liu, Yu Tian, Rui XieFDR Control Large Vision-Language Models
School of Data, Mathematical, and Statistical Sciences Institute of Artificial Intelligence University of Central Florida
Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery
Sep 29, 2026Siqi Lu, Suo Wei, Yongbin Zheng +3Vision-Language Models Efficient VLM Inference
College of Intelligence Science and Technology, National University of Defense Technology, China · School of Computer Science, Northwestern Polytechnical University, China · Ningbo Institute, Northwestern Polytechnical University, China +1
DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing
Sep 29, 2026Jae-Ho Lee, Jeong-Eun Lee, Gyeong-Moon ParkVision-Language Models Large Vision-Language Models
Korea University, Seoul, Republic of Korea
Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection
Sep 28, 2026Chia-Ling Chen, Yu-Ting Ta, Jian-Yu Jiang-Lin +8Explainable Deepfake Detection Deepfake Detection
National Taiwan University · National Tsing Hua University · National Yang Ming Chiao Tung University
Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models
Sep 28, 2026Xingming Long, Jie Zhang, Yuecong Min +2Credit Assignment in RL Step-Level Credit Assignment in RL
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Zhongguancun Academy
Before the Token Commits: Trajectory-Level Benchmarking of Visual Hallucinations in Diffusion VLMs
Sep 28, 2026Yadong Wang, Siping Yue, Yu Tian +2VLM Evaluation Vision-Language Models
Nanjing University of Aeronautics and Astronautics · State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China · Security Capability Center
Where Hallucinations Live: A Cross-Architecture Circuit in VQ-Tokenized Vision-Language Models
Sep 24, 2026Shamanthak Hegde, Xiangrui Liu, Maitreya Patel +1Vision-Language Models Object Hallucination in VLMs
Arizona State University · Adobe
DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs
Sep 23, 2026Yingxuan Zhuang, Miao Pan, Wangjie Gan +6Multimodal Large Language Models VLM Hallucination Mitigation
Zhejiang University · Shanghai Artificial Intelligence Laboratory
Semantic-Spatial Agreement Verification for Mitigating Object Hallucination in Multimodal Large Language Models
Sep 15, 2026Ziheng Ren, Qian Gao, Jun Fan +3Hallucination Detection in VLMs Object Hallucination in VLMs
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan 250014, China · Shandong Engineering Research Center of Big Data Applied Technology, Faculty of Computer Science and Technology, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, China · Shandong Provincial Key Laboratory of Industrial Network and Information System Security, Shandong Fundamental Research Center for Computer Science, Jinan 250014, China +2
What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes
Sep 15, 2026Zhipeng Zhao, Wenxu Wang, Peishun Liu +1Hallucination Detection in VLMs Multimodal Grounding
1Ocean University of China
SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigation
Sep 15, 2026Zixiu Ding, Zilin Zhao, Yingjie He +3Object Hallucination in VLMs Visual Grounding
Central South University · Peking University
Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods
Sep 1, 2026Mehrdad Fazli, Sina Mansouri, Mohit Marvania +1VLM Evaluation Faithfulness in VLMs
Department of Computer Science, George Mason University Fairfax, VA 22030, USA
IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals
Sep 1, 2026Md. Atabuzzaman, Christian Alexander, Chris ThomasVLM Evaluation Large Vision-Language Models
Department of Computer Science Virginia Tech
Beyond Language Priors: Diagnosing and Fixing Visual-Origin Hallucinations in Multimodal LLM
Aug 31, 2026Peiyang Xu, Xiaopei Zhu, Jun Zhu +1Object Hallucination in VLMs Multimodal Contrastive Learning
Tsinghua University Beijing, China
Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models
Aug 30, 2026Aditi Sarker, Nazreen Shah, Rafi Ibn Sultan +3VLM Evaluation VLM Unlearning
Department of Computer Science, Wayne State University, MI, USA · Department of Electronics & Communications Engineering, IIIT Delhi, India · Institute for AI and Data Science (AIDaS), Wayne State University, MI, USA
Hallucination Mitigation for Large Vision-Language Models via Implicit Feature Stabilization
Aug 30, 2026Aditi Sarker, Rafi Ibn Sultan, Hui Zhu +2VLM Robustness Vision-Language Models
Department of Computer Science, Wayne State University, MI, USA · Institute for AI and Data Science (AIDaS), Wayne State University, MI, USA
Dual-Stream Cross-Anchor Correction Grounding Long-Form Captions and the Domain Limits of Object-Level Anchors
Aug 13, 2026LingKai BuImage Captioning Object Hallucination in VLMs
Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization
Aug 12, 2026Byungoh Ko, Jinyoung Park, Jongha Kim +3Direct Preference Optimization Object Hallucination in VLMs
Korea University, Seoul, Republic of Korea · KAIST, Daejeon, Republic of Korea
Test-Time Hallucination Control in Large Vision-Language Models
Aug 11, 2026Mehran Tamjidi, Hamidreza Dastmalchi, Ali Cheraghian +3Test-Time Adaptation of VLMs Large Vision-Language Models
University of Technology Sydney · York University, Canada · Australian National University +2
When Visual Signals Mislead: A Mechanistic Study of Attribute Hallucination in Vision-Language Models
Aug 11, 2026Yufei Zhang, Chenlu Zhan, Hongwei WangHallucination Detection in VLMs VLM Adaptation
Zhejiang University
CARE: Confidence-Aware Reasoning for Reliable Medical VQA
Aug 11, 2026Yuetian Du, Yucheng Wang, Zhenyuan Chen +9Medical VQA Medical VLMs
Zhejiang University · Ant Group · University of Michigan +1
UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations
Aug 11, 2026Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca +4Hallucination Detection in VLMs VLM Hallucination
NVIDIA Research, Tel Aviv, Israel · Technion, Haifa, Israel · Bar-Ilan University, Ramat Gan, Israel +1
Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching
Aug 10, 2026Yuke Li, Xuehan HouGUI Grounding Multimodal Large Language Models
School of Electronic and Computer Engineering, Peking University, Shenzhen, China
Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs
Aug 10, 2026Ali Cheraghian, Hamidreza Dastmalchi, Hamed Barzamini +4Large Vision-Language Models Model Editing
Macquarie University, Australia · Australian National University, Australia · York University, Canada +4
VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models
Aug 9, 2026Dong Xing, Jiaxin Chen, Hang Yang +3Video-Language Models VLM Hallucination Mitigation
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences Changchun, Jilin 130033, China · University of Chinese Academy of Sciences, Beijing 100049, China · City University of Hong Kong, Hong Kong, China
Wiener Representation Filtering for VLM Hallucination Suppression
Aug 8, 2026Ameen Ali, Tamim Zoabi, Lidor Brami +1Vision-Language Models Object Hallucination in VLMs
Tel Aviv University
Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination
Aug 7, 2026Zichuan Wang, Songlin Yang, Bo Peng +4Hallucination Detection in VLMs Vision-Language Models
School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Hong Kong University of Science and Technology · 2New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Science
Domain-Grounded Candidate Selection for Agentic Image Editing: A Shadow Removal Case
Aug 6, 2026Shilin Hu, Jingyi Xu, Dimitris Samaras +1Shadow Removal Physics-Guided Image Restoration
Stony Brook University, Stony Brook NY 11794, USA · UNC Charlotte, Charlotte NC 28223, USA
Teaching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO
Aug 5, 2026Xuzheng Yang, Jun Ling, Tao Huang +2RL for Language Models Group Relative Policy Optimization
School of Computer Science and Engineering, University of Electronic Science and Technology of China

Oct 5, 2026, cs.CVJ/K move · Enter open · S save

ROT: Rotating Hidden States towards Contextual Vectors for Hallucination Mitigation in LVLMs

Yijing Du, Xiangcheng Zhan, Shuo Yang

Harbin Institute of Technology, Shenzhen

Object Hallucination in VLMs VLM Hallucination Mitigation

Open paper AskPDF arXiv

Large Vision-Language Models (LVLMs) frequently suffer from object hallucination. Existing training-free interventions primarily manipulate attention weights, which indirectly affect the deep semantics reaching the final predictive layers. In this work, we shift our focus to the hidden state vectors extracted after self-attention and residual addition. Empirical analysis reveals that hallucinated tokens do not simply over-rely on linguistic priors; instead, they exhibit an anomalous contextual deviation, showing significantly lower similarities to both textual and visual contexts in intermediate layers. Motivated by this, we propose ROT, a layer-specific, training-free framework. ROT dynamically detects semantic deviation in the middle layers and applies a norm-preserving rotation to steer the hidden states back toward the local multimodal context plane spanned by the contexts. For subsequent layers, a representational smoothing mechanism is introduced to stabilize the calibrated trajectory. Extensive experiments on multiple benchmarks demonstrate that ROT consistently reduces hallucinations across various model architectures and scales, offering an efficient, geometry-driven solution for grounded generation.

Momentum

Related topics

Latest papers 131