Hallucination in Language Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 160% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 187

All topics

Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
May 20, 2026Meng Shen, Minghao Wu, Deepu RajanVision-Language Models Hallucination in Language Models
Nanyang Technological University · Monash University
Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
May 20, 2026Zhe Cheng, Wenyu Chen, Fode Zhang +1VLM Robustness Hallucination in Language Models
Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China. · Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore.
ACL-Verbatim: hallucination-free question answering for research
May 20, 2026Gábor Recski, Szilveszter Tóth, Nadia Verdha +2Retrieval-Augmented Generation LLM Hallucination Mitigation
TU Wien · KR Labs
Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy
May 20, 2026Yutong Xie, Zhenglin Hua, Ran Wang +3Vision-Language Models Hallucination in Language Models
School of Computer Science and Engineering, Southeast University, Nanjing, China · School of Artificial Intelligence, Shenzhen University, Shenzhen, China · National Engineering Laboratory for Big Data Systems Computing Technology, Shenzhen University, Shenzhen, China +3
VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering
May 20, 2026Jiayi Chen, Benteng Ma, Zehui Liao +3Hallucination Detection in VLMs Healthcare
Department of Data Science & AI, Faculty of Information Technology, Monash University, Melbourne, VIC 3800, Australia · Hong Kong Polytechnic University, Hong Kong SAR, China · Alfred Health Radiology, Alfred Health, Melbourne, VIC 3004, Australia +1
Do No Harm? Hallucination and Actor-Level Abuse in Web-Deployed Medical Large Language Models
May 20, 2026Sunday Oyinlola Ogundoyin, Muhammad Ikram, Rahat MasoodLanguage Model Safety Evaluation Healthcare
Macquarie University, Sydney, Australia · The University of New South Wales, Sydney, Australia
HalluCXR: Benchmarking and Mitigating Hallucinations in Medical Vision-Language Models for Chest Radiograph Interpretation
May 19, 2026Haoyu Wang, Zitong LiVision-Language Models Audio-Language Model Hallucination Detection
Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models
May 19, 2026Emmy Liu, Varun Gangal, Michael Yu +4LLM Evaluation Hallucination in Language Models
Carnegie Mellon University · All authors are affiliated with DegenAI Labs. · Patronus AI +3
Causal Evidence for Attention Head Imbalance in Modality Conflict Hallucination
May 19, 2026Jinrui Jiang, Zhangtai Wu, Zhen Wu +1Hallucination in Language Models Multimodal Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
TRACE: Trajectory Correction from Cross-layer Evidence for Hallucination Reduction
May 18, 2026Tej Sanibh RanadeLLM Hallucination Mitigation Hallucination in Language Models
Independent Researcher
FedSDR: Federated Self-Distillation with Rectification
May 18, 2026Ziheng Ren, Zhanming Shen, Hao Wang +2LLM Fine-Tuning Hallucination in Language Models
Beijing University of Aeronautics and Astronautics · Zhejiang University · Stevens Institute of Technology +1
The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort
May 16, 2026Aleksandr ChurilovLLM Evaluation Hallucination in Language Models
Independent Researcher
PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts
May 16, 2026Khizar Hussain, Murat KantarciogluLLM Evaluation Language Model Error Detection
Virginia Tech
HalluScore: Large Language Model Hallucination Question Answering Benchmark
May 16, 2026Aisha Alansari, Hamzah LuqmanMultilingual Language Model Evaluation LLM Evaluation
Department of Information and Computer Science, King Fahd University of Petroleum and Minerals · SDAIA-KFUPM Joint Research Center for Artificial Intelligence
How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study
May 16, 2026Shuqi Zhu, Yi Zhong, Ziyi Ye +4Hallucination Detection Hallucination in Language Models
Department of Computer Science and Technology, Tsinghua University, Beijing, China · Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China
LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning
May 15, 2026Shradha Agarwal, Deepak Rajbhar, Tariq JMathematical Reasoning Benchmarks LLM Evaluation
Department of Nuclear Engineering and Computer Science Missouri University of Science and Technology Rolla, Missouri
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
May 14, 2026Wei Ding, Yilin Li, Yudong Zhang +4Vision-Language Models LLM Hallucination Mitigation
Tsinghua University Beijing, China · Tsinghua University, Tencent Beijing, China · Tencent Beijing, China +2
Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution
May 14, 2026Tian Qin, Junzhe Chen, Yuqing Shi +3Efficient VLM Inference Hallucination in Language Models
Tsinghua University · The University of Sydney · Stanford University +1
When Answers Stray from Questions: Hallucination Detection via Question-Answer Orthogonal Decomposition
May 14, 2026Siyang Yao, Erhu Feng, Yubin XiaHallucination in Language Models LLM Hallucination Detection
Shanghai Jiao Tong University Shanghai,China
Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometry
May 13, 2026Tyler Alvarez, Ali BaheriLanguage Model Error Detection Hallucination Detection
Rochester Institute of Technology, Rochester, NY, USA.
Reducing Hallucination in Vision-Language Models via Stage-wise Preference Optimization under Distribution Shift
May 13, 2026Qinwu XuVLM Robustness Hallucination in Language Models
Meta AI
REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
May 12, 2026Buyun Liang, Jinqi Luo, Liangzu Peng +6Adversarial Prompt Generation Adversarial Attacks
University of Pennsylvania
Scalable Token-Level Hallucination Detection in Large Language Models
May 12, 2026Rui Min, Tianyu Pang, Chao Du +2LLM Evaluation Hallucination in Language Models
Sea AI Lab · Hong Kong University of Science and Technology · 1Sea AI Lab 2Hong Kong University of Science and Technology +2
Instruction Lens Score: Your Instruction Contributes a Powerful Object Hallucination Detector for Multimodal Large Language Models
May 12, 2026Runhe Lai, Xinhua Lu, Yanqi Wu +3Hallucination Detection in VLMs Hallucination Detection
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Peng Cheng Laboratory, Shenzhen, China · Key Laboratory of Machine Intelligence and Advanced Computing, MOE, Guangzhou, China.
OptArgus: A Multi-Agent System to Detect Hallucinations in LLM-based Optimization Modeling
May 12, 2026Zhong Li, Zihan Guo, Xiaohan Lu +5LLM Auditing Hallucination in Language Models
♣Great Bay University · ▲Peking University · ♡Jilin University +4
When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
May 12, 2026Fanpu Cao, Xin Zou, Xuming Hu +1Hallucination Detection in VLMs Hallucination in Language Models
Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights
May 11, 2026Wenbo Chen, Veena Padmanabhan, Tootiya Giyahchi +2Hallucination Detection Hallucination in Language Models
Amazon · Carnegie Mellon University
A Theory of Time-Sensitive Language Generation: Sparse Hallucination Beats Mode Collapse
May 11, 2026Atul Ganju, Travis McVoy, Shaddin Dughmi +1Hallucination in Language Models Language Modeling
University of Southern California
Not Blind but Silenced: Rebalancing Vision and Language via Adversarial Counter-Commonsense Equilibrium
May 11, 2026Qingxin Xiao, Peilin Zhao, Yangyang Zhao +2VLM Robustness Hallucination in Language Models
South China University of Technology · Institute for Super Robotics (Huangpu) · Shanghai Jiao Tong University +1
Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination
May 11, 2026Yangneng Chen, Junlin Li, Weijun Yao +4Hallucination in Language Models VLM Hallucination
1Harbin Institute of Technology (Shenzhen), China · 2Huawei Technologies Co., Ltd. · 3The Hong Kong Polytechnic University +1

May 20, 2026, cs.CVJ/K move · Enter open · S save

Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens

Meng Shen, Minghao Wu, Deepu Rajan

Nanyang Technological University · Monash University

Vision-Language Models Hallucination in Language Models Large Vision-Language Models Object Hallucination in VLMs Adaptive Loss Weighting VLM Hallucination Mitigation

Open paper AskPDF arXiv

Object hallucination is a significant challenge that hinders the application of large vision-language models (LVLMs) in practice. We hypothesize that one possible origin of hallucination is the model's tendency to prioritize text generation over meaningful interaction with images. To explore this, we examine the generation process and categorize text tokens into three groups: image-positive, invariant, and negative, based on their visual dependence on input image tokens. Our analysis reveals that most generated tokens are minimally influenced by the image information. This suggests that during the model's training stage, more emphasis is placed on learning how to follow textual instructions, rather than extracting information from images. Based on this finding, we propose adjusting the training weights of different tokens depending on their visual dependence to control hallucination. Additionally, we remove a portion of the training data that potentially contains more hallucinations as a data filtering strategy. Both methods achieve a reduction in hallucination without compromising response length or introducing additional computational costs during inference. We validate our methods across three LVLM variants, demonstrating the effectiveness and general applicability.

Momentum

Related topics

Latest papers 187