VLM Hallucination

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 58

All topics

PAIQ: Patch-Aligned Semantic Injection via Residual Rotation
Sep 29, 2026Pinze Ren, Yuwei Zhang, Hao Chen +3Visual Question Answering Image Captioning
Tsinghua University · Nanyang Technological University
Before the Token Commits: Trajectory-Level Benchmarking of Visual Hallucinations in Diffusion VLMs
Sep 28, 2026Yadong Wang, Siping Yue, Yu Tian +2VLM Evaluation Vision-Language Models
Nanjing University of Aeronautics and Astronautics · State Key Laboratory of Ocean Sensing, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China · Security Capability Center
HALDETECT at ImageEval 2026 Shared Tasks: Answer-First Contrastive Grounding with QLoRA
Sep 12, 2026Syed Mohaiminul Hoque, Md Sakhawat HossainVLM Evaluation Hallucination Detection in VLMs
Independent Author
Reliability Challenges in Diffusion Vision-Language Models
Sep 1, 2026Md. Atabuzzaman, Chris ThomasVLM Evaluation VLM Robustness
Department of Computer Science Virginia Tech
VisER: Visual Evidence and Reliance for Object Hallucination Detection in LVLMs
Aug 31, 2026Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie +1Hallucination Detection in VLMs VLM Hallucination
School of Computing and Information Systems The University of Melbourne Melbourne, Australia
How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures
Aug 13, 2026Paul Osemudiame Oamen, Owusu-Banahene Osei, Ananya Mukherjee +4VLM Evaluation VLM Robustness
The Aberdeen NLP Research Group, University of Aberdeen, UK · International Institute of Information Technology Hyderabad, India · University of Technology Nuremberg, Germany +1
UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations
Aug 11, 2026Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca +4Hallucination Detection in VLMs VLM Hallucination
NVIDIA Research, Tel Aviv, Israel · Technion, Haifa, Israel · Bar-Ilan University, Ramat Gan, Israel +1
Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination
Aug 7, 2026Zichuan Wang, Songlin Yang, Bo Peng +4Hallucination Detection in VLMs Vision-Language Models
School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Hong Kong University of Science and Technology · 2New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Science
UHP Detection: LVLMs have their Unique Hallucination Pattern in the Consistency Space
Aug 4, 2026Amir Mohammad Ezzati, Kiyan Rezaee, Bardiya Kariminia +4Hallucination Detection in VLMs Vision-Language Models
Sharif University of Technology · Michigan State University · Shahid Beheshti University +2
Confident but Unreliable: A Behavioral Safety Audit of Vision-Language Models on Brain MRI
Aug 3, 2026Amir Sabbaghziarani, Mohammadsajad Abavisani, Sergey PlisVLM Evaluation VLM Hallucination
TReNDS Center Atlanta, GA, USA
Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking
Aug 2, 2026Timothee Mickus, Claudio Savelli, Eduardo Calò +10VLM Evaluation Hallucination Detection in VLMs
University of Helsinki
Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs
Jul 31, 2026Mingyu Wang, Weilin Jin, Wenbo Li +5Hallucination Detection in VLMs Attention Head Analysis
Peking University, Beijing, China · Joy Future Academy, Beijing, China
When Model Priors Conflict with Visual Evidence: Mitigating Commonsense-Driven Hallucinations by Selective Prior Calibration
Jul 31, 2026Kesheng Chen, Yamin Hu, Wenjian LuoVLM Hallucination VLM Calibration
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies Institute of Cyberspace Security School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China
Hearsay: Vision-Language Medical Diagnoses Without an Image
Jul 29, 2026Siddharth VohraVLM Evaluation Medical Image Analysis
Carnegie Mellon University Pittsburgh, PA, USA · Amazon Web Services AI Native Pittsburgh, PA, USA
Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels
Jul 27, 2026Zhuchenyang Liu, Yao Zhang, Yu XiaoVLM Hallucination VLM Interpretability
Aalto University · Espoo, Finland
When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documents
Jul 27, 2026Marina Gardella, Camilo Mariño, Diego Belzarena +3VLM Evaluation VLM Hallucination
CB · Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, France · UDELAR, CB +4
Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions
Jul 16, 2026Yijiang Li, Huiqi Zou, Bingyang Wang +1VLM Evaluation VLM Hallucination
UC San Diego · Northeastern University · Georgia Institute of Technology +1
HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models
Jul 8, 2026Feng He, Zhenting Wang, Qifan Wang +4VLM Evaluation VLM Hallucination
Purdue University · Rutgers University · Meta AI +2
Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty
Jun 26, 2026Xiao Song, Haonan Qin, Zhaoxu Zhang +3Hallucination Detection in VLMs VLM Hallucination
School of Intelligent Science and Technology, Nanjing University, Suzhou, China · School of Biomedical Engineering, Nanjing University, Suzhou, China · National Institute of Healthcare Data Science at Nanjing University, Nanjing University, Nanjing, China
A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy
Jun 23, 2026Aminu Lawal, Niyoj Oli, Sachin Acharya +3VLM Evaluation Hallucination Detection in VLMs
University of Aberdeen, Aberdeen, United Kingdom · Nepal Applied Mathematics and Informatics Institute for Research, Lalitpur, Nepal · West Virginia University, Morgantown, WV, United States
SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images
Jun 10, 2026Ruotian Zhang, Xiaoxiao Sun, Junzhe Huang +2VLM Evaluation VLM Hallucination
Zhejiang University · The University of Queensland
How Many Counterfactuals Does It Take? Probing VLM Hallucinations Through Circuits and Causal Effects
Jun 7, 2026Abhivansh Gupta, Simardeep Singh, Advika Sinha +2Hallucination Detection in VLMs VLM Robustness
Data Science Group, IIT Roor- kee, Roorkee, Uttarakhand, India.
Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation
Jun 2, 2026Ruipeng Zhang, Zhihao Li, C. L. Philip Chen +1VLM Hallucination Large Vision-Language Models
Guangdong Provincial Key Laboratory of Computational AI Models and Cognitive Intelligence, School of Computer Science & Engineering, South China University of Technology, Guangzhou, China · Engineering Research Center of the Ministry of Education on Health Intelligent Perception and Paralleled Digital-Human, Guangzhou, China · Pazhou Lab, Guangzhou, China.
What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness
May 29, 2026Yusheng He, Jizhe Zhou, Xia Du +3VLM Evaluation VLM Robustness
School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University, Chengdu, 610020, China · School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, 361000, China · Department of Electrical and Computer Engineering, University of Hong Kong, Pok Fu Lam, Hong Kong, China +1
Risk-aware Selective Prompting for Hallucination Mitigation in Large Vision-Language Models
May 27, 2026Yuang Huang, Yafeng Zhang, Yu ZilanVision-Language Models Hallucination in Language Models
Shanghai Jiao Tong University · iFLYTEK · Tsinghua University
Reading or Guessing? Visual Grounding Failures of Vision-Language Models for OCR in Ancient Greek Editions
May 26, 2026Antonia Karamolegkou, Nicolas Angleraud, Benoît Sagot +1VLM Evaluation Document Understanding
Inria, Paris, France
Hallucination Behavior in Multimodal LLMs Across Agricultural Image Interpretation and Generation Tasks
May 26, 2026Partho Ghose, Al Bashir, Prem Raj +1Agricultural Image Analysis VLM Hallucination
Department of Biological & Agricultural Engineering, Texas A&M AgriLife Research, Texas A&M University System, Dallas, TX 75252, USA.
Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigation
May 25, 2026Ruoxi Cheng, Haoxuan Ma, Zhengfei Hai +6Adversarial Representation Learning Vision-Language Models
Fudan University · Tencent · Nanjing University +3
Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow
May 21, 2026Chengsheng Zhang, Chenghao Sun, Zhining Xie +1VLM Hallucination VLM Interpretability
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · AIPD, Tencent
Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
May 20, 2026Zhe Cheng, Wenyu Chen, Fode Zhang +1VLM Robustness Hallucination in Language Models
Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China. · Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore.

Sep 29, 2026, cs.CVJ/K move · Enter open · S save

PAIQ: Patch-Aligned Semantic Injection via Residual Rotation

Pinze Ren, Yuwei Zhang, Hao Chen, Linghao Meng+2

Tsinghua University · Nanyang Technological University

Visual Question Answering Image Captioning VLM Hallucination Vision-Language Alignment

Open paper AskPDF arXiv

Language-aligned and self-supervised visual encoders offer complementary strengths in semantic abstraction and spatial detail. Harnessing this complementarity requires enriching local features while retaining distinctions between semantically related patches. We introduce PAIQ, a patch-aligned semantic injection framework that combines content-based cross-encoder matching with orthogonally constrained residual updates. Using DINOv3 patch features as the spatial base, PAIQ aggregates complementary SigLIP features through joint source allocation and injects the aggregate--base differences through a shared orthogonal transformation Q. This rotation adapts update directions while preserving residual norms and pairwise angles. For fixed projected features, we derive conditions for patch separability under similar semantic aggregates and show that rotation adds a nonnegative separation term over direct interpolation when the aggregate is shared. Only the projection and fusion parameters are trained; both visual encoders and the language model remain frozen, and fusion retains 196 visual tokens. Across diverse language backbones, PAIQ yields broad gains in judge-assessed correctness and reductions in hallucination severity over single-encoder interfaces on image description and visual question answering. On the 2B and 9B Qwen backbones, this compact interface outperforms the strongest evaluated fusion or token-compression baselines by about 2.9 correctness points on average.

Momentum

Related topics

Latest papers 58