Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 491

All topics

Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models
Jul 6, 2026Khang Nhat Hoang Vo, Artem Vazhentsev, Artem Shelmanov +2VLM Evaluation Vision-Language Models
1MBZUAI · 2The University of Melbourne
IRIS: An Intelligent Vision-Language System for Ocular Surface Diseases via Topic Tree and Scene-Driven VQA Generation
Jul 5, 2026Hao Wei, Wenjin Qi, Dasen Dai +2Visual Question Answering Vision-Language Models
The Chinese University of Hong Kong, Shatin, Hong Kong, China · Westlake University, Xihu District, Hangzhou, Zhejiang Province, China
RADIO1D: Elastic Representations for Condensed Vision Modeling
Jul 3, 2026Greg Heinrich, Mike Ranzinger, Collin McCarthy +6Vision-Language Models Efficient VLM Inference
1NVIDIA
Pathways of Visual Information Flow in Vision-Language Models
Jul 3, 2026Israfel Salazar, Stella Frank, Dan Oneata +2Vision-Language Models VLM Interpretability
University of Copenhagen · 2Technical University of Denmark · 3POLITEHNICA Bucharest +1
VISTA: Auditing Semantic Divergence in Vision-Language Models
Jul 3, 2026Junchi Liao, Jiawen Deng, Fuji RenVLM Evaluation Vision-Language Models
University of Electronic Science and Technology of China
Prior Bias in Vision Language Models on UML Diagram Interpretation
Jul 3, 2026Zaiyu Cheng, Khai-Nguyen Nguyen, Antonio MastropaoloVLM Evaluation Software Engineering
Dept. of Computer Science William & Mary, USA
Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning
Jul 2, 2026Liyan Tang, Fangcong Yin, Greg DurrettVLM Robustness Vision-Language Models
The University of Texas at Austin · New York University
Show Me Examples: Inferring Visual Concepts from Image Sets
Jul 2, 2026Nick Stracke, Kolja Bauer, Stefan Andreas Baumann +3Vision-Language Models Conditional Image Generation
Munich Center for Machine Learning · Apple
FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval
Jul 2, 2026Zhenqi He, Ziqi Jiang, Yuanpei Liu +3Vision-Language Models Composed Image Retrieval
The Hong Kong University of Science and Technology, Hong Kong SAR · The University of Hong Kong, Hong Kong SAR
MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding
Jul 2, 2026Wenda Wang, Yihan Tong, Yuwei Hu +1Vision-Language Models Vision-Language Alignment
Renmin University of China · Beijing
SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models
Jul 2, 2026Qi Lyu, Jiahua Dong, Baichen Liu +7Model Compression Vision-Language Models
State Key Laboratory of Robotics and Intelligent Systems · Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences +2
LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression
Jul 2, 2026Bowen Yuan, Zijian Wang, Yadan Luo +2Vision-Language Models Large Vision-Language Models
The University of Queensland
Disentangling Pictorial Cue Understanding from Language Bias in VLMs via Depth Ordering Task
Jul 1, 2026Yiqian Liu, Iuliia Kotseruba, John K. TsotsosVLM Evaluation Vision-Language Models
York University, Toronto, Canada · University of Guelph, Guelph, Canada
Latent Visual Cache for Video Reasoning
Jul 1, 2026Yongheng Zhang, Zhipeng Xu, Hao Wu +4Memory-Augmented VLMs Vision-Language Models
1Tencent Youtu Lab · 2Tsinghua University · University of Illinois at Chicago
AdaBoosting Text Prompts for Vision-Language Models
Jul 1, 2026Seokhee Jin, Changhwan Sung, Sunung Mun +2Vision-Language Models Ensemble Learning
KT Corporation, Republic of Korea · Pohang University of Science and Technology (POSTECH), Republic of Korea · National AI Research Lab, Republic of Korea
Caption Bottleneck Models
Jul 1, 2026Seref Baris Cagliyan, Umut Ozdemir, Merve Tapli +1Vision-Language Models Concept Bottleneck Models
Dept. of Computer Eng., Middle East Technical University (METU) · Robotics & AI Center (ROMER), METU
HyFL-CLIP: Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding
Jul 1, 2026Ji Ha Jang, Hayeon Kim, Chulwon Lee +2VLM Robustness Vision-Language Models
1Dept. of Electrical and Computer Engineering, 2IPAI, 3INMC & AIIS, Seoul National University, Republic of Korea
From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA
Jun 29, 2026Sena Korkut, María Alejandra Bravo Sarmiento, Sanghwan Kim +1VLM Evaluation Vision-Language Models
Technical University of Munich, Munich, Germany · Helmholtz Zentrum München, Munich, Germany · Munich Center for Machine Learning, Munich, Germany
SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation
Jun 29, 2026Filippo Ruffini, Marco Salmé, Rosa Sicilia +2VLM Evaluation Radiology Report Generation
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Universit`a Campus Bio-Medico di Roma, Rome, Italy · Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Ume˚a University, Ume˚a, Sweden · UniCamillus-Saint Camillus International University of Health Sciences, Rome, Italy
RiverONE: Generating Knowledge-Intensive VLM by Simulated Quantum Machines
Jun 29, 2026Xindian Ma, Xinyu Long, Yefei Zhang +11Vision-Language Models Quantum-Inspired ML
Thewake RiverYtz Lab
FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models
Jun 28, 2026Yichen Guo, Kai Tang, Fenglai Lin +5Vision-Language Models Large Vision-Language Models
1Nanyang Technological University · 2State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · 3Tsinghua University
Dynamic Parsing and Updating Natural Language Specification using VLMs for Robust Vision-Language Tracking
Jun 28, 2026Xiao Wang, Liye Jin, Dan Xu +5Vision-Language Models Visual Object Tracking
School of Computer Science and Technology, Anhui University, Hefei 230601, China · School of Electronic and Information Engineering, Anhui University, Hefei 230601, China · Harbin Institute of Technology, Shenzhen, China +3
ComMem: Complementary Memory Systems for Test-Time Adaptation of Vision-Language Models
Jun 27, 2026Guanglong Sun, Shuang Cui, Bo Lei +6Memory-Augmented VLMs Vision-Language Models
School of Life Sciences, IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China · Institute of Software Chinese Academy of Sciences, Beijing, China · 3Beijing Academy of Artificial Intelligence, Beijing, China +2
MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving
Jun 26, 2026Nan Yang, Zhanwen Liu, Linfeng Zhang +4Vision-Language Models VLMs for Autonomous Driving
School of Information Engineering, Chang’an University, China · School of Artificial Intelligence, Shanghai Jiaotong University, China
From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
Jun 24, 2026Haoxiang Sun, Tao Wang, Li Yuan +2Vision-Language Models Multimodal Large Language Models
School of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu, 610065, Sichuan, China · School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School, 2199 Lishui Road, Shenzhen, 518055, Guangdong, China · Institute of Artificial Intelligence (TeleAI), China Telecom and Northwestern Polytechnical University, 127 Youyi West Road, Xi’an, 710072, Shaanxi, China
What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs
Jun 24, 2026Ziran Li, Qiang Wang, Zhengyu Chen +4Vision-Language Models VLM Evaluation
Meituan · Tsinghua University
Do vision-language models search like humans? Reasoning tokens as a reaction-time analog in classic visual-search paradigms
Jun 23, 2026Farahnaz WickVLM Evaluation Vision-Language Models
1Independent Researcher
Data Selection Through Iterative Self-Filtering for Vision-Language Settings
Jun 22, 2026Andrei Liviu Nicolicioiu, Sarvjeet Singh Ghotra, Morgane M. Moss +1Vision-Language Models Training Data Curation
Mila, Université de Montréal
FlowTrain: Flow-Based Decoupled Training for Industrial-Grade Vision-Language Models
Jun 22, 2026Zhida Jiang, Zhaolong Xing, Yang Pei +14Vision-Language Models
Huawei
Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation
Jun 22, 2026Yu Cao, Ziquan Liu, Zhensong Zhang +3VLM Evaluation Physical Consistency in Video Generation
Queen Mary University of London · Huawei Darwin Research Center · Imperial College London

Jul 6, 2026, cs.CVJ/K move · Enter open · S save

Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models

Khang Nhat Hoang Vo, Artem Vazhentsev, Artem Shelmanov, Timothy Baldwin+1

1MBZUAI · 2The University of Melbourne

VLM Evaluation Vision-Language Models

Open paper AskPDF arXiv

Vision-language models (VLMs) perform well on visual question answering with high-quality images but struggle when questions require knowledge beyond what is clearly and directly visible. In such settings, uncertainty quantification should not only indicate whether the model is likely to fail but also diagnose why it is uncertain, across dimensions such as perception, entity recognition, and knowledge retrieval. While prior work has focused on individual failure modes in isolation or treated incorrect answers as monolithic failures, we propose a unified framework for disentangling these failure modes and investigate whether pre-generation signals can predict these failure sources. Across a range of datasets and model families, we find a consistent pattern in VLM errors: some failures arise from visual or recognition bottlenecks, while others persist after the relevant entity is identified. Our main finding is that these failure sources can be predicted before decoding: recognition-related failures are best captured by visual-token representations, while failures that remain after recognition are better captured by prompt-conditioned hidden states. This pre-generation signal enables efficient failure-source prediction before the model produces an answer, allowing uncertain cases to be routed to targeted interventions such as image repair, entity recognition support, or external retrieval.

Momentum

Related topics

Latest papers 491