VLM Interpretability

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 158

All topics

Compositional Failure in Audio-Visual LLMs: Late-Layer Prior Dominance Under Cross-modal Conflict
Aug 27, 2026Adarsh Sudheer, David Li, Omar Elbanna +3VLM Interpretability Audio-Visual Alignment
Independent Researcher.
TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint
Aug 13, 2026Fnu Pramono, John Cai, Sourabh KulkarniVLM Evaluation Vision-Language Models
Meta Superintelligence Labs · Reflection AI
When Visual Signals Mislead: A Mechanistic Study of Attribute Hallucination in Vision-Language Models
Aug 11, 2026Yufei Zhang, Chenlu Zhan, Hongwei WangHallucination Detection in VLMs VLM Adaptation
Zhejiang University
Where To Look? : Causal Tracing of Vision Encoders in VLM
Aug 11, 2026Naren Kumar S, Tirth Bhatt, Mayank SinghLarge Vision-Language Models VLM Interpretability
LINGO Research Group, Indian Institute of Technology Gandhinagar, India
From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodability
Aug 9, 2026Alexander Hackett, Arnaud Denis-Remillard, Axel CassouVLM Interpretability Vision-Language-Action Models
New York University · Reflex · Université de Montréal +1
Not All Redundant Tokens Are Alike: Analyzing Visual Token Pruning through Token Roles
Aug 5, 2026Hyeonyu Kim, Sehwan Lim, Youngwon Choi +2Efficient VLM Inference VLM Interpretability
Maum AI Inc., Republic of Korea
Attention is Case-Sensitive
Aug 4, 2026Maximilian Dillitzer, Tin Stribor Sohn, Jason J. Corso +1LLM Interpretability VLM Interpretability
University of Applied Science Esslingen · Dr. Ing. h.c. F. Porsche AG · Karlsruhe Institute of Technology +1
Mechanistic Interpretability-Guided Selective Fine-Tuning of Vision-Language Models for Centimeter-Level Flood Depth Estimation
Aug 2, 2026Nafis Fuad, Xiaodong Qian, Dongxiao ZhuVLM Adaptation Depth Estimation
Department of Civil & Environmental Engineering Wayne State University Detroit, MI, USA · Department of Civil & Environmental Engineering Institute for AI and Data Science (AIDaS) Wayne State University Detroit, MI, USA · Department of Computer Science, Institute for AI and Data Science (AIDaS) Wayne State University Detroit, MI, USA
Through the LENS: Local Geometric Decomposition of Vision-Language Model Representations
Aug 1, 2026Shalom Kachko, Raz Lapid, Margarita Vald +2Vision-Language Models VLM Interpretability
1Ben-Gurion University of the Negev · 2Intuit Inc. · 3Deepkeep
DiffuseAgent-MI: Distributionally-Grounded,Tool-Integrated Self-Evolving Agents for Faithful Visual Reasoning
Aug 1, 2026An Lanji, Dawei Liu, Jin Li +3AI Agent Reliability VLM Interpretability
University of Electronic Science and Technology of China, Chengdu, China.
Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications
Jul 29, 2026Piyush Jain, Kousik Dasgupta, Rajarshi Roy +1Visual Spatial Reasoning 3D Spatial Reasoning
Heritage Institute of Technology, Kolkata, India · Kalyani Government Engineering College, Kalyani, India · IAIRO, India +1
Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels
Jul 27, 2026Zhuchenyang Liu, Yao Zhang, Yu XiaoVLM Hallucination VLM Interpretability
Aalto University · Espoo, Finland
Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing
Jul 25, 2026Jakub Rymarski, Adam Rempała, Bartłomiej Sobieski +1VLM Interpretability Shapley Value Attribution
University of Warsaw, Poland · Centre for Credible AI, Warsaw University of Technology, Poland
Sparse Concept Channels in Frozen 3D CT Vision Encoders
Jul 23, 2026Farhad Nooralahzadeh, Lea Bogensperger, Christian Bluethgen +1Radiology Report Generation 3D Medical Imaging
Department of Quantitative Biomedicine, University of Zurich, Switzerland · Center for Artificial Intelligence in Medicine and Imaging, Stanford University, USA
Linguistic Context Recodes Visual Representations in Vision-Language Models
Jul 21, 2026Brian Song, Michael A. Lepori, Ellie PavlickVision-Language Models VLM Interpretability
Brown University
Attributes Should Come from Images, Not Class Names: Distribution-Conditioned Attribute Selection for Vision-Language Models
Jul 21, 2026Gautam Rajendrakumar Gare, Jia Shi, Zhiqiu Lin +3Vision-Language Models Distribution Shift
Carnegie Mellon University, USA
Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs
Jul 20, 2026Binesh Sadanandan, Vahid BehzadanVLM Evaluation VLM Interpretability
SAIL Lab, University of New Haven, West Haven, CT, USA
Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs
Jul 17, 2026Maeve Hutchinson, Abderrahmane Wassim Mehdaoui, Pranava MadhyasthaData Visualization Visual Attention
City St George’s · City St George’s, University of London · The Alan Turing Institute
How Do VLMs Fail? Vision-Operation Misalignment in Compositional VQA
Jul 17, 2026Navya Gupta, Bingjie Xu, Avinash Anand +2Visual Question Answering VLM Interpretability
Singapore Institute of Technology Singapore · NVIDIA Singapore
What Keeps Vision-Language Models Looking at the Image?
Jul 14, 2026Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi +3Vision-Language Models VLM Interpretability
The University of Tokyo
What Does a Temporal Benchmark Score Measure? Decomposing Channel Use in Video VLM Evaluation
Jul 14, 2026Farrukh RahmanVLM Evaluation Temporal Reasoning
Microsoft · Georgia Institute of Technology
The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning
Jul 13, 2026Wencheng Ye, Yi Bin, Yujuan Ding +7VLM Interpretability Multimodal Reasoning
School of Computer Science and Technology, Tongji University · School of Fashion and Textiles, The Hong Kong Polytechnic University · School of Artificial Intelligence, Shanghai Jiao Tong University
Mixture of Cognitive Experts in Large Vision-Language Models
Jul 12, 2026Robert Wijaya, Ngai-Man CheungVision-Language Models Large Vision-Language Models
Singapore University of Technology and Design (SUTD), Singapore
The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs
Jul 10, 2026Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQuabeh +2Vision-Language Models VLM Interpretability
MBZUAI · Imperial College London
When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalities
Jul 9, 2026Weiduo Liao, Yunqiao Yang, Ying WeiVLM Interpretability Sparse Autoencoders
Zhejiang University · Nanyang Technological University
A Good Initialization is All You Need for Faithful Visual Attribution
Jul 7, 2026Zihan Gu, Jiayu Wang, Hua Zhang +1Feature Attribution Perturbation-Based Feature Attribution
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences
Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
Jul 7, 2026Yoav Baron, Sara Dorfman, Roni Paiss +2Visual Place Recognition VLM Interpretability
Tel Aviv University, Tel Aviv, Israel · Google DeepMind.
Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History
Jul 3, 2026Chih-Ting Liao, Xin CaoVLM Interpretability Vision-Language-Action Models
University of New South Wales
Brand-as-Memory: Vision-Language Models Encode Causal, Mechanistically Localizable Credibility Priors for News Sources
Jul 3, 2026Chih-Ting Liao, Xin CaoVLM Evaluation VLM Interpretability
University of New South Wales
Pathways of Visual Information Flow in Vision-Language Models
Jul 3, 2026Israfel Salazar, Stella Frank, Dan Oneata +2Vision-Language Models VLM Interpretability
University of Copenhagen · 2Technical University of Denmark · 3POLITEHNICA Bucharest +1

Aug 27, 2026, cs.CLJ/K move · Enter open · S save

Compositional Failure in Audio-Visual LLMs: Late-Layer Prior Dominance Under Cross-modal Conflict

Adarsh Sudheer, David Li, Omar Elbanna, Ishaan Kodarapu+2

Independent Researcher.

VLM Interpretability Audio-Visual Alignment Multimodal Learning

Open paper AskPDF arXiv

We study audio-visual conflict as a compositional generalization test for AV-LLMs: the model must combine synchronized but semantically incompatible audio and video evidence and decide whether the pair matches. On VideoLLaMA 2-7B-AV, three alignment configurations remain nearchance on the scored exact-string Yes/No subset of AVHBench, even though their output priors shift substantially. Similarly, off-the-shelf InternVideo2 experienced a 32.3% accuracy decrease specifically under cross-modal conflict, accompanied by a 17.3% instruction-following failure. We call this failure mode prior dominance: late-layer commitment to an internally preferred answer pattern that is weakly grounded in the conflicting inputs. To explain this behavior, we conduct a mechanistic interpretability analysis and find that commitment remains concentrated at 25.5

\pm

1 layers. We show that stronger temporal alignment changes answer bias, but do not improve compositional conflict resolution. Code and data to reproduce our mechanistic audit and behavioral evaluations are available at https://github.com/AdarshSudheer09/AVHBench-dmai.

Momentum

Related topics

Latest papers 158