VLM Interpretability

VLM: Vision-Language Model

Overview

Vision-language model interpretability studies how models encode visual and textual information, combine it, and use it to produce answers, predictions, or actions. Researchers use saliency and attribution maps, probing and sparse autoencoders, and mechanistic methods such as activation patching and circuit analysis to locate relevant features, layers, and attention heads; causal interventions test whether those components actually affect behavior. Evaluations span image-text retrieval and classification, visual question answering, grounding, counting, spatial reasoning, and hallucination benchmarks such as POPE and CHAIR. A central tension is that information may be decodable or highlighted by attention without guiding the model’s answer, so explanation faithfulness and actual use of visual evidence remain open challenges.

Latest papers

CardsList

275 papers13 papers in the last four weeks, down 66% on the four weeks before. 0.1% of all new papers.

  1. Bits, Not Boxes: Label-Free Layer Selection for Visual Localization in VLMs

    Nov 17, 2026Haisheng Fu, Gexin Huang, Haikai Zhao +4Vision-Language GroundingVLM Interpretability

  2. How VLMs Plan What's Next

    Nov 16, 2026Rohit Gandikota, Arnab Sen Sharma, Adam Belfki +1VLM InterpretabilityReasoning for VLMs

  3. Do the Details Help the Whole? One-Way Transfer from Person Actions to Group Intent in VLMs on JRDB-Social

    Nov 12, 2026Chengguang Gan, Zhixi Cai, Yunhao Liang +3Intent ClassificationVLM Interpretability

  4. A General Concept-based Decomposition for Vision-Language Embeddings

    Nov 11, 2026Simone Alberto Peirone, Ortal Senouf, Francesca Pistilli +2Vision-Language ModelsVLM Interpretability

  5. Uncertainty, Not Concealment: A Mechanistic Reappraisal of Sycophancy in Vision-Language Models

    Nov 6, 2026Mohamed I. Eldagla, Zeyad H. ElShafei, Ahmed M. AnterSycophancy in VLMsRadiology VLMs

  6. Heads That Write, Not Just Point: Image Retrieval Heads in Vision-Language Models

    Nov 3, 2026Junsung Park, Sangwon Yu, Donghun Kang +6Large Vision-Language ModelsVLM Interpretability

  7. Label-Free Localization and Test-Time Suppression of CLIP's Typographic Circuits

    Nov 1, 2026Bohan Liu, Wenqian Ye, Guangzhi Xiong +3VLM RobustnessAdversarial Attacks on VLMs

  8. A Heart Means Recursion: Exploring Visual Meaning and Communication in VLMs

    Oct 27, 2026Alexa R. Tartaglini, Zhengxuan Wu, Thaddäus Wiedemer +3Emergent CommunicationVLM Interpretability

  9. What Sketches Tell Us about LVLMs: Conventions, Grounding, and Localisation

    Oct 25, 2026Chaitat Utintu, Ahmed Bourouis, Arkaprabha Basu +1Vision-Language ModelsVLM Interpretability

  10. Evaluating Visual Language Models on Visual Grounding for Visually Impaired People

    Oct 21, 2026Fairooz Azim, Lydia Achour, Claudia Borg +1Vision-Language GroundingVLM Evaluation

  11. Encoded but Unused: Bounding the Sources of VLM Errors

    Oct 21, 2026Anat Veikherman, Mor Ventura, Nitay Calderon +2VLM InterpretabilityVisual Evidence Sensitivity in VLMs

  12. VLMProbe: An Open-source Library for VLM Probing and Interpretation

    Oct 21, 2026Gexin Huang, Haisheng Fu, Yanting Yang +4VLM InterpretabilitySparse Autoencoders

  13. DICL-Qwen3VL: Decoupled Confidence Calibration for Interpretable Workwear Detection

    Oct 16, 2026Meng Wang, Wu Zeng, Mei Li +1VLM CalibrationVLM Interpretability

  14. Probing Negation Understanding in Vision-Language Models With Object Attention

    Oct 14, 2026Beth Pearson, Michael Wray, Martha LewisVLM ReasoningVLM Evaluation

  15. RL Teaches VLMs Where to Attend but Not How to Perceive

    Oct 14, 2026Ziqian Zhang, Junjie Liao, Zhining Zhang +1RL for VLMsVisual Evidence Sensitivity in VLMs

  16. Seeing Depth but Choosing Cues in Vision-Language Models

    Oct 13, 2026Zhengtao Zou, Pekka MarttinenVLM Interpretability3D VLMs

  17. When, Where, and Why: Hierarchical Causal Explainability for Video Face Understanding with MLLMs

    Oct 13, 2026Anubhav Kataria, Shreya Ghosh, Jingwen Ye +1VLM InterpretabilityFacial Expression Recognition

  18. Attention Sinks in VLMs Are Borrowed, Not Seen

    Oct 11, 2026Dengjia Zhang, Alexander Martin, Pengyu Yan +6Attention Intervention in VLMsVLM Interpretability

  19. [Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning

    Oct 10, 2026Anton Nuzhdin, Andrei Gruzitski, Balázs Egyed +1VLM InterpretabilitySaliency Map Evaluation

  20. Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models

    Oct 9, 2026Laura Yao, Shixing Yu, Jie Peng +3VLM InterpretabilityTraining Data Attribution

  21. When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors

    Oct 9, 2026Shubh Garg, Chen ShaniFaithfulness in VLMsVLM Evaluation

  22. UkrMeme: Multimodal Explainable Persuasion Detection in Ukrainian Memes

    Oct 9, 2026Nazar Andrushko, Vera Schmitt, Veronika SolopovaMultimodal Meme ClassificationVLM Evaluation

  23. When MLLMs miss what humans read: diagnosing and learning to read squint text

    Oct 9, 2026Chen Peng, Shan Huang, Jianbo Huang +3Multimodal Large Language ModelsOptical Character Recognition

  24. What Do Visual-Head Discovery Scores Actually Find? Attention Source, Necessity, and Random-Control Pitfalls in Vision-Language Models

    Oct 9, 2026Md Abdul Kadir, Daniel SonntagAttention Head AnalysisVision-Language Models

  25. Feature Competition and Its Representational Origins in Vision Language Models: Evidence from the Stroop Paradigm

    Oct 8, 2026Nurbüke Teker, Rui Xiao, Zeynep Akata +1Representation Geometry in VLMsVision-Language Models

  26. FearCaut-Qwen: Affective Steering in a Vision-Language Model Shifts the Decision Criterion for Hazard Assessment

    Oct 8, 2026Xiaoshan ZhouActivation Steering in VLMsDisaster Damage Assessment

  27. Visual Evidence Under Cross-Examination: Evaluating and Controlling Decision-Level Evidence Use in Vision-Language Models

    Oct 7, 2026Huiyao Zhang, Jin Bai, Zilong Su +6VLM EvaluationFaithfulness in VLMs