VLM Interpretability

VLM: Vision-Language Model

Overview

Vision-language model interpretability studies how models encode visual and textual information, combine it, and use it to produce answers, predictions, or actions. Researchers use saliency and attribution maps, probing and sparse autoencoders, and mechanistic methods such as activation patching and circuit analysis to locate relevant features, layers, and attention heads; causal interventions test whether those components actually affect behavior. Evaluations span image-text retrieval and classification, visual question answering, grounding, counting, spatial reasoning, and hallucination benchmarks such as POPE and CHAIR. A central tension is that information may be decodable or highlighted by attention without guiding the model’s answer, so explanation faithfulness and actual use of visual evidence remain open challenges.

Latest papers

CardsList

275 papers13 papers in the last four weeks, down 66% on the four weeks before. 0.1% of all new papers.

  1. Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

    Sep 26, 2025Jiawei Liang, Jianjie Huang, Ruoyu Chen +4Feature AttributionMultimodal Large Language Models

  2. Explainable embeddings with Distance Explainer

    May 21, 2025Christiaan Meijer, E. G. Patrick BosFeature AttributionPerturbation-Based Feature Attribution

  3. Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection

    Apr 1, 2025Ruoyu Chen, Siyuan Liang, Jingzhi Li +4Feature AttributionVLM Interpretability