Multimodal Model Interpretability

Follow this topic Ask about this topic

It asks which modalities, regions, tokens, concepts, or internal circuits support a model’s output, helping people audit errors, shortcuts, bias, and safety. Common approaches include saliency and Shapley attribution, concept bottlenecks and sparse autoencoders, and probing or causal interventions that test what happens when evidence is altered. Researchers evaluate these explanations on visual question answering, hallucination and grounding tasks, cross-modal benchmarks, and clinical settings where inputs may be missing or incomplete. A central tension is that an explanation can look plausible without being faithful: models may encode useful evidence yet ignore it, favor one modality, or rely on shortcuts, and evaluation of genuine cross-modal reasoning remains difficult.

Latest papers

130 papers13 papers in the last four weeks, down 7% on the four weeks before. 0.1% of all new papers.

Beyond the Readout: A Latent Numerical Frame Shared Across Modality and Operation
Oct 27, 2026Oriol Juan Sabater, Alex Ferrando de las Morenas, Pau Rodríguez +1Multimodal Model Interpretability Multimodal Reasoning
Universitat Autònoma de Barcelona · Apple
DocXEval: A Systematic Evaluation of Explainability in Multimodal Document Understanding
Oct 27, 2026Saifullah Saifullah, David Dembinsky, Stefan Agne +2Multimodal Model Interpretability Explainability Evaluation
Smart Data and Knowledge Services (SDS), German Research Center for Artificial Intelligence (DFKI), Trippstadter Str. 122, Kaiserslautern, 67663, Germany. · Department of Computer Science, University of Kaiserslautern-Landau (RPTU), Erwin-Schrödinger-Str. 52, Kaiserslautern, 67663, Germany. · DeepReader GmbH, Kaiserslautern, 67663, Germany.
Reliability-Aware Short-Term Glycemic Risk Prediction Using an Anchor-Based Comparative Framework of Sequence, Multimodal, and State-Updating Digital-Twin Models
Oct 26, 2026Pir Bakhsh Khokhar, Carmine Gravino, Fabio PalombaDigital Twins Multimodal Model Evaluation
Department of Computer Science, University of Salerno, Fisciano, Italy
Faithful Explanations under Missing Modalities in Multimodal Dementia Prognosis
Oct 26, 2026Firuz KamalovFaithfulness of Language Model Explanations Multimodal Model Interpretability
Canadian University Dubai, City Walk, P.O. Box 117781, Dubai, United Arab Emirates
All the Facts, Still the Wrong Answer: Integration Gaps in Vision-Language Models
Oct 24, 2026Yuyang Dai, Yuxia Wang, Zhuohan XieVLM Evaluation Compositional Reasoning in VLMs
Florida State University · INSAIT, Sofia University “St. Kliment Ohridski” · MBZUAI
A Broader Look into MLLM's Retrieval Heads
Oct 23, 2026Jia Wang, Yuqing Li, Jiangnan Li +2Multimodal Large Language Models Multimodal IR
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Tencent
CRANE: Post-Hoc Counterfactual Retrieval for Multimodal Classifiers
Oct 22, 2026Franco Rugolon, Ioanna Miliou, Panagiotis PapapetrouCounterfactual Explanations Multimodal Model Interpretability
Dept. of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden
I $^2$ -KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis
Oct 22, 2026Xiaoting Xie, Feng Qiao, Xianwei Zhang +5Multimodal Sentiment Analysis Multimodal Fusion
School of Information Science and Engineering, Shandong University, Qingdao, Shandong, China · Department of Computer Science, University of Exeter, Exeter, UK · School of Computing Science, University of Glasgow, Glasgow, UK +1
Encoder-Free Multimodal LLMs Learn Implicit Encoders
Oct 16, 2026David Harwath, Desh Raj, Xavier Velez +5Multimodal Large Language Models Multimodal Model Interpretability
The University of Texas at Austin · NVIDIA · Johns Hopkins University +2
Omni-Modal LLMs Know More Than They Say
Oct 12, 2026Na Min An, Taekyung Kim, Jaemyung Yu +2Multimodal Large Language Models Cross-Modal Alignment
KAIST AI · NAVER AI Lab
S-Space: Exploring Spatial Workspace in Multimodal Language Models
Oct 9, 2026Zhiyuan Shi, Xiang Li, Xingzhuo Guo +8Spatial Reasoning in Language Models Spatial Reasoning
OA-MAP: Evidence-Grounded Multi-Agent Multimodal Framework for Interpretable Knee Osteoarthritis Progression
Oct 8, 2026Sixu Chen, Mingrui Yang, Qiang Guan +1Clinical Outcome Prediction Multimodal Agents
Cleveland Clinic, Cleveland, OH, USA
From Sparse Representations to Behavioral Insights for Multimodal Depression Assessment
Oct 8, 2026Guimin Hu, Zihao Song, Jiachen Luo +2Depression Detection Multimodal Model Interpretability
Guangdong University of Technology · Queen Mary University of London · The Hong Kong Polytechnic University
Multimodal Safety Evaluation Should Measure Controllability Beyond Classification
Oct 5, 2026Junhyeong Park, Hanwool Lee, DongGeon Lee +4Multimodal Model Interpretability Multimodal Model Evaluation
KAIST · AIM Intelligence · Seoul National University +3
Architecture-Dependent Fusion Pathways in MLLMs
Oct 2, 2026Hebao Zhu, Dongxia WuMultimodal Large Language Models Multimodal Fusion
Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, UAE
Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning
Sep 29, 2026Bowen Yuan, Danny Wang, Ruihong Qiu +2Large Vision-Language Models VLM Interpretability
The University of Queensland
Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs
Sep 29, 2026Yueran Ma, Ronghao LinAudio QA Multimodal Large Language Models
The University of Queensland, Brisbane, Australia · Shenzhen University, Shenzhen, China
T-MoXAI: A Hierarchical Explainability Framework for Temporal Multimodal Data
Sep 27, 2026Ali Inha, Mo Vali, Saaliha Vali +2Explainable Time Series Forecasting Multimodal Model Interpretability
Department of Computer Science, University of Cambridge, Cambridge, UK · Cavendish Laboratory, Department of Physics, University of Cambridge, Cambridge, UK · Imperial College Healthcare NHS Trust, London, UK +1
Virtual Encoders in Multimodal Transformers
Sep 22, 2026Katsuya Ogata, Yuta NakashimaVisual Representation Learning Multimodal Large Language Models
The University of Osaka
FCA-Guided Counterfactual Explanations for Multi-Modal Breast Cancer Diagnosis: A Framework Achieving Perfect Validity with Emergent Sparsity
Sep 17, 2026Abdullahi Isa, Souley Boukari, Muhammad AliyuBreast Cancer Counterfactual Explanations
Department of Computer Science, Faculty of Computing, Abubakar Tafawa Balewa University, Bauchi, Nigeria.
What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes
Sep 15, 2026Zhipeng Zhao, Wenxu Wang, Peishun Liu +1Hallucination Detection in VLMs Multimodal Grounding
1Ocean University of China
IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies
Sep 14, 2026Jinwoong Kim, Sangjin ParkRobotic Manipulation Vision-Language-Action Models
Department of Industrial Data Engineering, Hanyang University, Seoul, Republic of Korea.
A Multimodal Explainable Deep Learning Framework for Alzheimer's Disease Diagnosis using 3D Magnetic Resonance Imaging and Clinical Data
Sep 11, 2026Yusuf Brima, Marcellin Atemkeng, Lakshmana Rao Namamula +13D Medical Imaging Multimodal Classification
Computer Vision, Institute of Cognitive Science, Osnabrück University, Germany. · Department of Mathematics, Rhodes University, South Africa. · National Institute for Theoretical and Computational Sciences (NITheCS), Stellenbosch, 7600, South Africa. +2
LatentVerse: A Framework for Understanding Shared and Modality-Specific Information in Multimodal Latent Representations
Sep 11, 2026Majd Alafrange, Samuel Friedman, John Kitonyo +2Multimodal Embedding Multimodal Model Interpretability
Machine Learning for Health (ML4H), Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA · Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA
Toward Interpretable Multimodal Fusion: Heat Conduction Modeling for Hyperspectral and LiDAR Joint Classification
Sep 10, 2026Kan Wei, Jiahui Cui, Jing Yao +3Multimodal Model Interpretability Hyperspectral Image Classification
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China · Key Laboratory of Target Cognition and Application Technology, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China +3
The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models
Sep 10, 2026Pengfei Zhang, Biao Tian, Xiangang Li +1Neural Decoding Multimodal Foundation Models
AI Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Alibaba Token Hub, Alibaba Group, Beijing, China.
(How) Do MLLMs Report Bistable Images Like Humans?
Sep 6, 2026Ryota Takatsuki, Tomoki Doi, Amane Watahiki +2Multimodal Large Language Models Multimodal Model Interpretability
Sussex Centre for Consciousness Science, University of Sussex · AI Alignment Network · The University of Tokyo +3
SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis
Sep 4, 2026Yuqing Yang, Alexander Schmatz, Zhaozhao Ma +3Multimodal Model Interpretability Medical Diagnosis
Independent Researcher · Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands · Quantitative Data Analytics Group, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands +3
Capsule Lens: Locating and Tracking Concept Geometry in Model Representations
Sep 4, 2026Yiming Tang, Harshvardhan Saini, Samyak Jha +3Neural Network Interpretability Neural Representation Geometry
Indian Institute of Technology (ISM) Dhanbad · The Chinese University of Hong Kong · National University of Singapore
Towards reliable multimodal disaster severity assessment through preference optimization and explainable vision-language reasoning
Sep 1, 2026Yuanjun Zhang, Fuzel Ahamed Shaik, Suvojit Acharjee +2Disaster Damage Assessment Multimodal Classification
Centre for Machine Vision & Signal Processing, University of Oulu, Pentti Kaiteran katu 1, PO Box 8000, Oulu, 90570, Finland · Department of CSE and Institute of Engineering and Management, 120 SDF Building, Saltlake Electronics Complex, Kolkata, 700091, India

Oct 27, 2026J/K move · Enter open · S save

Beyond the Readout: A Latent Numerical Frame Shared Across Modality and Operation

Oriol Juan Sabater, Alex Ferrando de las Morenas, Pau Rodríguez, Xavier Suau

Universitat Autònoma de Barcelona · Apple

Multimodal Model Interpretability Multimodal Reasoning Mechanistic Interpretability

Open paper AskPDF arXiv

When a multimodal model computes 17+35 from text or 91-39 from a rendered image, it produces the same answer, 52. Does it use one internal concept of number for both, or separate machinery for each path? Similarity metrics such as RSA or CKA cannot settle this question. They give the same score to identical and to rotated directions, and two representations can look alike only because both prepare the same output digits. We therefore use causal interventions. For four conditions (text or image, addition or subtraction), we learn a small subspace of Gemma4-12B-it's residual stream in which swapping coordinates swaps the model's answer. The subspaces answer three questions. First, do the conditions write numbers along the same directions? They do not. Reusing a subspace from text in an image on the same operation recovers only 53.2% of its causal effect. A single rotation and rescaling maps one subspace onto the other and raises recovery to 91.4%, so the conditions share one geometry expressed in different coordinates. Second, does this shared geometry only reflect the digit the model is about to print? To test this, we split each subspace into 9 directions that the output layer reads to choose the next digit and 13 directions that it cannot see. The 13 hidden directions still carry the number, yet changing them barely alters the digit being written. Third, what are these hidden directions for? A model must emit a multi-digit answer one digit at a time, so it can compute the full result early and keep it in reserve. When two numbers share their first digit, swapping the number in the hidden directions leaves the digit being written intact and redirects the next digit in 77.6% of cases. The model thus forms an abstract numerical concept upstream and unfolds it across time: multimodal representations share meaning through relational geometry across distinct coordinate frames.