Multimodal Model Interpretability

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 267% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

Multimodal Safety Evaluation Should Measure Controllability Beyond Classification
Oct 5, 2026Junhyeong Park, Hanwool Lee, DongGeon Lee +4Multimodal Model Interpretability Multimodal Model Evaluation
KAIST · AIM Intelligence · Seoul National University +3
Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning
Sep 29, 2026Bowen Yuan, Danny Wang, Ruihong Qiu +2Large Vision-Language Models VLM Interpretability
The University of Queensland
Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs
Sep 29, 2026Yueran Ma, Ronghao LinAudio QA Multimodal Large Language Models
The University of Queensland, Brisbane, Australia · Shenzhen University, Shenzhen, China
T-MoXAI: A Hierarchical Explainability Framework for Temporal Multimodal Data
Sep 27, 2026Ali Inha, Mo Vali, Saaliha Vali +2Explainable Time Series Forecasting Multimodal Model Interpretability
Department of Computer Science, University of Cambridge, Cambridge, UK · Cavendish Laboratory, Department of Physics, University of Cambridge, Cambridge, UK · Imperial College Healthcare NHS Trust, London, UK +1
Virtual Encoders in Multimodal Transformers
Sep 22, 2026Katsuya Ogata, Yuta NakashimaVisual Representation Learning Multimodal Large Language Models
The University of Osaka
FCA-Guided Counterfactual Explanations for Multi-Modal Breast Cancer Diagnosis: A Framework Achieving Perfect Validity with Emergent Sparsity
Sep 17, 2026Abdullahi Isa, Souley Boukari, Muhammad AliyuBreast Cancer Counterfactual Explanations
Department of Computer Science, Faculty of Computing, Abubakar Tafawa Balewa University, Bauchi, Nigeria.
What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes
Sep 15, 2026Zhipeng Zhao, Wenxu Wang, Peishun Liu +1Hallucination Detection in VLMs Multimodal Grounding
1Ocean University of China
IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies
Sep 14, 2026Jinwoong Kim, Sangjin ParkRobotic Manipulation Vision-Language-Action Models
Department of Industrial Data Engineering, Hanyang University, Seoul, Republic of Korea.
LatentVerse: A Framework for Understanding Shared and Modality-Specific Information in Multimodal Latent Representations
Sep 14, 2026Majd Alafrange, Samuel Friedman, John Kitonyo +2Multimodal Embedding Multimodal Model Interpretability
Machine Learning for Health (ML4H), Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA · Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA
A Multimodal Explainable Deep Learning Framework for Alzheimer's Disease Diagnosis using 3D Magnetic Resonance Imaging and Clinical Data
Sep 14, 2026Yusuf Brima, Marcellin Atemkeng, Lakshmana Rao Namamula +13D Medical Imaging Multimodal Classification
Computer Vision, Institute of Cognitive Science, Osnabrück University, Germany. · Department of Mathematics, Rhodes University, South Africa. · National Institute for Theoretical and Computational Sciences (NITheCS), Stellenbosch, 7600, South Africa. +2
Toward Interpretable Multimodal Fusion: Heat Conduction Modeling for Hyperspectral and LiDAR Joint Classification
Sep 12, 2026Kan Wei, Jiahui Cui, Jing Yao +3Multimodal Model Interpretability Hyperspectral Image Classification
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China · Key Laboratory of Target Cognition and Application Technology, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China +3
The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models
Sep 10, 2026Pengfei Zhang, Biao Tian, Xiangang Li +1Neural Decoding Multimodal Foundation Models
AI Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Alibaba Token Hub, Alibaba Group, Beijing, China.
Towards reliable multimodal disaster severity assessment through preference optimization and explainable vision-language reasoning
Sep 1, 2026Yuanjun Zhang, Fuzel Ahamed Shaik, Suvojit Acharjee +2Disaster Damage Assessment Multimodal Classification
Centre for Machine Vision & Signal Processing, University of Oulu, Pentti Kaiteran katu 1, PO Box 8000, Oulu, 90570, Finland · Department of CSE and Institute of Engineering and Management, 120 SDF Building, Saltlake Electronics Complex, Kolkata, 700091, India
TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment
Aug 31, 2026Merna Bibars, Bolaji Omofojoye, Allan I. Levey +3Missing-Modality Learning Depression Detection
Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA · Department of Systems and Biomedical Engineering, Cairo University, Giza, Egypt · Department of Biomedical Informatics, Emory University, Atlanta, GA, USA +2
Multimodal Model Diffing for Feature Discovery and Control
Aug 10, 2026Hunar Batra, Lachin Naghashyar, Ashkan Khakzar +4Sparse Autoencoders Multimodal Model Interpretability
University of Oxford · Microsoft
Simile Understanding in Text-to-Image Models: An Evaluation Framework
Aug 5, 2026Luecheng Wang, Shintaro Ozaki, Hidetaka Kamigaito +4Figurative Language Understanding Vision-Language Grounding
The University of Tokyo · Nara Institute of Science and Technology · Chungnam National University +1
Persistent Convolution: A Topological Framework for AI Alignment Testing and Semantic Space Characterization
Jul 31, 2026Tyler Ashoff, Jordan RoduAI Alignment Cross-Modal Alignment
Department of Statistics, University of Virginia. · Department of Statistics, University of Virginia
Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers
Jul 30, 2026Fexiang Liu, Shiye Wang, Qiang Qiu +1VLM Evaluation Multimodal Large Language Models
1State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences
Which Modality Decides? Counterfactual Modality Attribution for Multimodal LLMs
Jul 29, 2026Vahidin Hasic, Chao Wang, Luis C. Garcia-Peraza-Herrera +2Multimodal Large Language Models Multimodal Model Interpretability
Faculty of Electrical Engineering, University of Sarajevo, Bosnia and Herzegovina · King’s College London, United Kingdom
Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering
Jul 29, 2026Yu Wang, Sharon LiUnified Multimodal Models Cross-Modal Alignment
Department of Computer Sciences, University of Wisconsin-Madison
Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
Jul 28, 2026Jiaang Li, Chengzu Li, Zhaochong An +4VLM Evaluation Multimodal Large Language Models
University of Copenhagen · University of Cambridge · 3ETH Zürich +1
CARA: Concept-Aware Risk Attention for Interpretable Collision Anticipation
Jul 24, 2026Zhishan Tao, Ruoyu Wang, Yucheng Wu +6Autonomous Driving Traffic Accident Anticipation
Shanghai Jiao Tong University · The University of Hong Kong · Fudan University +3
M $^3$ -Gen: Interpretable Multimodal Generation of Gene Expression Profiles Using Clinical and Imaging Data
Jul 23, 2026Francesca Pia Panaccione, Carlo Sgaravatti, Marco VenereMultimodal Generation Multimodal Model Interpretability
DEIB - Dipartimento Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, 20133, Italy
HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models
Jul 23, 2026Weilin Jin, Mingyu Wang, Wenbo Li +5Hallucination Detection in VLMs Multimodal Hallucination
Peking University Beijing, China · Joy Future Academy Beijing, China
MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning
Jul 18, 2026Sana Tonekaboni, Viktoria Schuster, Caroline UhlerCross-Modal Representation Learning Multimodal Disentangled Representation Learning
Massachusetts Institute of Technology, Cambridge, MA, USA · Eric and Wendy Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, MA, USA · Vector Institute, Toronto, Canada +1
From Modalities to Propositions: A Language-Centric Framework for Multimodal Intelligence
Jul 18, 2026Nadine Chang, Maying Shen, Shizhe Diao +6Multimodal IR Cross-Modal Representation Learning
NVIDIA · University of Ottawa
DELUGE: Towards Continental-Scale Daily Pluvial Flood Damage Prediction via Interpretable Conditioning on Foundation Model Embeddings
Jul 17, 2026Yuya Kawakami, Daniel Cayan, Dongyu Liu +2Disaster Damage Assessment Multimodal Model Interpretability
University of California, Davis Davis, California, USA · Scripps Institution of Oceanography University of California, San Diego La Jolla, California, USA
The Hyperspherical Geometry of CLIP Latent Space: A Semantic Mixture Model
Jul 15, 2026Zijie Yu, Gaowen Liu, Ramana Rao Kompella +2Latent Variable Models Hyperspherical Representation Learning
Tsinghua University · Cisco Research · University of Illinois Chicago
Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction
Jul 10, 2026Nikkie Hooman, Zhongjie Wu, Eric C. Larson +1Multimodal Robustness Clinical Prediction
Southern Methodist University Dallas, USA
When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalities
Jul 9, 2026Weiduo Liao, Yunqiao Yang, Ying WeiVLM Interpretability Sparse Autoencoders
Zhejiang University · Nanyang Technological University

Oct 5, 2026, cs.AIJ/K move · Enter open · S save

Multimodal Safety Evaluation Should Measure Controllability Beyond Classification

Junhyeong Park, Hanwool Lee, DongGeon Lee, Dasol Choi+3

KAIST · AIM Intelligence · Seoul National University+3

Multimodal Model Interpretability Multimodal Model Evaluation

Open paper AskPDF arXiv

VLM safety is commonly evaluated through input- and output-level classification. Such classification is necessary, but it does not reveal whether a safety state is accessible or controllable inside the model. We argue that multimodal safety evaluation should therefore report a \emph{controllability profile} alongside behavioral classification, separating representation-level detectability, cross-modal specificity, intervention sensitivity, and benign-preserving selectivity. Using implicit toxicity as a stress case, we instantiate this profile on LlavaGuard and Qwen3.5 with sparse feature decompositions. LlavaGuard admits localized handles with a narrow benign-preserving intervention range and modest downstream safety gains, whereas Qwen3.5 supports strong representation-level readout but no comparable selective-control regime under the tested operators. These results show that internal readout and controllability can diverge. Future multimodal safety benchmarks should therefore report not only behavioral safety metrics, but also whether safety-relevant internal signals can be intervention-tested and controlled within a validated operating range.

Momentum

Related topics

Latest papers 70