Feature Attribution

Follow this topic Ask about this topic

Feature attribution studies how to assign responsibility for a model’s prediction to parts of its input, such as image regions, text tokens, time-series measurements, or graph features. Common approaches include gradients and relevance propagation, perturbation and masking, Shapley values, and counterfactual or causal interventions; some methods instead build explanations into the model. Researchers evaluate explanations for faithfulness, stability, sparsity, localization, and usefulness across tasks such as image classification, language modeling, medical prediction, and time-series forecasting. A central challenge is that attributions can change with the model architecture, baseline, perturbation choices, or correlations among features, so no single method or metric is reliable in every setting.

Latest papers

234 papers20 papers in the last four weeks, down 17% on the four weeks before. 0.1% of all new papers.

Mar 5, 2026

How important are the genes to explain the outcome - the asymmetric Shapley value as an honest importance metric for high-dimensional features

In clinical prediction settings the importance of a high-dimensional feature like genomics is often assessed by evaluating the change in predictive performance when adding it to a set of traditional clinical variables. This approach is questionable, because it does not account for collinearity nor known directionality of dependencies between variables. We suggest to use asymmetric Shapley values as a more suitable alternative to quantify feature importance in the context of a mixed-dimensional prediction model. We focus on a setting that is particularly relevant in clinical prediction: disease state as a mediating variable for genomic effects, with additional confounders for which the direction of effects may be unknown. We derive efficient algorithms to compute local and global asymmetric Shapley values for this setting. The former are shown to be very useful for inference, whereas the latter provide interpretation by decomposing any predictive performance metric into contributions of the features. Throughout, we illustrate our framework by a leading example: the prediction of progression-free survival for colorectal cancer patients.

Mark A. van de Wiel, Jeroen Goedhart, Martin Jullum +1

Dept of Epidemiology and Data Science, Amsterdam Public Health research institute, Amsterdam University Medical Centers, Amsterdam, The Netherlands · Norwegian Computing Center, Oslo, Norway

Feature Attribution Shapley Value Attribution Clinical Prediction

Mar 5, 2026

From Baselines to Transport Geodesics: Axiomatic Attribution via Optimal Generative Flows

Feature attributions often hide a critical modeling choice: they explain a prediction along a counterfactual path from a reference state to an input. Different baselines, interpolations, and generative trajectories define different paths and can therefor produce different explanations. We study this path ambiguity as a modeling problem. Our central question is whether the path can be chosen by the data-generating transport process, rather than by a hand-designed interpolation or by the sensitivity geometry of the model being explained. We separate attribution into fixed-path credit allocation and path selection. For a fixed path, we prove that the Aumann-Shapley line integral is the unique attribution rule under standard fixed-path axioms and explicit coordinate-trace regularity. For path selection, we minimize kinetic action over flows that transport a reference distribution to the data distribution, yielding a transport-geodesic attribution principle. We approximate this ideal with Rectified Flow and Reflow and derive stability bounds linking vector-field error to attribution error. Experiments show that lower-action, transport-consistent paths produce more stable and structured explanations, preserving competitive deletion faithfulness, without claiming data-manifold membership. Our code is available at https://github.com/cenweizhang/OTFlowSHAP.

Cenwei Zhang, Lin Zhu, Manxi Lin +1

Aalto University, Espoo, Finland · Alibaba, Hangzhou, China · Technical University of Denmark, Copenhagen, Denmark

Feature Attribution Shapley Value Attribution Optimal Transport

Mar 5, 2026

Missingness Bias Calibration in Feature Attribution Explanations

Popular explanation methods often produce unreliable feature importance scores due to missingness bias, a systematic distortion that arises when models are probed with ablated, out-of-distribution inputs. Existing solutions treat this as a deep representational flaw that requires expensive retraining or architectural modifications. In this work, we challenge this assumption and show that missingness bias can be effectively treated as a superficial artifact of the model's output space. We introduce MCal, a lightweight post-hoc method that corrects this bias by fine-tuning a simple linear head on the outputs of a frozen base model. Surprisingly, we find this simple correction consistently reduces missingness bias and is competitive with, or even outperforms, prior heavyweight approaches across diverse medical benchmarks spanning vision, language, and tabular domains.

Shailesh Sridhar, Anton Xue, Eric Wong

University of Pennsylvania · University of Texas at Austin

Feature Attribution Perturbation-Based Feature Attribution Explainable Medical Image Analysis

Mar 4, 2026

Feature-level Interaction Explanations in Multimodal Transformers

Multimodal Transformers often produce predictions without clarifying how different modalities jointly support a decision. Most existing multimodal explainable AI (MXAI) methods extend unimodal saliency to multimodal backbones, highlighting important tokens or patches within each modality, but they rarely pinpoint which cross-modal feature pairs provide complementary evidence (synergy) or serve as reliable backups (redundancy). We present Feature-level I2MoE (FL-I2MoE), a structured Mixture-of-Experts layer that operates directly on token/patch sequences from frozen pretrained encoders and explicitly separates unique, synergistic, and redundant evidence at the feature level. We further develop an expert-wise explanation pipeline that combines attribution with top-K% masking to assess faithfulness, and we introduce Monte Carlo interaction probes to quantify pairwise behavior: the Shapley Interaction Index (SII) to score synergistic pairs and a redundancy-gap score to capture substitutable (redundant) pairs. Across three benchmarks (MMIMDb, ENRICO, and MMHS150K), FL-I2MoE yields more interactionspecific and concentrated importance patterns than a dense Transformer with the same encoders. Finally, pair-level masking shows that removing pairs ranked by SII or redundancy-gap degrades performance more than masking randomly chosen pairs under the same budget, supporting that the identified interactions are causally relevant. Code is available at https://github.com/dut0817/FL-I2MoE.

Yeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim +1

University of Alberta, Edmonton, AB, Canada · University of Waikato, Hamilton, New Zealand · University of Alberta, Camrose, AB, Canada

Multimodal Understanding Feature Attribution Explainable Artificial Intelligence Transformer Interpretability Multimodal Model Interpretability Mixture-of-Experts Models+1

Mar 3, 2026

Exact Functional ANOVA Decomposition for Categorical Inputs Models

Functional ANOVA offers a principled framework for interpretability by decomposing a model's prediction into main effects and higher-order interactions. For independent features, this decomposition is well-defined, strongly linked with SHAP values, and serves as a cornerstone of additive explainability. However, the lack of an explicit closed-form expression for general dependent distributions has forced practitioners to rely on costly sampling-based approximations. We completely resolve this limitation for categorical inputs. By bridging functional analysis with the extension of discrete Fourier analysis, we derive a closed-form decomposition without any assumption. Our formulation is computationally very efficient. It seamlessly recovers the classical independent case and extends to arbitrary dependence structures, including distributions with non-rectangular support. Furthermore, leveraging the intrinsic link between SHAP and ANOVA under independence, our framework yields a natural generalization of SHAP values for the general categorical setting.

Baptiste Ferrere, Nicolas Bousquet, Fabrice Gamboa +2

IMT, SINCLAIR AI Lab · EDF R&D, SINCLAIR Lab · Université de Toulouse +6

Feature Attribution Shapley Value Attribution Feature Interaction Modeling

Mar 2, 2026

What Helps---and What Hurts: Bidirectional Explanations for Vision Transformers

Vision Transformers (ViTs) achieve strong performance in visual recognition, yet their decision-making remains difficult to interpret. We propose BiCAM, a bidirectional class activation mapping method that captures both supportive (positive) and suppressive (negative) contributions to model predictions. Unlike prior CAM-based approaches that discard negative signals, BiCAM preserves signed attributions to produce more complete and contrastive explanations. BiCAM further introduces a Positive-to-Negative Ratio (PNR) that summarizes attribution balance and enables lightweight detection of adversarial examples without retraining. Across ImageNet, VOC, and COCO, BiCAM improves localization and faithfulness while remaining computationally efficient. It generalizes to multiple ViT variants, including DeiT and Swin. These results suggest the importance of modeling both supportive and suppressive evidence for interpreting transformer-based vision models.

Qin Su, Tie Luo

Department of Electrical and Computer Engineering University of Kentucky, Lexington, KY, USA

Transformer Interpretability Explainable Image Classification Feature Attribution Vision Transformer

Mar 1, 2026

AG-REPA: Causal Layer Selection for Representation Alignment in Audio Flow Matching

REPresentation Alignment (REPA) improves the training of generative flow models by aligning intermediate hidden states with pretrained teacher features, but its effectiveness in token-conditioned audio Flow Matching critically depends on the choice of supervised layers, which is typically made heuristically based on the depth. In this work, we introduce Attribution-Guided REPresentation Alignment (AG-REPA), a novel causal layer selection strategy for representation alignment in audio Flow Matching. Firstly, we find that layers that best store semantic/acoustic information (high teacher-space similarity) are not necessarily the layers that contribute most to the velocity field that drives generation, and we call it Store-Contribute Dissociation (SCD). To turn this insight into an actionable training guidance, we propose a forward-only gate ablation (FoG-A) that quantifies each layer's causal contribution via the induced change in the predicted velocity field, enabling sparse layer selection and adaptive weighting for alignment. Across unified speech and general-audio training (LibriSpeech + AudioSet) under different token-conditioning topologies, AG-REPA consistently outperforms REPA baselines. Overall, our results show that alignment is most effective when applied to the causally dominant layers that drive the velocity field, rather than to layers that are representationally rich but functionally passive.

Pengfei Zhang, Tianxin Xie, Minghao Yang +1

AI Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou).

Flow-Based Generative Modeling Flow Matching Audio Representation Learning Feature Attribution

Feb 28, 2026

A Polynomial-Time Axiomatic Alternative to SHAP for Feature Attribution

In this paper, we provide a theoretically grounded and computationally efficient alternative to SHAP. To this end, we study feature attribution through the lens of cooperative game theory by formulating a class of XAI--TU games. Building on this formulation, we investigate equal-surplus-type and proportional-allocation-type attribution rules and propose a low-cost attribution rule, ESENSC_rev2, constructed by combining two polynomial-time closed-form rules while ensuring the null-player property in the XAI--TU domain. Extensive experiments on tabular prediction tasks demonstrate that ESENSC_rev2 closely approximates exact SHAP while substantially improving scalability as the number of features increases. These empirical results indicate that equal-surplus-type attribution rules can achieve favorable trade-offs between computational cost and approximation accuracy in high-dimensional explainability settings. To provide theoretical foundations for these findings, we establish an axiomatic characterization showing that ESENSC_rev2 is uniquely determined by efficiency, the null-player axiom, a restricted differential marginality principle, an intermediate inessential-game property, and axioms that reduce computational requirements. Our results suggest that axiomatically justified and computationally efficient attribution rules can serve as practical and theoretically principled substitutes for SHAP-based approximations in modern explainability pipelines.

Kazuhiro Hiraki, Shinichi Ishihara, Takumi Kongo +1

International Monetary Fund · Independent Researcher · Fukuoka University +1

Feature Attribution Shapley Value Attribution

Feb 27, 2026

What You Read is What You Classify: Highlighting Attributions to Text and Text-Like Inputs

At present, there are no easily understood explainable artificial intelligence (AI) methods for discrete token inputs, like text. Most explainable AI techniques do not extend well to token sequences, where both local and global features matter, because state-of-the-art models, like transformers, tend to focus on global connections. Therefore, existing explainable AI algorithms fail by (i) identifying disparate tokens of importance, or (ii) assigning a large number of tokens a low value of importance. This method for explainable AI for tokens-based classifiers generalizes a mask-based explainable AI algorithm for images. It starts with an Explainer neural network that is trained to create masks to hide information not relevant for classification. Then, the Hadamard product of the mask and the continuous values of the classifier's embedding layer is taken and passed through the classifier, changing the magnitude of the embedding vector but keeping the orientation unchanged. The Explainer is trained for a taxonomic classifier for nucleotide sequences and it is shown that the masked segments are less relevant to classification than the unmasked ones. This method focused on the importance the token as a whole (i.e., a segment of the input sequence), producing a human-readable explanation.

Daniel S. Berman, Brian Merritt, Stanley Ta +4

Johns Hopkins Applied Physics Laboratory, 11100 Johns Hopkins Rd., Laurel, MD 20723, USA. · Current affiliation: Google DeepMind, 14–18 Handyside Street, London, N1C 4DN, United Kingdom

Explainable Text Classification Explainable Artificial Intelligence Feature Attribution Perturbation-Based Feature Attribution

Feb 26, 2026

Fair feature attribution for multi-output prediction: a Shapley-based perspective

In this article, we provide an axiomatic characterization of feature attribution for multi-output predictors within the Shapley framework. While SHAP explanations are routinely computed independently for each output coordinate, the theoretical necessity of this practice has remained unclear. By extending the classical Shapley axioms to vector-valued cooperative games, we establish a rigidity theorem showing that any attribution rule satisfying efficiency, symmetry, dummy player, and additivity must necessarily decompose component-wise across outputs. Consequently, any joint-output attribution rule must relax at least one of the classical Shapley axioms. This result identifies a previously unformalized structural constraint in Shapley-based interpretability, clarifying the precise scope of fairness-consistent explanations in multi-output learning. Numerical experiments on a biomedical benchmark illustrate that multi-output models can yield computational savings in training and deployment, while producing SHAP explanations that remain fully consistent with the component-wise structure imposed by the Shapley axioms.

Umberto Biccari, Alain Ibáñez de Opakua, José María Mato +3

Feature Attribution Shapley Value Attribution Cooperative Game Theory

Feb 23, 2026

cc-Shapley: Measuring Multivariate Feature Importance Needs Causal Context

Explainable artificial intelligence promises to yield insights into relevant features, thereby enabling humans to examine and scrutinize machine learning models or even facilitating scientific discovery. Considering the widespread technique of Shapley values, we find that purely data-driven operationalization of multivariate feature importance is unsuitable for such purposes. Even for simple problems with two features, spurious associations due to collider bias and suppression arise from considering one feature only in the observational context of the other, which can lead to misinterpretations. Causal knowledge about the data-generating process is required to identify and correct such misleading feature attributions. We propose cc-Shapley (causal context Shapley), an interventional modification of conventional observational Shapley values leveraging knowledge of the data's causal structure, thereby analyzing the relevance of a feature in the causal context of the remaining features. We show theoretically that this eradicates spurious association induced by collider bias. We compare the behavior of Shapley and cc-Shapley values on various, synthetic, and real-world datasets. We observe nullification or reversal of associations compared to univariate feature importance when moving from observational to cc-Shapley.

Jörg Martin, Stefan Haufe

Physikalisch-Technische Bundesanstalt, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · Charité – Universitätsmedizin Berlin, Berlin, Germany

Feature Attribution Shapley Value Attribution Causal Attribution Spurious Correlation Causal Inference

Feb 22, 2026

DD-CAM: Minimal Sufficient Explanations for Vision Models Using Delta Debugging

We introduce a gradient-free framework for identifying minimal, sufficient, and decision-preserving explanations in vision models by isolating the smallest subset of representational units whose joint activation preserves predictions. Unlike existing approaches that aggregate all units, often leading to cluttered saliency maps, our approach, DD-CAM, identifies a 1-minimal subset whose joint activation suffices to preserve the prediction (i.e., removing any unit from the subset alters the prediction). To efficiently isolate minimal sufficient subsets, we adapt delta debugging, a systematic reduction strategy from software debugging, and configure its search strategy based on unit interactions in the classifier head: testing individual units for models with non-interacting units and testing unit combinations for models in which unit interactions exist. We then generate minimal, prediction-preserving saliency maps that highlight only the most essential features. Our experimental evaluation demonstrates that our approach can produce more faithful explanations and achieve higher localization accuracy than the state-of-the-art CAM-based approaches.

Krishna Khadka, Yu Lei, Raghu N. Kacker +1

Department of Computer Science & Engineering The University of Texas at Arlington Arlington, Texas, USA · Information Technology Laboratory National Institute of Standards and Technology (NIST) Gaithersburg, Maryland, USA

Explainable Image Classification Feature Attribution Saliency Map Evaluation

Feb 19, 2026

Owen-based Semantics and Hierarchy-Aware Explanation (O-Shap)

Shapley value-based methods have become foundational in explainable artificial intelligence (XAI), offering theoretically grounded feature attributions through cooperative game theory. However, in practice, particularly in vision tasks, the assumption of feature independence breaks down, as features (i.e., pixels) often exhibit strong spatial and semantic dependencies. To address this, modern SHAP implementations now include the Owen value, a hierarchical generalization of the Shapley value that supports group attributions. While the Owen value preserves the foundations of Shapley values, its effectiveness critically depends on how feature groups are defined. We show that commonly used segmentations (e.g., axis-aligned or SLIC) violate key consistency properties, and propose a new segmentation approach that satisfies the

T

-property to ensure semantic alignment across hierarchy levels. This hierarchy enables computational pruning while improving attribution accuracy and interpretability. Experiments on image and tabular datasets demonstrate that O-Shap outperforms baseline SHAP variants in attribution precision, semantic coherence, and runtime efficiency, especially when structure matters.

Xiangyu Zhou, Chenhan Xiao, Yang Weng

Department of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, Arizona, United States

Feature Attribution SHAP Feature Attribution Shapley Value Attribution

Feb 18, 2026

Functional Decomposition and Shapley Interactions for Interpreting Survival Models

Hazard and survival functions are natural, interpretable targets in time-to-event prediction, but their inherent non-additivity fundamentally limits standard additive explanation methods. We introduce Survival Functional Decomposition (SurvFD), a principled approach for analyzing feature interactions in machine learning survival models. By decomposing higher-order effects into time-dependent and time-independent components, SurvFD offers a previously unrecognized perspective on survival explanations, explicitly characterizing when and why additive explanations fail. Building on this theoretical decomposition, we propose SurvSHAP-IQ, which extends Shapley interactions to time-indexed functions, providing a practical estimator for higher-order, time-dependent interactions. Together, SurvFD and SurvSHAP-IQ establish an interaction- and time-aware interpretability approach for survival modeling, with broad applicability across time-to-event prediction tasks.

Sophie Hanna Langbein, Hubert Baniecki, Fabian Fumagalli +3

Leibniz Institute for Prevention Research and Epidemiology – BIPS · Faculty of Mathematics and Computer Science, University of Bremen · University of Warsaw +3

Feature Attribution Feature Interaction Modeling Survival Analysis

Jan 19, 2026

Explanation Multiplicity in SHAP: Characterization and Assessment

SHAP explanations are widely used in high-stakes settings to justify decisions, yet they can differ substantially across repeated runs, even when the model, the input instance, and the prediction are held fixed. Prior work has documented disagreement between explanation methods; we show that substantial disagreement arises even within SHAP across reruns of the same estimator on the same trained model and instance. We call this phenomenon explanation multiplicity and develop an evaluation methodology for characterizing it under deployment-realistic computational budgets, combining a dual-seed protocol that compares model-induced and explainer-induced variability, a hierarchy of magnitude-based, rank-based, and set-based metrics, and randomized Dirichlet and Mallows null models that provide reference scales for observed disagreement. Across multiple datasets, models, and sampling strategies, we find that explanation multiplicity is pervasive and persists even for high-confidence predictions. The relative contribution of each source depends on the data regime and model: model-induced disagreement is generally greater on smaller datasets, while explainer-induced disagreement is greater on larger datasets. Commonly used L2 distance can understate this instability, while rank-based metrics reveal substantial changes in top-ranked features, including the leading feature. Improved sampling methods such as CTE do not eliminate rank-level multiplicity, and K-Means reduces run-to-run variation while its compressed-background explanations can diverge from the empirical-distribution reference. Practitioners should treat single-run SHAP outputs as realizations of a distribution rather than as authoritative artifacts.

Hyunseung Hwang, Seungeun Lee, Lucas Rosenblatt +2

KAIST, Daejeon, Republic of Korea · New York University, New York, NY, USA

SHAP Feature Attribution Feature Attribution Explainability Evaluation

Jan 13, 2026

Motion Attribution for Video Generation

Despite the rapid progress of video generation models, the role of data in influencing motion is poorly understood. We present Motive (MOTIon attribution for Video gEneration), a motion-centric, gradient-based data attribution framework that scales to modern, large, high-quality video datasets and models. We use this to study which fine-tuning clips improve or degrade temporal dynamics. Motive isolates temporal dynamics from static appearance via motion-weighted loss masks, yielding efficient and scalable motion-specific influence computation. On text-to-video models, Motive identifies clips that strongly affect motion and guides data curation that improves temporal consistency and physical plausibility. With Motive-selected high-influence data, our method improves both motion smoothness and dynamic degree on VBench, achieving a 74.1% human preference win rate compared with the pretrained base model. To our knowledge, this is the first framework to attribute motion rather than visual appearance in video generative models and to use it to curate fine-tuning data.

Xindi Wu, Despoina Paschalidou, Jun Gao +5

NVIDIA · Princeton University · University of Michigan +3

Gradient-Based Attribution Feature Attribution Training Data Attribution Temporal Consistency in Video Generation

Jan 10, 2026

Explainability of Complex AI Models with Correlation Impact Ratio

Complex AI systems make better predictions but often lack transparency, limiting trustworthiness, interpretability, and safe deployment. Common post hoc AI explainers, such as LIME, SHAP, HSIC, and SAGE, are model agnostic but are too restricted in one significant regard: they tend to misrank correlated features and require costly perturbations, which do not scale to high dimensional data. We introduce ExCIR (Explainability through Correlation Impact Ratio), a theoretically grounded, simple, and reliable metric for explaining the contribution of input features to model outputs, which remains stable and consistent under noise and sampling variations. We demonstrate that ExCIR captures dependencies arising from correlated features through a lightweight single pass formulation. Experimental evaluations on diverse datasets, including EEG, synthetic vehicular data, Digits, and Cats-Dogs, validate the effectiveness and stability of ExCIR across domains, achieving more interpretable feature explanations than existing methods while remaining computationally efficient. To this end, we further extend ExCIR with an information theoretic foundation that unifies the correlation ratio with Canonical Correlation Analysis under mutual information bounds, enabling multi output and class conditioned explainability at scale.

Poushali Sengupta, Rabindra Khadka, Sabita Maharjan +5

Institute of Informatics, University of Oslo, Oslo, Norway · Department of Computer Science, Oslo Metropolitan University, Oslo, Norway · Aalborg University, Denmark

Feature Attribution Interpretable ML Explainability Evaluation

Nov 19, 2025

CID: Measuring Feature Importance Through Counterfactual Distributions

Assessing the importance of individual features in Machine Learning is critical to understand the model's decision-making process. While numerous methods exist, the lack of a definitive ground truth for comparison highlights the need for alternative, well-founded measures. This paper introduces a novel post-hoc local feature importance method called Counterfactual Importance Distribution (CID). We generate two sets of positive and negative counterfactuals, model their distributions using Kernel Density Estimation, and rank features based on a distributional dissimilarity measure. This measure, grounded in a rigorous mathematical framework, satisfies key properties required to function as a valid metric. We showcase the effectiveness of our method by comparing with well-established local feature importance explainers. Our method not only offers complementary perspectives to existing approaches, but also improves performance on faithfulness metrics (both for comprehensiveness and sufficiency), resulting in more faithful explanations of the system. These results highlight its potential as a valuable tool for model analysis. Link to repository: https://github.com/EddieConti/CID

Eddie Conti, Álvaro Parafita, Axel Brando

TAIES Group, HPES Lab, Barcelona Supercomputing Center (BSC)

Feature Attribution Explainability Evaluation

Oct 26, 2025

Data-driven Sensor Placement for Predictive Applications: A Correlation-Assisted Attribution Framework (CAAF)

Optimal sensor placement (OSP) is critical for efficient, accurate monitoring, control, and inference in complex physical systems. We propose a machine-learning-based feature attribution (FA) framework to identify OSP for target predictions. FA quantifies input contributions to a model output; however, it struggles with highly correlated input data often encountered in practical applications for OSP. To address this, we propose a Correlation-Assisted Attribution Framework (CAAF), which introduces a clustering step on the candidate sensor locations before performing FA to reduce redundancy and enhance generalizability. We first illustrate the core principles of the proposed framework through a series of validation cases, then demonstrate its effectiveness in realistic dynamical systems such as structural health monitoring, airfoil lift prediction, and wall-normal velocity estimation for turbulent channel flow. The results show that the CAAF outperforms alternative approaches that typically struggle due to the presence of nonlinear dynamics, chaotic behavior, and multi-scale interactions, and enables the effective application of FA for identifying OSP in real-world environments.

Sze Chai Leung, Di Zhou, H. Jane Bae

Department of Mechanical and Civil Engineering, California Institute of Technology, Pasadena, CA 91125, USA. · Lynn Booth & Kent Kresa Department of Aerospace, California Institute of Technology, Pasadena, CA 91125, USA. · Department of Mechanical and Aerospace Engineering, University of Tennessee, Knoxville, TN 37996, USA.

Feature Attribution

Sep 26, 2025

Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

Multimodal large language models (MLLMs) have achieved strong vision-language performance, yet their token-level visual evidence remains difficult to inspect. Recent logit-lens attribution methods project each visual-token hidden state into the vocabulary space to explain generated words, but this token-wise readout introduces a mismatch: visual tokens are context-mixed by the model, while the attribution score is decoded independently at each token location. This often produces fragmented attribution maps and can be further affected by autoregressive context signals from preceding text tokens. We propose ERCR, an attribution framework built from Evidence Recomposition (ER) and Predictive Context Residualization (PCR). ER aggregates target evidence across multiple views with different token-to-region assignments, reducing attribution fragmentation caused by a single readout grid. PCR estimates a preceding-token context map with RBO-based rank relevance and subtracts its fitted component from the ER map to suppress context-token interference. Experiments on LLaVA, Qwen2-VL, and InternVL families across COCO Caption, GranDf, and OpenPSG show that ERCR improves visual evidence for target tokens and mitigates preceding-token context interference under the existing evaluation protocol. On Qwen2-VL-2B, ERCR improves TAM F1-IoU from 39.10 to 44.45 on COCO Caption and from 30.83 to 37.20 on GranDf. Overall, ERCR provides a practical refinement for token-level visual evidence inspection.

Jiawei Liang, Jianjie Huang, Ruoyu Chen +4

Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · Zhongguan-cun Academy, Beijing 100094, China · University of Chinese Academy of Sciences, Beijing 100049, China +2

Feature Attribution Multimodal Large Language Models VLM Interpretability Multimodal Model Interpretability Multimodal Understanding

Sep 25, 2025

Frequency-Aware Model Parameter Explorer: A new attribution method for improving explainability

State-of-the-art attribution methods such as AttEXplore rely on adversarial sample generation that applies an all-pass filter across the frequency spectrum: they perturb all frequencies of the input image alike, from the low frequencies that carry coarse shape to the high frequencies that carry edges and textures. However, models rely on low and high frequencies to different degrees, and such an all-pass perturbation cannot adapt to the model being explained, which limits the faithfulness of the resulting attribution maps. We propose FAMPE (Frequency-Aware Model Parameter Explorer), an attribution method that overcomes this limitation by perturbing the low- and high-frequency bands of the input independently, so that the perturbation can be shifted toward the band that matters for the model. The boundary between the two bands is set for each image automatically from its spectral energy. On ImageNet with five convolutional neural network and Vision Transformer architectures, FAMPE significantly improves the Insertion Score over the strongest baseline, AttEXplore, on four of them, by up to 15.4%, at essentially the same computational cost. How the perturbed frequency bands are weighted matters: the best and the worst balance between them differ by 19-36% in Insertion Score, and the best balance depends on the architecture. Choosing the balance for each image with a probe that needs no ground truth raises the Insertion Score significantly further on all five architectures, reaching up to 24.4% above AttEXplore. Controlling which frequency band is perturbed is therefore a simple, low-cost way to improve attribution faithfulness.

Ali Yavari, Alireza Mohamadi, Elham Beydaghi +2

Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria · Independent · Dept. of Computer Engineering, Sharif University of Technology, Tehran, Iran +1

Feature Attribution Perturbation-Based Feature Attribution Explainable Image Classification Saliency Map Evaluation

May 21, 2025

Explainable embeddings with Distance Explainer

While eXplainable AI (XAI) has advanced significantly, few methods address interpretability in embedded vector spaces where dimensions represent complex abstractions. We introduce Distance Explainer, a novel method for generating local, post-hoc explanations of embedded spaces in machine learning models. Our approach adapts saliency-based techniques from RISE to explain the distance between two embedded data points by assigning attribution values through selective masking and distance-ranked mask filtering. We evaluate Distance Explainer on cross-modal embeddings (image-image and image-caption pairs) using established XAI metrics including Faithfulness, Sensitivity/Robustness, and Randomization. Experiments with ImageNet and CLIP models demonstrate that our method effectively identifies features contributing to similarity or dissimilarity between embedded data points while maintaining high robustness and consistency. We also explore how parameter tuning, particularly mask quantity and selection strategy, affects explanation quality. This work addresses a critical gap in XAI research and enhances transparency and trustworthiness in deep learning applications utilizing embedded spaces.

Christiaan Meijer, E. G. Patrick Bos

Netherlands eScience Center, Science Park 402, 1098 XH Amsterdam, Netherlands · Center for Information Technology, University of Groningen, P.O. Box 11044, 9700 CA Groningen, Netherlands

Feature Attribution Perturbation-Based Feature Attribution VLM Interpretability Neural Network Interpretability

Apr 1, 2025

Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection

To develop a trustworthy AI system, it is essential to understand its behavior using attribution methods, which aim to identify the input regions that most influence the model's decisions. For discrete inputs such as images, however, accurately identifying input-prediction interactions is particularly challenging because of combinatorial explosion. We formulate black-box attribution as an ordered interpretable subset-selection problem. Inspired by the marginal-gain principle of submodular optimization, the formulation permits redundancy, positive complementarity, and negative interference among input sub-regions. We propose LiMA (Less input is More faithful for Attribution), which ranks sub-regions by their contributions along a sequence of selected subsets. LiMA employs a composite objective that evaluates subsets through semantic consistency, collaboration, prediction confidence, and semantic effectiveness. We further design a bidirectional greedy search to efficiently identify important and non-critical sub-regions. Experiments on eight foundation models (CLIP, ImageBind, LanguageBind, QuiltNet, etc.) and six datasets (ImageNet, VGG-Sound, CUB-200-2011, etc.) demonstrate strong generalization and faithful interpretations with fewer sub-regions. LiMA achieves state-of-the-art Insertion, Deletion, and average highest confidence, with average improvements of 36.3% in Insertion and 39.6% in Deletion; it is also 1.6 times faster than naive greedy search. For failure attribution on incorrect predictions, LiMA improves average highest confidence by 86.1% over state-of-the-art methods. The code is available at https://github.com/RuoyuChen10/LIMA.

Ruoyu Chen, Siyuan Liang, Jingzhi Li +4

Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China · University of Chinese Academy of Sciences, Beijing 100049, China · College of Computing and Data Science, Nanyang Technological University, 639798, Singapore +4

Feature Attribution VLM Interpretability Explainability Evaluation Feature Selection

Jun 17, 2024

CHAOSMINING: Benchmarking Post-Hoc Attribution with Sparse Informative Features in High Dimensions

Post-hoc attribution is widely used to identify important model inputs, but evaluating whether these attributions identify truly informative features is difficult because real datasets rarely provide reliable ground truth. We introduce a multimodal benchmark containing symbolic tabular, vision, and audio tasks with known informative feature sets. In the main benchmark conditions, informative variables, spatial regions, or channels occupy fixed input coordinates while the remaining inputs provide irrelevant or distracting information. We use the benchmark to study how attribution quality depends on predictive performance, irrelevant-feature burden and structure, model configuration, and attribution mechanism, while separately measuring identification, stability, and computational cost. In most symbolic-data sweeps, informative-set identification co-varies with predictive performance, while the relative ordering of attribution methods remains largely stable. Across modalities, no method dominates all architectures and conditions, and greater attribution complexity does not consistently improve identification. Simple gradient attribution is often competitive at lower computational cost, while the vision and audio results show that architecture and the form of irrelevant content materially affect attribution quality.

Ge Shi, Fangyi Liu, Ziwen Kan +1

Department of Computer Science University of California, Davis

Gradient-Based Attribution Feature Attribution Explainability Evaluation