Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology

Mina Jamshidi Idaji; Julius Hense; Tom Neuhäuser; Augustin Krause; Yanqing Luo; Oliver Eberle; Thomas Schnake; Laure Ciernik

cs.CVMar 9, 2026

Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology

Authors: Mina Jamshidi Idaji, Julius Hense, Tom Neuhäuser, Augustin Krause, Yanqing Luo, Oliver Eberle, Thomas Schnake, Laure Ciernik,

Organizations: Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Department of Chemistry, Chemical Physics Theory Group, University of Toronto, Canada · Vector Institute for Artificial Intelligence, Toronto, Canada · Acceleration Consortium, University of Toronto, Canada · Department of Computer Science and Engineering, The Chinese University of Hong Kong · Aignostics GmbH, Berlin, Germany · Institute of Pathology, Ludwig Maximilian University, Munich, Germany · German Cancer Research Center, Heidelberg, and German Cancer Consortium, Munich, Germany · Institute of Pathology, Charité Universitätsmedizin, Berlin, Germany · Department of Artificial Intelligence, Korea University, Seoul, Korea · Max-Planck Institute for Informatics, Saarbrücken, Germany

Abstract

Multiple instance learning (MIL) has enabled substantial progress in computational histopathology, where a large amount of patches from gigapixel whole slide images are aggregated into slide-level predictions. Heatmaps are widely used to validate MIL models and to discover tissue biomarkers. Yet, the validity of these heatmaps has barely been investigated. In this work, we introduce a general framework for evaluating the quality of MIL heatmaps without requiring additional labels. We conduct a large-scale benchmark experiment to assess six explanation methods across histopathology task types (classification, regression, survival), MIL model architectures (Attention-, Transformer-, Mamba-based), and patch encoder backbones (UNI2, Virchow2). Our results show that explanation quality mostly depends on MIL model architecture and task type, with perturbation ("Single"), layer-wise relevance propagation (LRP), and integrated gradients (IG) consistently outperforming attention-based and gradient-based saliency heatmaps, which often fail to reflect model decision mechanisms. We further demonstrate the advanced capabilities of the best-performing explanation methods: (i) We provide a proof-of-concept that MIL heatmaps of a bulk gene expression prediction model can be correlated with spatial transcriptomics for biological validation, and (ii) showcase the discovery of distinct model strategies for predicting human papillomavirus (HPV) infection from head and neck cancer slides. Our work highlights the importance of validating MIL heatmaps and establishes that improved explainability can enable more reliable model validation and yield biological insights, making a case for a broader adoption of explainable AI in digital pathology. Our code is provided in a public GitHub repository: https://github.com/bifold-pathomics/xMIL/tree/xmil-journal

Figures & tables

Figure 1: Overview of our explanation pipeline for multiple instance learning (MIL) . a Block diagram of a general MIL approach to make predictions directly from whole slide images, and to obtain heatmaps to interpret the model predictions. The explanation methods produce patch-wise scores by explaining (only) the task head and the aggregation model, but not the patch encoder. In this work, we assess six explanation methods in the context of two patch encoders, three aggregation model architectures, and three types of prediction heads. b Composition of the task heads. In the survival head, the discrete-time model predicts hazard probabilities $\{h_{k}\}$ for survival time intervals $k=1,\cdots,K$ , and subsequently aggregates them into a patient risk score. See Section B for details. $\sigma$ = sigmoid function; FNN = feedforward neural network.

Figure 2: Overview of the evaluation framework for comparing heatmaps and explanation methods in multiple instance learning (MIL) .

Figure 3: The patch flipping procedure . Patches are sorted based on their heatmap scores either in ascending or descending order. They are then grouped into 100 mini-bags. The mini-bags are removed progressively from the main bag of patches, and the remaining patches are fed into the model. Two perturbation curves (model prediction vs. dropped fraction) are formed for the two orderings. A larger area under the curve (AUC) of the ascending (Asc.) curve is desired, whereas a smaller AUC of the descending (Desc.) curve is better. Therefore, the area between the two curves (called Symmetric Relevance Gain–SRG) quantifies the faithfulness of the heatmap. A larger SRG indicates a more faithful heatmap.

Figure 4: Datasets and model performances. The bar plots in each panel show the mean test performance over the cross-validation folds with the error bars depicting the standard deviations (see Tables 2 and 3 for numerical model performance results). For the classification tasks in panel (a), the ratio of the cases in each class is depicted. In panels (b) and (c), the endpoint target’s histograms are illustrated.

Figure 5: Comparison of explanation methods for two example settings . a / b Faithfulness comparison from three complementary perspectives: ( Left ) The distribution of Symmetric Relevance Gain (SRG) faithfulness scores per explanation method computed for each slide in the test set. ( Center ) Wilcoxon’s signed-rank test effect sizes comparing SRG scores between explanation methods for each slide in the test set. Colors and marker sizes indicate effect size magnitude as shown on the colorbar. Triangle, square, and circle markers are used to denote the negligible ( $<0.2$ ), weak to moderate ( $0.2-0.5$ ), and moderate to strong ( $\geq 0.5$ ) effect sizes. These effect size scores assess the magnitude of the faithfulness differences between explanation methods. ( Right ) Mean Rank Score (MRS) per explanation method (vertical axis). Lower averaged MRS = better position in the faithfulness ranking . c / d Heatmaps for example slides from the two experimental settings. For LRP, values greater/lower than zero indicate evidence for/against the model prediction (visualized by shades of red/blue). Single and Attention (Attn) have no natural boundary between positive and negative evidence, but only provide an ordering (visualized by shades of red). In the HRD regression task, however, the $0$ value for Single may be interpreted as such a boundary; therefore, we visualize this heatmap using a red/blue color scheme as well.

Figure 6: Comparisons of explanation methods aggregated across experimental settings . a Pairwise Wilcoxon signed-rank effect sizes comparing SRG scores between explanation methods, averaged over various experimental settings. “Overall” considers the mean effect size across all experimental settings, “Classification” the mean over all classification experiments, etc. The filling colors and marker sizes indicate effect size magnitude as shown on the colorbar. Triangle, square, and circle markers are used to denote the negligible ( $<0.2$ ), weak to moderate ( $0.2-0.5$ ), and moderate to strong ( $\geq 0.5$ ) effect sizes. Notably, two consistent groups emerge: LRP, IG, Single vs. Attn, G $\times$ I, Grad2, Rand, with the first group generally outperforming the second. All aggregated comparisons were additionally replicated using a robust median/MAD effect size, and non-significant pairwise comparisons after FDR correction are reported separately in Section E . b Mean Rank Score (MRS) per explanation method (vertical axis), averaged across selected MIL pipeline settings (horizontal axis). Lower averaged MRS = better position in the faithfulness ranking .

Figure 7: Proof-of-concept for spatial mapping of bulk gene expression using explanation heatmaps of a regression MIL model . a Experimental pipeline. Whole-slide H&E images are subdivided into cutouts, from which patch-level feature vectors are extracted. For each cutout, pseudo-bulk gene expression targets are derived by aggregating spatial transcriptomics (ST) measurements spatially aligned to the same region. Model explanation methods are then applied to obtain patch-level relevance scores, which are then realigned to build a WSI-level heatmap and compared against the original spatial gene expression. b Example heatmaps for gene FASN on a slide, shown alongside the corresponding ST ground truth. Heatmaps indicate the patch-level model-attributed relevance for predicting bulk expression. c Pearson correlation coefficients between patch scores derived from explanation heatmaps and the corresponding ST ground truth. Results are shown for four explanation methods: Layer-wise Relevance Propagation (LRP), Integrated Gradients (IG), Single, and Attention. Individual slides are indicated by distinct symbols on the violin plots.

Figure 8: Histological analysis of HPV heatmaps for discovery . a We extract the most relevant heatmap regions (=top 10% LRP patch scores) and characterize them by computing interpretable and quantifiable histological features. b The cell composition of the most relevant regions determined via CellViT++ ( 95 ) and tissue compartment segmentation exposes clusters among HPV-positive slides. See also Figure 14 for a 2D scatter matrix of the five most relevant features. The samples in panels c-f are marked. c-f Example heatmaps and relevant histological features for HPV-positive prediction, consisting of a thumbnail of the most relevant regions (top row) and a selected zoomed-in slide region (middle row) with CellViT-annotated neoplastic (red) and inflammatory cells (blue) (bottom row).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Model	Hyperparameter Grid
Classification	TransMIL	train bag size={2048}, batch size={5}, lr={2e-4, 2e-3}, wd={0, 0.01}, dropouts={(0, 0, 0), (0.5, 0.5, 0.2)}, opt={SGD}, epochs=200, warmup=0
	AttnMIL	train bag size=2048, batch size=32, lr={0.002, 0.0002}, wd={0, 0.01}, dropouts={0, 0.5}, opt=SGD, epochs=1000, warmup=0
	MambaMIL	train bag size={2048}, batch size={5}, lr={2e-4, 2e-3}, wd={0,0.01},opt={SGD}, epochs=200, warmup=0
Regression	TransMIL	train bag size={2048}, batch size={8}, lr={1e-6, 1e-5, 1e-4}, wd={0.01, 0.1}, dropouts={(0.2, 0.5, 0.2), (0.5, 0.75, 0.5)}, opt={SGD, Adam}, epochs=300, warmup=5
	AttnMIL	train bag size={2048}, batch size={32}, lr={2e-5, 2e-4}, wd={0.01, 0.1}, dropouts={0.2, 0.75}, opt={SGD, Adam}, epochs=150, warmup=5
	MambaMIL	train bag size={2048}, batch size={5}, lr={1e-6, 1e-5, 1e-4}, wd={0, 0.01}, opt={SGD}, epochs=200, warmup=0

Appendix

Table 1: Hyperparameter grids for all models trained in this work

	Classification				Regression		Survival
	CAMELYON	NSCLC	HPV	TP53	AURKA	HRD	BRCA	GBMLGG	KIRC	COADREAD
TransMIL	1.00(0.00)	0.97(0.02)	0.90(0.04)	0.71(0.05)	0.66(0.02)	0.62(0.02)	0.65(0.08)	0.74(0.04)	0.70(0.03)	0.63(0.06)
AttnMIL	1.00(0.00)	0.98(0.01)	0.92(0.04)	0.68(0.07)	0.69(0.03)	0.60(0.02)	0.62(0.05)	0.76(0.04)	0.71(0.03)	0.61(0.07)
MambaMIL	0.97(0.03)	0.90(0.04)	0.89(0.03)	0.63(0.08)	0.65(0.03)	0.60(0.03)	0.59(0.05)	0.76(0.12)	0.68(0.06)	0.64(0.06)

Appendix

Table 2: Performance results for the models in faithfulness experiments, with backbone Virchow2. The numbers are mean (std) of the performance over the cross-validation folds.

	Classification				Regression		Survival
	CAMELYON	NSCLC	HPV	TP53	AURKA	HRD	BRCA	GBMLGG	KIRC	COADREAD
TransMIL	1.00(0.00)	0.98(0.01)	0.92(0.04)	0.68(0.08)	0.66(0.03)	0.62(0.05)	0.65(0.07)	0.75(0.04)	0.71(0.06)	0.62(0.08)
AttnMIL	1.00(0.00)	0.97(0.02)	0.92(0.04)	0.65(0.07)	0.69(0.03)	0.63(0.03)	0.58(0.05)	0.75(0.03)	0.71(0.04)	0.58(0.06)
MambaMIL	0.98(0.02)	0.86(0.03)	0.88(0.04)	0.62(0.07)	0.66(0.04)	0.61(0.04)	0.62(0.06)	0.75(0.12)	0.71(0.07)	0.62(0.09)

Appendix

Table 3: Performance results for the models in faithfulness experiments, with backbone UNI2. The numbers are mean (std) of the performance over the cross-validation folds.

Figure 9: Experimental configurations with at least one non-significant pairwise faithfulness differences after multiple comparison correction. Each panel corresponds to one experimental configuration (dataset × task × backbone × MIL architecture) in which at least one pairwise Wilcoxon signed-rank test between explanation methods was non-significant after FDR correction ( $\alpha=0.05$ ). Cells indicate method pairs, and the red color marks the pairs for which no statistically significant difference in SRG faithfulness was detected. Most non-significant comparisons occur within the two groups {Single, LRP, IG} and {Attention, G×I, Grad2, Random}. Only two configurations in panels 4 and 7 (both CAMELYON16 tumor detection experiments) exhibit non-significant differences across groups, consistent with the simplicity of the task and the aggregation-level results reported in Section 4.4 .

Figure 10: Pairwise median/MAD-based comparison of SRG scores between explanation methods, averaged over various experimental factors. This figure corresponds to Figure 6 . The filling colors indicate the magnitude of median/MAD as shown on the colorbar. Triangle, square, and circle markers are used to denote the negligible ( $<0.2$ ), weak to moderate ( $0.2-0.5$ ), and moderate to strong ( $\geq 0.5$ ) effect sizes.

Figure 11: Ground truth and heatmaps for the ERBB2 prediction model in Section 5 . The ground-truth spatial transcriptomics and the heatmaps for the four explanation methods used in the ST validation study presented in Section 5 .

Figure 12: Ground truth and heatmaps for the FASN prediction model in Section 5 . The ground-truth spatial transcriptomics and the heatmaps for the four explanation methods used in the ST validation study presented in Section 5 .

Figure 13: Most relevant features for HPV prediction in TCGA HNSC . Mean and standard deviation of the cell type shares per tissue compartment in the top-10% heatmap regions across all HPV positive samples.

Figure 14: Comparison of histological feature profiles for HPV prediction in TCGA HNSC . Scatter matrix displaying the distribution of cell type shares per tissue compartment in the HL10 regions across all HPV positive samples for five selected histological features. The prediction scores indicate the model prediction for these samples (closer to 1 = higher HPV-positive likelihood). The cluster markers represent the clusters revealed with Agglomerative Clustering. See also Figure 8 -b.

Saliency Map Evaluation

Saliency map evaluation measures whether visual attributions faithfully, stably, and usefully reflect model decisions and human-relevant evidence.

See all 34 papers

Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology
Jun 4, 2026Yanqing Luo, Julius Hense, Niklas Prenißl +4Computational Pathology Explainable Medical Image Analysis
Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Institute of Pathology, Charité Universitätsmedizin, Berlin, Germany +10
SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology
Aug 3, 2026Abdallah Lamane, Abdul Rahman Diab, Ren-Chin Wu +1Computational Pathology Cancer Survival Prediction
Massachusetts Institute of Technology, Cambridge, MA, USA · Department of Data Science, Dana-Farber Cancer Institute, Boston, MA · Department of Pathology, Dana-Farber Cancer Institute, Boston, MA +2
Mixture of Mini Experts: Overcoming the Linear Layer Bottleneck in Multiple Instance Learning
Mar 23, 2026Daniel Shao, Joel Runevic, Richard J. Chen +4Multiple Instance Learning Histopathology Image Classification
Massachusetts Institute of Technology, Cambridge MA · Harvard University, Cambridge MA · Emory University, Atlanta GA +1

Jun 4, 2026J/K move · Enter open · S save

Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology

Yanqing Luo, Julius Hense, Niklas Prenißl, Andreas Mock+3

Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Institute of Pathology, Charité Universitätsmedizin, Berlin, Germany+10

Computational Pathology Explainable Medical Image Analysis Histopathology Image Classification Multiple Instance Learning

Open paper AskPDF arXiv

Explanations of multiple instance learning (MIL) models are widely used for validation and discovery in digital histopathology. Existing methods primarily rely on heatmaps that highlight influential regions but do not explain how evidence from different tissue regions is combined to produce a prediction. This limits interpretability, especially when decisions depend on interactions between tissue features. We introduce Symbolic explainable MIL (Symb-xMIL), a post-hoc explanation framework that quantifies how a MIL model's behavior aligns with human-readable decision rules, expressed as logical relationships (e.g., AND, OR, NOT) between input features. These alignment scores reveal semantic patterns underlying the model's predictions. We evaluate Symb-xMIL on synthetic and real-world histopathology datasets. On synthetic MIL data, Symb-xMIL reliably recovers ground-truth logical rules. In a clinical tumor detection task, the best-aligned rules uncover heterogeneous decision patterns and expose hidden model errors. On an HPV-prediction task on TCGA-HNSCC, a cohort of head and neck cancer, our framework refines patient survival stratification beyond HPV status with potential clinical relevance. Overall, Symb-xMIL extends MIL explainability beyond visual attribution toward structured, rule-based reasoning, enabling more transparent and semantically grounded interpretation of model predictions.

Abstract

Saliency Map Evaluation

Explore similar work

Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology

SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology

Mixture of Mini Experts: Overcoming the Linear Layer Bottleneck in Multiple Instance Learning