cs.AIAug 28, 2026
SaveExplainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations
Organizations: Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA · Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianapolis, IN, USA · Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK · Department of Biomedical Engineering, Emory University, Atlanta, GA, USA · Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands · Singapore General Hospital, Singapore · Department of Computer Science, School of Computer Science and Mathematics, Kingston University London, London, UK · Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada · Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · University of California, Irvine, School of Medicine, CA, USA · Toronto Metropolitan University, Toronto, ON, Canada · Departments of Biostatistics and Health Data Science; Radiology and Imaging Sciences; Neurological Surgery, Indiana University School of Medicine, Indianapolis, IN, USA · Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University, Indianapolis, IN, USA
Abstract
Computational pathology (CompPath) is transforming medicine by leveraging artificial intelligence (AI) algorithms to support diagnosis, prognosis, and treatment prediction from gigapixel whole-slide images. Clinical adoption is progressing, but is constrained by concerns about safety, accountability, and regulatory oversight in high-stakes clinical environments. Explainable AI (XAI) systems hold promise for building trust and enabling verification, yet the literature remains fragmented due to inconsistent terminology, overlapping methodological families, ad hoc validation, and current reviews. This review aims to formalize XAI methods in CompPath through the: i) introduction of a pathology-centric vocabulary comprising seven core terms; ii) development of a taxonomy across methodological families and three orthogonal axes (stage, type, scope); and iii) establishment of a task-driven framework that maps five clinical questions to recommended methods, method evaluation, and deployment context. Five key gaps between current XAI capabilities and clinical deployment are identified, and actionable steps are proposed to advance XAI for CompPath.
Figures & tables
Figure 1: CompPath pipelines comprise data acquisition, model training, and outcome generation. Inputs span WSIs and tissue microarrays, as well as pathology reports, structured clinical and laboratory data, molecular profiles, and prior radiology findings. Outcomes include classification, segmentation and biomarker scores to text outputs generated by language and vision-language models, including structured-report drafts, concept-grounded retrieval results, and natural-language rationales. XAI can be integrated at any part of such pipeline to assess whether predictions rely on clinically relevant evidence.
| Term | Core Question | Pathology Example | Evaluation Focus |
|---|---|---|---|
| Interpretability | Can a pathologist understand the model’s reasoning directly from its structure or native outputs, without an added explanation? | Predicting tumor grade from nuclear area, mitotic count, & gland density (structure-level); nuclear segmentation (output-level: inspectable output map). | Evaluated without generating external explanation. Two modes: (a) structure-level , for simple parametric models (linear, generalised additive, sparse concept bottleneck) (b) output-level , for complex models with a pathologist-recognisable quantity (segmentation map, mitotic count, Gleason grade) |
| Explainability | Is faithful, context-appropriate evidence provided alongside the model’s output? | Grad-CAM heatmap, concept-activation score for cribriform architecture, or calibrated probability accompanying a (for example) Gleason 4 prediction. | Explanation’s faithfulness to the model, stability across scanners, stains, & clinical plausibility; Categorizations of explanation-evaluation criteria identify 6 categories (faithfulness, robustness, localization, complexity, axiomatic, randomisation) [ 47 ] . Explanations are evaluated as artifacts and not assumed to reveal the model’s true internal reasoning. |
| Transparency | Is enough information disclosed to scrutinise, reproduce, monitor, and govern the system? | Public documentation of model architecture & parameters; preprocessing details; subgroup performance across variables; known failure modes. | Completeness of the system-level disclosure; checklist-style against the relevant regulatory framework (FDA PCCP, EU AI Act). |
| Causal validity | Does the highlighted feature influence the model’s prediction? | Occluding a Grad-CAM-highlighted tumor region degrades the model’s prediction, confirming model-causal influence. | Interventional or counterfactual evidence the highlighted feature influences the prediction (AOPC, insertion/deletion AUC, ROAR); distinguishes correlation from model-level causation. |
| Causability | Does the explanation help a pathologist form a useful, clinically meaningful causal understanding? | Highlighting slide edges or artifacts (low causability) vs cribriform glandular architecture for a high-grade prostate prediction (high causability). | Expert agreement with the explanation; alignment with established clinical criteria (Gleason, Nottingham, WHO); evaluates the explanation from the user’s clinical perspective. |
| Justifiability | Can the deployment of the model and the use of its outputs be reasonably defended? | Defending a Gleason-grading tool with evidence of analytical & clinical validation, calibration, subgroup fairness, & standard of care alignment. | Completeness & quality of the validation, fairness, & regulatory dossier for the deployment decision. |
Table 1: CompPath XAI terminology, treated as distinct, complementary concepts with different scopes and evaluation criteria.
Figure 2: Taxonomies of XAI methods in CompPath. (a) the Stage-Type-Scope taxonomy describes 3 orthogonal dimensions for each XAI method. Stage (intrinsic vs. post hoc) addresses when the explanation is produced. Type (model-specific vs. model-agnostic) addresses how dependent the explanation is on the model’s architecture. Scope (local vs. regional/slide vs. global) addresses the level at which the explanation operates. (b) XAI Methodological Families and their positioning within the Stage-Type-Scope taxonomy.
Figure 3: Task-driven recommendation framework for selecting XAI methods in CompPath according to “Explanation questions". Five clinical questions (spatial localization, feature attribution, causal validation, concept-/example-based reasoning, and decision-boundary assessment) map to recommended methods, validation evidence, and deployment use. All the rows are complementary rather than sequential: spatial localization and feature attribution provide associational evidence, causal validation tests whether highlighted regions influence the model output, concept- and example-based methods translate model behaviour into pathologist-aligned reasoning, and decision-boundary methods identify unstable, ambiguous, or out-of-distribution cases that warrant review. Validation evidence and deployment requirements for each method family are summarised in Table 3 – 7 and discussed in Supplementary Section S4 .
| Method | Section | Stage | Type | Scope |
|---|---|---|---|---|
| Backpropagation-based | ||||
| Trainable Attention [ 48 ] | 3.1.1 | |||
| Transformer-based Attention [ 49 ] | 3.1.2 | |||
| CAM [ 50 ] | 3.1.3 | |||
| Grad-CAM [ 41 ] | 3.1.4 | |||
| Saliency Maps [ 51 ] | 3.1.5 | |||
Table 2: XAI methods in CompPath categorized by families and the Stage-Type-Scope taxonomy. Legend: Type: Agnostic , Specific ; Scope: Local , Global ; Stage: Post hoc , Intrinsic
| Method | Strengths | Limitations | Cautions & best practices |
|---|---|---|---|
| Attention (MIL / Transformer) | Highlights decision-driving patches; interpretable at the instance level; integrates naturally with weakly supervised slide-level learning. | Attention weights capture correlation, not causation; sensitive to site-specific bias, stain, and scanner artefacts; diffuse across heads/layers in transformers. | Validate attention maps against expert ROIs and morphometric features; test stability across scanners, stains, and institutions [ 12 ] . |
| CAM | Simple, fast, intuitive; identifies broad discriminative regions with minimal overhead. | Requires a GAP layer; coarse spatial resolution misses subtle nuclear or sub-cellular features. | Use for rapid qualitative visualisation; confirm findings with higher-resolution methods or expert annotations. |
| Grad-CAM | Architecture-agnostic; widely supported; produces visually intuitive, class-specific localisations on standard CNNs. | Coarse heatmaps; poor at localising multiple instances of the same class; gradients can be attenuated or distorted under MIL pooling; no signed relevance. | Report the target convolutional layer; run sanity checks [ 18 ] ; verify faithfulness via insertion/deletion [ 20 ] and cross-site stability [ 12 ] . |
| Grad-CAM++ | Better multi-instance localisation than Grad-CAM through higher-order gradient weighting; handles multiple objects in a single image. | Same correlational limitation as Grad-CAM; sensitive to gradient noise in very deep networks. | Use when multiple instances of the target class are expected; benchmark against Grad-CAM on the same model; verify cross-site stability [ 12 ] . |
| Saliency maps | Pixel-level sensitivity; single-pass computation per tile; useful as a baseline against which newer methods are evaluated. | Noisy and unstable; highly sensitive to preprocessing, stain normalisation, and scanner colour variation; often emphasises edges over morphology. | Smooth or combine with IG; assess stability under stain/rotation perturbations; confirm overlap with expert annotations. |
| SmoothGrad | Reduces high-frequency noise in saliency by averaging over noise-perturbed inputs; can wrap around any gradient-based method; produces spatially coherent maps. | extra forward and backward passes per attribution; sensitive to noise level and sample count ; can over-smooth and lose fine morphological detail. | Report and ; benchmark against vanilla saliency; verify cross-site stability [ 12 ] . |
Table 3: Summary of backpropagation-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting.
| Method | Strengths | Limitations | Cautions & best practices |
|---|---|---|---|
| LIME | Model-agnostic; provides an intuitive local surrogate of black-box behaviour. | Unstable across runs; sensitive to kernel width, segmentation, and sampling; standard pixel perturbations are not biologically plausible in WSIs. | Use meaningful superpixel or tissue-region segmentations; report kernel width and sampling parameters; assess stability via rank correlation across repeated runs. |
| Occlusion sensitivity | Interventional test of region importance; gradient-free; conceptually simple. | Slow; sensitive to mask size and shape; zero-masking introduces non-tissue artefacts that distort attribution. | Use biologically plausible perturbations such as blur, inpainting, or stain jitter; test multiple mask sizes; report insertion/deletion curves and cross-site consistency [ 12 ] . |
| RISE | Model-agnostic Monte Carlo estimator; averages noise across many random masks; extends to transformers via token sampling. | Requires thousands of mask samples for convergence; coarse spatial resolution; computationally intensive at WSI scale. | Report number, size, and stride of masks; cross-validate against occlusion or IG; evaluate overlap with expert-defined regions. |
Table 4: Summary of perturbation-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting.
| Method | Strengths | Limitations | Cautions & best practices |
|---|---|---|---|
| SHAP | Additive, theoretically grounded attributions; bridges local and global explanations; interpretable at biologically meaningful feature scales. | Computationally expensive, especially Kernel SHAP; results depend on feature definition and background dataset; noisy at the pixel level. | Define features at biologically interpretable scales such as cells, glands, and tissue regions; report background set and sampling strategy; compare global SHAP rankings with established pathology markers. |
| Activation layer visualisation (ALV) | Reveals hierarchical feature learning across CNN layers; useful as a representation-analysis and debugging tool. | Qualitative only; interpretation is subjective and observer-dependent; storing feature maps is memory-intensive at the WSI scale. | Correlate activated channels with morphometric features; assess consistency across staining protocols and scanner domains [ 12 ] ; do not present as case-level evidence. |
Table 5: Summary of feature-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting.
| Method | Strengths | Limitations | Cautions & best practices |
|---|---|---|---|
| TCAV | Quantifies the influence of human-defined concepts on model predictions; enables hypothesis-driven auditing. | Assumes linear concept separability in latent space; sensitive to quality and diversity of concept exemplars and to choice of network layer. | Evaluate multiple concept sets across layers; use held-out exemplars from diverse sites and scanners; include negative-control concepts to detect spurious correlations. |
| Concept-bottleneck models (CBM) | Intrinsically interpretable; predicts pathologist-defined concepts as intermediate outputs; supports quantitative auditing of concept-level reasoning. | Requires dense, concept-specific expert annotations; fixed concept sets may miss composite morphologies; bottleneck can constrain learning. | Validate concept predictions against expert annotation and inter-reader variability; assess cross-site and cross-scanner concept consistency [ 12 ] ; monitor concept leakage via ablation. |
| Image captioning | Generates natural-language summaries of regions or slides; supports report-style outputs and multi-modal reasoning. | Prone to hallucination; data-hungry; requires curated paired image-text corpora; quality depends heavily on the underlying language model. | Develop standardised terminology with pathologists; ground captions to ROI-level evidence; verify factual correctness against ground-truth reports. |
| Vision-language models (VLM) | Bidirectional alignment of image regions and pathology terminology; zero-shot classification and semantic retrieval without explicit labels. | Susceptible to semantic drift; needs large, high-quality paired datasets; visual–semantic grounding requires additional verification. | Evaluate both visual grounding and semantic grounding; conduct expert review of retrieved examples; test across institutions and scanners. |
Table 6: Summary of concept-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting.
| Method | Strengths | Limitations | Cautions & best practices |
|---|---|---|---|
| Prototypes | Intuitive case-based reasoning that aligns with routine clinical practice; surfaces typical supporting and opposing examples; identifies out-of-distribution cases. | Representativeness depends on training-data diversity and embedding quality; clustering choices can over-represent frequent artefacts. | Curate multi-institutional prototype banks; assess pathologist agreement on prototype relevance; analyse class boundaries via counter-prototypes. |
| Counterfactuals | Test decision boundaries through targeted, interpretable edits; support causal-style reasoning and failure-mode discovery. | Generating realistic counterfactual histology is hard; synthetic images may introduce artefacts or violate biological constraints; interpretation requires expert review. | Conduct expert plausibility review; confirm that small, realistic edits produce consistent class changes; enforce morphological and staining constraints. |
| Trust scores | Lightweight, data-driven proxy for prediction reliability; flags out-of-distribution predictions; complements visual attribution as a triage signal. | Sensitive to embedding quality and domain shift; not an explanation of model reasoning; may assign high confidence to artefact-driven predictions. | Calibrate thresholds separately by site and scanner [ 12 ] ; correlate with reader disagreement and known errors; jointly inspect low-trust cases with attribution maps. |
Table 7: Summary of example-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting.
| Anatomy | Method |
|---|---|
| Bladder,Brain | CAM [ 69 ] |
| Grad-CAM [ 110 , 111 ] | |
| Attention [ 112 , 113 , 67 , 114 ] | |
| Head & Neck | Grad-CAM [ 115 ] |
| Breast | Activation Layer Visualization [ 116 ] |
| Grad-CAM [ 117 , 118 ] |
Table 8: Studies reviewed in this article with corresponding XAI method in specific anatomies.