EXPOSE: Explainable and Domain-Robust Embeddings from Pathology Vision Foundation Models using Sparse Autoencoders
Authors: Anja Witte, Maximilian Lennartz, Jan Baumbach, Guido Sauter, Stefan Bonn, Patrick Fuhlert, Marina Zimmermann
Organizations: Institute of Medical Systems Bioinformatics, Center for Biomedical AI (bAIome), Center for Molecular Neurobiology Hamburg (ZMNH), Hamburg Center for Translational Immunology (HCTI), University Medical Center Hamburg-Eppendorf, Hamburg, Germany · Institute of Pathology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany · Institute of Computational Systems Biology, University of Hamburg, Hamburg, Germany · German Center for Child and Adolescent Health (DZKJ), Hamburg, Germany · III. Department of Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
Vision Foundation Models (VFMs) are widely used in computational pathology but remain sensitive to domain shifts arising from variations in staining, tissue preparation, and scanner hardware. A key limitation is that VFM embeddings entangle biological with domain-specific information, hindering cross-domain generalization. We propose Explainable Probing of Cross-Domain Sparse Embeddings (EXPOSE), a framework that uses Sparse Autoencoders (SAEs) as an explainable bottleneck to identify and suppress domain-specific components in VFM embeddings. We train a sparse representation of VFM features, use a linear classifier to identify domain-specific latent dimensions, and mask these features prior to downstream relapse prediction without retraining the backbone model. Experiments on a large prostate cancer dataset with multiple acquisition domains show that SAE features capture both domain- and task-specific information, which are partially disentangled in the latent space. Removing domain-specific features improves cross-domain performance and increases embedding robustness as measured by the Domain Robustness Index (DoRI). Code is available at https://github.com/imsb-uke/expose .
Figures & tables
Figure 1: Overview of expose . (A) An sae is trained in an unsupervised manner on vfm patch embeddings of two domains to learn cross-domain and disentangled representations. (B) The sae encoded features are mean-pooled and a linear classifier is trained to predict the domain of these slide-level representations. Feature dimensions are ranked according to their weights in the domain classifier. The features with the top- k highest weights are defined as domain-specific features. (C) The mean-pooled features identified as domain-specific are set to zero. A linear classifier is trained to predict the binary downstream task of relapse prediction. The most relevant features for both domain and relapse classes are linked to visual features.
Figure 2: Evaluation of relapse prediction while masking the top- k domain-specific features chosen with different selection strategies based on the linear weights for domain classification. Top negative and top positive weights can be defined as relevant for first and scanner classification, respectively. Features that are relevant for both domains in the domain classifier are chosen by selecting the features with the top absolute domain weights. Validation results are shown for the evaluation of first (A) and scanner (B) dataset.
Figure 3: Visualization of highly activating images and patches for domain- and relapse-specific sparse features, respectively. For each ID, examples are chosen from the first (left panel) and scanner (right panel) subdataset separately. For the top negative (A) and top positive (B) domain-specific features, we show the three most strongly activating tma spots together with activation heatmaps. The top negative (C) and top positive (D) relapse-specific features are visualized with three highly activating patches and their surrounding tissue context.
Figure 4: Analysis of the sae embedding space. In (A) and (B) the sae feature dimension are visualized as activation delta between first and scanner . The UMAPs are coloured by the feature weights for domain classification (A), and relapse classification (B). In (C) and (D), we evaluate the Domain Robustness Index (DoRI) for different numbers of masked features. Gray and purple bars represent masked features that are chosen randomly and according to its domain weight, respectively. The dori is calculated within the embedding space of in-domain (C) and all subdatasets combined (D).
Dataset
Base
SAE
SAE 256+
SAE 3500+
first
68.20 ± 0.26
69.20 ± 0.41
69.39 ± 0.35
69.23 ± 0.08
scanner
68.58 ± 0.31
69.44 ± 0.39
69.75 ± 0.17
69.13 ± 0.16
spot
65.38 ± 0.56
65.53 ± 0.76
65.83 ± 0.71
67.74 ± 0.11
thin
60.48 ± 1.47
57.78 ± 0.48
57.80 ± 0.47
58.63 ± 0.15
thick
58.17 ± 0.74
58.40 ± 1.48
58.44 ± 0.69
59.62 ± 0.17
long
61.67 ± 1.09
61.79 ± 1.09
62.04 ± 0.58
62.67 ± 0.06
Table 1: Cross-domain evaluation of relapse prediction. Results show auroc5 (mean ± std) for Base, sae , and sae with masking of domain-specific sparse features ( k=256 , k=3500 ). Highlighted values indicate per-dataset best and second-best performance.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Dataset
Images
Tissue Core
Slicing Thickness
Staining Time in min (H/E)
Scanner
Mag.
first
8141
1
2.5 μ m
4:00 / 1:20
Aperio
40x
scanner
8141
1
2.5 μ m
4:00 / 1:20
3DHistech
80x
spot
8181
2
2.5 μ m
4:00 / 1:20
Aperio
40x
thin
1951
1
1 μ m
4:00 / 1:20
Aperio
40x
thick
1950
1
10 μ m
4:00 / 1:20
Aperio
40x
long
1951
1
2.5 μ m
40:00 / 10:00
Aperio
40x
Appendix
Table 2: Overview of additional dataset variants derived from the first dataset. first follows the standard acquisition protocol, while other datasets introduce controlled variations in scanner, staining time, slicing thickness, or sampling location. Differences from the standard protocol are shown in bold. Tissue cores are obtained from two different regions of the prostate. first and scanner contain the registered images of the same tma spots scanned with different scanners. All other subdatasets of tissue core 1 are different slices of the same core.
Pathology foundation models (FMs) produce powerful tile-level representations which remain sensitive to scanner and staining variability, undermining deployment across laboratories. We develop a novel fine-tuning recipe that improves the robustness of pathology FMs to acquisition factors. Applied to ten different FMs, our fine-tuning strategy consistently improves robustness for every model as well as downstream performance, with no observed trade-off. On average, it raises the PathoROB robustness index by 23% (from 0.72 to 0.87) and increases the overall cross-benchmark performance by 43% on Patho-Bench, HEST and THUNDER combined, with individual gains reaching up to 72% in robustness (Phikon-v2) and 76% in performance (Midnight-12k). We publicly release the fine-tuned versions of Phikon-v2 (Phaet) and Midnight-12k (Mascaret) at https://huggingface.co/wearewaiv/models.
Alexandre Filiot, Oskar Thaeter, Benoit Schmauch +1
1Waiv · Institute of Pathology, Technical University of Munich · School of Computation, Information and Technology, Technical University of Munich
Pathology foundation models (PFMs) have advanced rapidly in recent years and support training classifiers for a range of histopathology tasks. However, their robustness across hospitals remains limited: performance often degrades when training a classifier on data from one hospital and evaluating it on another target hospital. We address this challenge by fine-tuning PFMs with a local maximum mean discrepancy (LMMD) objective that applies to two settings: domain adaptation, where unlabeled target-hospital data is available, and domain generalization, where target-hospital data is unavailable at all. Experiments at both the patch- and slide-level show consistent improvements across multiple PFMs and tasks.
Ben Vardi, Dana Schonberger, Yuval Friedmann +4
Reichman University, Herzliya, Israel · Institute of Pathology, Sheba Medical Center, Ramat-Gan, Israel · Technion - Israel Institute of Technology, Haifa, Israel
Pathology foundation models (PFMs) have emerged as a core approach for learning transferable representations from whole slide images (WSIs), and they are typically benchmarked through downstream clinical endpoints. While such task level evaluations are indispensable, they offer limited insight into what the representations themselves encode, particularly whether PFM embeddings can distinguish meaningful tissue regions and capture their spatial relationships. We present SpaPath-Bench, a representation level benchmark designed to diagnose spatial representation capability in PFMs. SpaPath-Bench formulates spatial domain identification (SDI) on paired whole slide image and spatial transcriptomics (ST) data as a diagnostic task. It curates 42 public paired WSI and ST slides, enables large scale evaluation across 19 encoders and seven SDI methods, and measures partition quality using three complementary criteria: unsupervised spatial coherence, transcriptomics referenced agreement, and expert referenced agreement. Across 83K runs, SpaPath-Bench reveals that different pretraining paradigms capture distinct aspects of tissue spatial architecture, and it provides practical guidance for building the next generation of spatially aware computational pathology models. Code and data pipelines are publicly available at https://bokai-zhao.github.io/SpaPath-benchboard/.
Bokai Zhao, Yiyang Zhang, Yuanchi Zhu +6
School of Artificial Intelligence, University of Chinese Academy of Sciences. · Brainnetome Center, Institute of Automation, Chinese Academy of Sciences. · Beijing Key Laboratory of Brainnetome and Brain-Computer Interface, Institute of Automation, Chinese Academy of Sciences. +2