Can Protein-Derived Knowledge Improve Pathology Foundation Models?
Authors: Di Zhang, Zhangpeng Gong, Jiashuai Liu, Zhi Zeng, Jiusong Ge, Chunze Yang, Xitong Ling, Kai Yi, +5 more
Organizations: School of Computer Science and Technology, Xi’an Jiaotong University · Tsinghua Shenzhen International Graduate School, Tsinghua University · University of Cambridge · Saw Swee Hock School of Public Health, National University of Singapore · Bioinformatics Institute (BII), A*STAR · Department of Oncology, University of Cambridge · Institute of Intelligent Medicine, Peking Union Medical College
Molecularly guided pathology foundation models (PFMs) exploit transcriptomic or proteomic information to enrich whole-slide image (WSI) representations, yet effectively leveraging large standalone molecular corpora remains challenging. First, existing molecular foundation models encode protein sequences or single-cell states, not the patient-level bulk expression profiles paired with WSIs. Second, because cross-modal supervision is restricted to paired WSI-omics samples, knowledge from standalone molecular corpora reaches the pathology encoder only indirectly, creating a paired-support bottleneck. To address these challenges, we propose a three-stage framework that decouples proteomic knowledge acquisition from cross-modal transfer, yielding ProSlide, a slide-level hierarchical pathology foundation model. First, to close the modality gap, we pretrain a Proteomic Foundation Encoder (PFE) on 12,695 sample-level bulk protein profiles using virtual profile generation and expression-space multi-view pretraining. Second, we pretrain ProSlide, a patch-region-slide encoder, to predict protein expression from paired WSI-protein samples. Third, to relax the paired-support bottleneck, we introduce Prot2Path, a cross-modal relational distillation objective. For each paired sample, it aligns the similarity distributions of the WSI and its protein profile over a shared, frozen bank of PFE-encoded paired and standalone profiles. We evaluate ProSlide on 12 downstream tasks across breast, lung, and renal cancers. Despite being pretrained with only 2,229 WSIs and 12,695 sample-level protein profiles, ProSlide achieves the highest mean accuracy and AUC within each cancer group.
Figures & tables
Figure 1: (a) Granularity mismatch between existing molecular encoders and sample-level proteomic states. (b) Cross-modal transfer is restricted to paired WSI–omics anchors. (c) Our framework leverages standalone molecular profiles through cross-modal knowledge distillation.
Figure 2: Overview of the three-stage framework. (a) PFE learns proteomic representations through virtual profile generation and expression-space multi-view pretraining. (b) ProSlide learns hierarchical WSI representations through protein-expression prediction. (c) Prot2Path distills proteomic relational knowledge into ProSlide through similarity distribution alignment over a shared sampled protein bank.
Model
Renal
Breast
Lung
ACC
AUC
ACC
AUC
ACC
AUC
mean-pooling
57.82 ± 0.25
72.16 ± 0.80
67.74 ± 0.87
75.70 ± 1.01
59.61 ± 0.20
75.31 ± 0.20
FEATHER
54.95 ± 0.44
70.96 ± 1.01
66.79 ± 0.92
74.80 ± 0.82
60.22 ± 0.22
74.78 ± 0.26
GigaPath
56.04 ± 0.25
75.33 ± 0.72
67.19 ± 0.84
77.50 ± 1.08
58.93 ± 0.35
73.43 ± 0.27
PRISM
56.28 ± 0.36
72.72 ± 0.73
66.39 ± 0.88
74.88 ± 0.90
61.83 ± 0.16
74.86 ± 0.17
CARE
58.43 ± 0.31
75.24 ± 0.69
67.63 ± 0.67
75.07 ± 0.83
61.94 ± 0.20
76.99 ± 0.16
Table 1: Logistic-regression results averaged within the selected renal cell carcinoma (RCC), breast, and lung task groups. Each entry reports the mean of the four task-level means ± the arithmetic mean of their reported standard deviations.
Model
Mask Results
External Mask Results
Distillation Results
M1 R@1
M2 R@1
M3 R@1
M1 R@1
M2 R@1
M3 R@1
ACC
AUC
Protein-BERT
97.62
94.25
91.40
96.22
92.15
84.87
62.64
75.99
PFE & Bv=0
97.42
93.89
89.90
95.35
91.71
83.35
63.21
76.94
PFE & Bv=32
98.47
96.36
93.76
97.09
94.55
88.51
64.57
77.13
PFE & Bv=64
98.92
96.92
95.17
96.58
95.05
88.44
64.68
77.70
PFE & Bv=128
96.14
87.77
75.29
94.25
88.00
75.35
64.57
78.13
Table 2: Comparison of protein models on internal and external masked-protein retrieval tasks. The internal results are evaluated on 891 held-out TCGA samples, while the external results are evaluated using the combined 275-patient CPTAC gallery. All retrieval results are reported as the mean R@1 (%) over five runs.
S2-Sub
S2-Prot
S3-KD
Eff. rank
Renal
Breast
Lung
ACC
AUC
ACC
AUC
ACC
AUC
✓
✗
✗
11.95
52.61
69.79
57.94
69.56
60.12
73.18
✗
✓
✗
232.92
58.07
75.86
69.22
76.69
61.34
75.39
✗
✗
✓
295.43
59.44
74.88
68.20
77.51
61.92
76.94
✓
✗
✓
12.12
52.32
70.26
58.34
72.48
61.11
74.41
✗
✓
✓
201.29
60.85
76.67
70.02
78.53
63.17
77.90
Table 3: Ablation study of supervision and knowledge distillation. S2-Sub and S2-Prot denote Stage II subtype supervision and protein-expression supervision, respectively. S3-KD denotes Stage III knowledge distillation. Eff. rank denotes the effective rank of representations on the CPTAC datasets.
Method
Target
Reference
Renal
Breast
Lung
ACC
AUC
ACC
AUC
ACC
AUC
Direct alignment
Cosine
Paired only
60.07
77.86
69.22
78.69
62.34
77.39
Hard matching
CE
All profiles
60.83
76.49
68.47
76.57
62.89
77.69
Paired-bank KD
Soft KL
Paired only
60.62
76.61
69.35
77.65
62.69
77.47
Full-bank KD
Soft KL
All profiles
60.33
76.90
70.14
78.69
62.60
77.77
Prot2Path
Soft KL
Sub-bank
60.85
76.67
70.02
78.53
63.17
77.90
Table 4: Ablation of Prot2Path components. We evaluate relational targets, standalone references, and sub-bank construction.
Figure 3: Visualization comparison of original WSIs and heatmaps. Heatmaps overlay patch aggregation weights for multiple-instance models or final-layer [CLS]-to-image-token attention weights for transformer-based models onto the corresponding WSIs.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Resource
Records
Sample Domain
Protein Profile Type
TCGA
7,349
Patient tumors
Baseline tumor RPPA
TCPA
878
Cancer cell lines
Baseline cell-line RPPA
CCLE
899
Cancer cell lines
Baseline cell-line RPPA
CPPA
4,460
Perturbed cell lines
Treatment-induced Δp
Total
13,586
–
–
Appendix
Table S1: Composition of the proteomic cohort.
Panel Name
Standardized Target
Functional Annotation
ATM
ATM
DNA damage response
ATR
ATR
DNA damage response
BRCA2
BRCA2
Homologous recombination repair
P53
TP53
Tumor suppression / DNA damage response
RB
RB1
Cell-cycle regulation
BAP1C4
BAP1
Chromatin regulation / tumor suppression
Appendix
Table S2: Composition of the 38-protein panel. Total-protein and site-specific phosphorylation measurements are retained as distinct targets.
Component
Configuration
Description
Protein panel size
38
Shared protein vocabulary
Input sequence length
39
One [CLS] and 38 protein tokens
Value projection gval
1→32→768
Projects continuous abundance values
Missing-value embedding
1×768
Shared learnable value-mask embedding
Hidden dimension
768
Transformer token dimension
Transformer depth
7
Number of encoder blocks
Appendix
Table S3: Architecture configuration of the Proteomic Foundation Encoder (PFE). PFE represents each protein as the sum of a learnable identity embedding and a continuous abundance-value embedding, followed by Transformer-based contextual modeling.
Category
Hyperparameter
Value
Expression views
Profile-wide views
3
Profile-wide view composition
1 clean + 2 posterized
Subset-perturbed views
6
Subset-perturbed Gaussian noise
σ=0.05
Gaussian perturbation ratio
30% of observed proteins
Protein masking
Profile-wide student mask ratios
0.05 / 0.10 / 0.15
Appendix
Table S4: Self-supervised pretraining configuration of PFE. The model is trained using Distributed Data Parallel (DDP) on four NVIDIA RTX 4090 GPUs.
Model
Primary Input
Learning Objective
Representation / Output
ESM-2 [ Lin et al., 2022 ]
Amino-acid sequences
Masked amino-acid prediction
Residue and protein embeddings
AlphaFold 2 [ Jumper et al., 2021 ]
Sequences, alignments, and structural templates
Structure prediction
Three-dimensional protein structures
AlphaFold 3 [ Abramson et al., 2024 ]
Biomolecular sequences and ligand identities
Diffusion-based structure prediction
Three-dimensional biomolecular complexes
scPROTEIN [ Li et al., 2024 ]
Single-cell peptide and protein measurements
Graph contrastive learning and uncertainty estimation
Cell embeddings
scpFormer [ Zhou et al., 2026 ]
Single-cell protein abundance and sequence-derived embeddings
Multi-objective masked expression reconstruction
Cell and contextualized protein embeddings
VirTues [ Wenckstern et al., 2026 ]
Multiplex protein images and sequence-derived marker embeddings
Masked image reconstruction
Marker, cell, niche, and tissue representations
Appendix
Table S5: Comparison of representative protein modeling approaches. We summarize primary inputs, principal learning objectives, and representations or outputs.
Parameter
Configuration
Patch encoder
CONCH v1.5 (frozen)
Patch feature dimension
768
Region grid
4×4 patches
RSA depth
3
RSA attention heads
8
RSA MLP ratio
2
Appendix
Table S6: Architectural configuration of ProSlide.
Parameter
Configuration
Proteomic teacher
Frozen PFE
Teacher representation
Final [CLS] output after LayerNorm
Representation dimension
768
Candidate pool size
12,695 real profiles
Sub-bank size
Ksub=1,594
Paired references
All 1,338 paired training profiles
Appendix
Table S7: Implementation configuration of Prot2Path.
Task
Cohort(s)
Patients ( Np )
WSIs ( Ns )
Task 1
MUT-HETRCC
1292
1292
Task 2
MUT-HETRCC
1292
1292
Task 3
CPTAC-CCRCC
103
245
Task 4
MUT-HETRCC + CPTAC-CCRCC
1395
1537
Task 5
CPTAC-BRCA
97
106
Task 6
CPTAC-BRCA
82
89
Appendix
Table S8: Task-specific cohort sizes for downstream evaluation.
Model
Patch Encoder
Slide Aggregation
Pretraining Scale
Pretrain Sup.
CARE
CONCH v1.5
CARE
34,277 WSIs
iBoT & Clip
TITAN
CONCH v1.5
ViT
335,645 WSIs
iBoT & Coca
PRISM
Virchow
Perceiver
587,196 WSIs
Coca
GigaPath
GigaPath
LongNet
171,189 WSIs
MAE
FEATHER
CONCH v1.5
ABMIL
24,000 WSIs
Supervised classification
ProSlide
CONCH v1.5
Region ViT
12,695 protein profiles
Protein supervision &
Appendix
Table S9: Comparison of representative slide-level pathology foundation models. We compare the patch encoder, whole-slide aggregation strategy, pretraining scale, and supervision of the pathology foundation models considered in this work.
Task
Metric
Mean-pooling
FEATHER
GigaPath
PRISM
CARE
TITAN
ProSlide
Task 1
ACC
54.52 ± 0.04
54.88 ± 0.07
55.98 ± 0.04
55.93 ± 0.11
56.72 ± 0.02
56.29 ± 0.07
61.65 ± 0.20
AUC
70.32 ± 0.13
73.41 ± 0.10
72.71 ± 0.05
72.29 ± 0.10
73.54 ± 0.08
72.56 ± 0.06
76.09 ± 0.07
Task 2
ACC
60.95 ± 0.04
54.74 ± 0.03
61.44 ± 0.13
57.55 ± 0.15
61.44 ± 0.07
59.83 ± 0.14
62.87 ± 0.16
AUC
84.84 ± 0.06
84.28 ± 0.05
86.57 ± 0.01
86.49 ± 0.01
88.89 ± 0.02
85.98 ± 0.01
86.89 ± 0.03
Task 3
ACC
57.50 ± 0.83
58.14 ± 1.61
55.18 ± 0.65
53.98 ± 1.13
58.70 ± 1.03
61.20 ± 1.43
60.39 ± 1.14
AUC
62.61 ± 2.92
58.23 ± 3.68
60.08 ± 2.79
59.52 ± 2.78
65.83 ± 2.61
73.16 ± 2.08
67.69 ± 2.31
Appendix
Table S10: Logistic-regression results on the selected 12-task set. Values are mean ± standard deviation. The best and second-best means within each task-metric row are shown in bold and underlined, respectively.
Figure S1: Data and memory efficiency. ProSlide achieves strong performance with limited slide-level pretraining data (a) and low GPU memory usage during inference (b).
Figure S2: PFE protein identity embeddings. Pairwise cosine similarities among 38 learned protein identity embeddings. Both axes follow the same clustering order, with warmer colors indicating higher similarity.
Computational pathology needs whole-slide image (WSI) foundation models that transfer across diverse clinical tasks, yet current approaches remain largely slide-centric, often depend on private data and expensive paired-report supervision, and do not explicitly model relationships among multiple slides from the same patient. We present MOOZY, a patient-first pathology foundation model in which the patient case, not the individual slide, is the core unit of representation. MOOZY explicitly models dependencies across all slides from the same patient via a case transformer during pretraining, combining multi-stage self-supervision with scaled low-cost task supervision. In Stage 1, we pretrain a vision-only slide encoder on 77,134 public slide feature grids using masked self-distillation. In Stage 2, we align these representations with clinical semantics using a case transformer and multi-task supervision over 333 tasks from 56 public datasets, including 205 classification and 128 survival tasks across four endpoints. Across sixteen held-out tasks, MOOZY improves macro weighted F1, balanced accuracy, and macro weighted ROC-AUC relative to PRISM by +4.19%, +7.93%, and +6.95%, respectively. MOOZY is also parameter efficient with 85.77M parameters, 14× smaller than GigaPath. These results suggest that patient-level pretraining yields transferable embeddings, providing a path toward scalable patient-first histopathology foundation models.
Yousef Kotp, Vincent Quoc-Huy Trinh, Christopher Pal +1
CSSE, Concordia University, Montréal, Canada · Mila – Québec AI Institute, Montréal, Canada · CHUM, Université de Montréal, Montréal, Canada +4
Foundation models have recently achieved impressive success in computational pathology, demonstrating strong generalization across diverse histopathology tasks. However, existing models overlook the heterogeneous and non-uniform organization of pathological regions of interest (ROIs) because they rely on natural image backbones not tailored for tissue morphology. Consequently, they often fail to capture the coherent tissue architecture beyond isolated patches, limiting interpretability and clinical relevance. To address these challenges, we present Cross-modal Adaptive Region Encoder (CARE), a foundation model for pathology that automatically partitions WSIs into several morphologically relevant regions. Specifically, CARE employs a two-stage pretraining strategy: (1) a self-supervised unimodal pretraining stage that learns morphological representations from 34,277 whole-slide images (WSIs) without segmentation annotations, and (2) a cross-modal alignment stage that leverages RNA and protein profiles to refine the construction and representation of adaptive regions. This molecular guidance enables CARE to identify biologically relevant patterns and generate irregular yet coherent tissue regions, selecting the most representative area as ROI. CARE supports a broad range of pathology-related tasks, using either the ROI feature or the slide-level feature obtained by aggregating adaptive regions. Based on only one-tenth of the pretraining data typically used by mainstream foundation models, CARE achieves superior average performance across 33 downstream benchmarks, including morphological classification, molecular prediction, and survival analysis, and outperforms other foundation model baselines overall.
Di Zhang, Zhangpeng Gong, Xiaobo Pang +14
Xi’an Jiaotong University · University of Cambridge · KingMed +2
Foundation Models (FMs) have recently redefined the state-of-the-art in histopathology by providing robust representations for whole-slide image (WSI) analysis. However, selecting the optimal foundation model (FM) for a specific clinical cohort currently requires multiple preprocessing steps, followed by computationally expensive feature extraction and the training of a Multiple Instance Learning (MIL) aggregator for every model. In this work, we investigate whether efficient tile-level linear probing can serve as a reliable proxy for slide-level performance, reducing the need to run full slide-level pipelines for every candidate encoder. We benchmark 19 state-of-the-art FMs on 42 slide-level and 16 tile-level tasks, comparing tile probing metrics against slide-level outcomes using ABMIL and Mean Pooling aggregations. We observe a high correlation between tile and slide performance across varying task difficulties, indicating that encoder representation quality is the primary determinant of WSI success. Sensitivity analyses show that transferability is stable across models and is more influenced by cohort sizes and numbers of tiles per slide than by average task difficulty. We also measure the agreement in best performing models between tile and slide-level tasks, showing tile benchmarks reliably shortlist strong candidates. Overall, our study indicates that tile-level benchmarking provides an efficient and practical first step for narrowing down candidate models, while slide-level evaluation remains essential for final validation on clinical tasks.
Sofiène Boutaj, Leo Fillioux, Maria Vakalopoulou +2
Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, IHU PRISM, Cancer Data Science Unit, France · Université Paris-Saclay, CentraleSupélec, MICS Laboratory, France