WILSON - a pathology foundation model framework for patient-level analysis and diagnostic text generation
Authors: Saghir Alfasly, Wataru Uegami, Sobhan Hemati, Wenchao Han, Xiaojia Tang, Kevin Thompson, Daniel Stone, Ghazal Alabtah, +8 more
Organizations: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA. · Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA. · Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA. · Mayo Clinic Comprehensive Cancer Center, Mayo Clinic, Rochester, MN, USA. · Division of Breast and Melanoma Surgical Oncology, Department of Surgery, Mayo Clinic, Rochester, MN, USA. · Department of Oncology, Mayo Clinic, Rochester, MN, USA.
Pathologists integrate morphology across magnifications and across the slides of a patient case, whereas pathology foundation models encode thousands of tiles from single slides and aggregate their features. Here we present WILSON, a vision--language foundation model that represents whole-slide images and multi-slide cases as single multi-magnification composite images, trained on approximately 189k slides from Mayo Clinic spanning 42 organs and 829 diagnostic entities using pathology reports as supervision. Without task-specific training, WILSON exceeded a dedicated case-level model on all internal cohorts (macro-F1 0.52 versus 0.38) and matched slide-level models up to 9.4 times larger at 272- to 2,155-fold lower compute. End-to-end fine-tuning on 508 triple-negative breast cancer cases improved histologic subtyping and stromal tumor-infiltrating lymphocyte grading by 0.16 and 0.11 macro-F1. WILSON retrieved matching diagnostic text at 75.6% recall@1 (PRISM, 58.1%) and generated captions closer to report-derived references than PRISM and PRISM2 on the internal cohort and on most external comparisons. Composite images thus offer a compact, clinically aligned computational unit for pathology.
Figures & tables
Figure 1: Overview of WILSON. a , Slides and cases per organ (bars), with the cohort further grouped by organ and by diagnostic category (donuts). b , An LLM cleans each pathology report and generates three paraphrased caption variants per report. c , Composites are tiled from selected WSIs; 178,020 WSIs have both a rendered composite and a caption record, giving 1,759,787 composites ( ≈ 9.9/WSI) and 1,759,787×3=5,279,361 ( ≈ 5.28M) image–caption pairs. d , WILSON pretraining and training: tile-based SSL on 256×256 patches at 2.5×/10×/20×→ composite-based SSL on 512×512 four-tile composites → vision–language alignment of 2048×2048 WSI composites against report text (SigLIP distillation from Gemini embeddings, CLIP, and keyword regression) → CoCa-style report generator, a multimodal decoder that cross-attends over pooled WILSON visual tokens to generate a caption autoregressively. e , Vision-encoder size, pretraining data (WSIs) and cost (FLOPs per 10k-tile WSI) for WILSON (133.7M; ≈ 189k; 2.6 T) versus MOOZY (85.8M; ≈ 77k; 25.1 T), TITAN (354.6M; ≈ 335k; 703.2 T), PRISM (730.3M; ≈ 587k; 4.07 P), PRISM2 (1,251M; ≈ 2.3M; 4.13 P) and Prov-GigaPath (1,220M; ≈ 171k; 5.56 P). f , Report-generation performance averaged over three cohorts (Mayo, HistAI, TCGA), as a normalized score: PRISM 27.1, PRISM2 34.9, WILSON 58.1, WILSON exceeds PRISM2 by 23.2 points with ≈ 9 × fewer parameters and ≈ 1,600 × fewer FLOPs per WSI.
Figure 2: WILSON matches or exceeds specialized case- and WSI-level baselines and benefits from efficient end-to-end fine-tuning. a , Multi-magnification composites versus Yottixel tile selection as alternative WSI representations, both encoded with the pretrained UNI foundation model. Top-1 macro-averaged F1 retrieval is shown for four WSI-level benchmarks, split across two sub-axes with different y -ranges: BCTherapy and TCGA-Kidney (left, 0 – 1 ) and MayoSkin and MayoBreast (right, 0 – 0.5 ). Bars are single point estimates. b , Case-level zero-shot retrieval performance (macro-averaged F1 under top-5 majority voting, MV@5) of three WILSON variants (WILSON-distill, WILSON-KW, WILSON-CLIP) against MOOZY, a case-level pathology foundation model, on four case-level benchmarks: CPTAC-BRCA, MayoBreastSubtype, MayoCaseBreast and MayoCaseSkin. Error bars indicate the standard deviation across the 10 composite images generated per case; MOOZY operates directly on the WSI and therefore has no associated variance. Color-matched horizontal dashed lines mark each model’s mean across the four benchmarks. c , The same values as in b shown as a model × benchmark heatmap (rows, models; columns, benchmarks; shading, macro-averaged F1). d , WSI-level zero-shot MV@5 retrieval performance of the same three WILSON variants against three WSI-level foundation models (GigaPath, TITAN, PRISM) on four WSI-level benchmarks: TCGA-BRCA, BCTherapy, MayoBreast and MayoSkin. Error bars indicate the standard deviation across the 10 composite images generated per WSI; the three baselines operate on frozen tile-level features and therefore have no associated variance. Color-matched dashed lines mark each model’s mean across the four benchmarks. e , The same values as in d shown as a model × benchmark heatmap. f , The same six models and four WSI-level benchmarks shown as a radar plot (BCTherapy, top; TCGA-BRCA, right; MayoSkin, bottom; MayoBreast, left; radial axis, macro-averaged F1 from 0.0 to 0.8 ), with colors as in d . g , Effect of end-to-end fine-tuning of WILSON on MayoTNBC, a cohort of 508 triple-negative breast cancer cases, evaluated on histological subtyping (Histology) and stromal tumor-infiltrating lymphocyte (sTILs) grading. Accuracy (left) and macro-averaged F1 (right) are shown for three settings: the pretrained backbone evaluated zero-shot (Base Zero-shot), the same backbone after end-to-end fine-tuning and evaluated zero-shot with no task-specific head (FT-Zero-shot), and the fine-tuned backbone trained jointly with a task-specific classification head (FT-classification).
Figure 3: WILSON vs. PRISM across all six cohorts—MayoCaption, MayoDuodenum, MayoPancreas, MayoStomach, TCGA-Brain, TCGA-Kidney—at K∈{1,3,5,10} . (a) I2T recall as radar plots (one axis per cohort, R@1–R@10 left to right); (b) the same I2T values as grouped bars, with dashed lines marking each model’s across-cohort mean (WILSON ≈ 74.1%, PRISM ≈ 58.1% at R@1). (c,d) mirror (a,b) for T2I. The WILSON envelope contains the PRISM envelope almost everywhere and both narrow toward TCGA-Brain; the orange curve extends past blue only on the TCGA-Kidney axis (I2T, R@3–R@10) and the MayoPancreas axis (T2I, R@3/R@5), the two crossover cases detailed in Table 11 .
Figure 4: WILSON-CoCa generates pathology captions that more closely match reference reports than PRISM and PRISM2, on an internal held-out set and on two external cohorts. WILSON-CoCa is a CoCa-style Yu et al. (2022) generative extension of WILSON that decodes a free-text diagnostic caption directly from a composite image, compared against two general-purpose pathology vision–language foundation models capable of report generation, PRISM Shaikovski et al. (2024) and PRISM2 Shaikovski and others (2025) . a , Evaluation pipeline: a WSI is rendered as a multi-magnification composite and encoded by the model’s vision tower; the text decoder generates a caption autoregressively, which is scored against the report-derived reference using n -gram overlap (BLEU-4, CIDEr, ROUGE-1, ROUGE-L), embedding-based similarity to the reference (BERTScore precision/recall/F1) and CLIPScore. CLIPScore is computed between the slide’s image embedding and the generated caption’s text embedding in each model’s own image–text contrastive space, and therefore measures whether the caption is consistent with what the model encodes from the slide rather than whether it resembles the reference; because each contrastive space has its own scale, CLIPScore is interpretable within a model but not across models. It is not reported for PRISM2, which exposes no general-purpose text encoder for arbitrary strings; BERTScore provides an encoder-agnostic reference-similarity measure for all three models. b , BLEU-4, ROUGE-L and CLIPScore on three cohorts (rows): 1,000 image–caption pairs from a held-out subset of the Mayo Clinic cohort spanning multiple organs and diagnoses; HistAI (48 pairs); and TCGA (50), neither external cohort seen during training. Bars show the mean over pairs, error bars one standard deviation; y -axis ranges differ between panels, so heights are comparable within but not across panels. WILSON-CoCa leads both baselines on the overlap metrics, by roughly 60% in relative terms on the internal cohort and by smaller margins on the external cohorts, and achieves the highest BERTScore recall while BERTScore precision and F1 remain comparable across models. Per-metric values are given in Table 13 and metric definitions and interpretation caveats in Supplementary Note 9 . c , Qualitative examples (placenta, gallbladder) showing the WSI, its clinical report, the composite input and the captions generated by each model; further cases in Fig. 14
Figure 5: Data curation strategy for WILSON. a, Mayo Clinic pathology reports were segmented using an LLM. b, To maximize diagnostic diversity, the number of whole-slide images (WSIs) selected from each disease category was capped, and representative specimens were chosen from WSIs identified as key slides. c, Natural institutional distribution of WSIs before sampling. Colors indicate distinct disease groups, illustrating the strong imbalance typically observed in routine clinical archives. d, Distribution of the WILSON training dataset after the curation procedures described in panels a–d . Diversity-aware sampling substantially reduced organ- and disease-level imbalance while preserving broad diagnostic coverage. e, Overview of the WILSON-DS training dataset. f, Number of patients per diagnostic category (log scale).
Figure 6: COMPOSITE: Clustering-Oriented Composite Assembly Pipeline. a, Phase 1 builds a normal-tissue reference. Tissue regions in healthy whole-slide images (WSIs) are detected by Hue–Saturation–Density thresholding Bejnordi et al. (2016) and randomly tiled; a DINO pre-trained Vision Transformer (ViT) extracts 384-dimensional CLS-token embeddings in a single forward pass. Pooled embeddings are clustered with MiniBatch k -means ( k=200 ), and each cluster mean is replaced by its medoid so that centroids correspond to genuine tissue appearances. The 200 prototypes form a reusable organ-specific reference. b, Phase 2 scores tiles in disease slides. The same ViT emits both the CLS embedding and quality logits (valid/invalid) in one pass; sub-threshold tiles (marked x) are discarded as artifacts. Remaining tiles receive an abnormality score equal to their minimum Euclidean distance to the 200 normal centroids, higher values indicating greater deviation from normal tissue. c, Phase 3 selects tiles with spatial diversity. Naive top- k selection (red x) concentrates tiles in a single lesion, sacrificing representativeness. Instead, the tissue bounding box is divided into a spatial grid, the highest-scoring tile per cell is selected, and globally best unused tiles fill empty cells—guaranteeing coverage of all tissue zones while biasing toward diagnostically informative locations, yielding 60 positions per slide. Ranked locations are distributed round-robin across the 10 composites in order of abnormality score, so each composite spans the full abnormality spectrum and all spatial zones rather than concentrating the most atypical locations in one image. Locations per composite is configurable: the schematic shows a six-location layout ( 6×6 grid), whereas all experiments used eight locations ( 8×8 grid; panel d). Within each composite, rows are tissue locations ordered by abnormality score and columns encode multi-scale detail, from a 1.25 × context view through 5 × and 20 × magnifications. d, Example 8×8 composite. Eight low-power regions (A–H; 256×256 px, 8μ m/pixel) were drawn from four WSIs in the CaseLuminal dataset, each contributing three mid-power (green boxes; 256×256 px, 2μ m/pixel) and four high-power regions (blue boxes; 256×256 px, 0.5μ m/pixel), assembled into a single 2048×2048 px composite.
Appendix figures & tables39 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Case-level image retrieval performance of the vision encoders. Retrieval uses each model’s vision encoder only (no text encoder at inference). All whole-slide images of a case were represented in a single case-level composite before getting its corresponding embedding, and each query case was matched against the remaining cases by cosine similarity; the label of the retrieved neighbour(s) was transferred to the query. Columns give the label-transfer rule (Top-1, or majority vote over the k=3,5,7 nearest neighbours) and rows the resulting metric (accuracy, balanced accuracy, macro-averaged F1-score). Axes are the four evaluation cohorts; the radial scale spans 0 – 1 with gridlines every 0.2 . All three WILSON variants — which differ only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive) — exceed MOOZY on every cohort, metric and k , with the largest margins on balanced accuracy and macro-averaged F1-score. Chance level differs per axis because class counts and prevalences differ between cohorts.
Figure 8: Case-level image retrieval accuracy by cohort, macro-averaged F1-score. Per-cohort breakdown of the case-level retrieval results summarized in Fig. 7 , using the vision encoder of each model only. Panels correspond to the label-transfer rule (majority vote over the k=1,3,5,7 nearest neighbours); groups within each panel are the four evaluation cohorts (CPTAC-BRCA, Mayo Breast Subtyping, Mayo Breast Diagnosis, Mayo Skin). Bars show macro-averaged F1-score for MOOZY and three WILSON variants differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive); error bars denote [s.d. across n folds / seeds] and horizontal dashed lines mark each model’s mean across the four cohorts. Note the truncated vertical axis. WILSON-distill achieves the highest cohort mean at every k and the largest single-cohort margin on Mayo Skin (0.62 vs. 0.28 for MOOZY at k=1 ; 0.66 vs. 0.27 at k=7 ). Gains are smaller on the breast cohorts, where MOOZY is comparable to WILSON-CLIP on Mayo Breast Subtyping and exceeds it on CPTAC-BRCA at every k , indicating that the advantage is cohort-dependent rather than uniform.
Figure 9: Complete case-level image retrieval results. Numerical values underlying Figs. 7 and 8 , using the vision encoder of each model only. Panels report accuracy (left), balanced accuracy (centre) and macro-averaged F1-score (right) on a common color scale; columns are the four evaluation cohorts and rows every combination of model (MOOZY, WILSON-distill, WILSON-KW, WILSON-CLIP) and label-transfer rule (majority vote over the k=1,3,5,7 nearest neighbours), grouped by k . Three patterns are visible. First, the WILSON encoders’ advantage is largest on Mayo Skin, where all three exceed MOOZY by roughly 0.3 on every metric and at every k (e.g. 0.67 vs. 0.29 accuracy at k=7 ). Second, on CPTAC-BRCA the accuracy of all models (0.42–0.58) substantially exceeds their balanced accuracy and macro-averaged F1-score (0.29–0.40), indicating that raw accuracy there is inflated by class imbalance; MOOZY shows the widest gap (0.57 vs. 0.35 at k=3 ), consistent with a bias toward the majority class. Third, performance generally improves with increasing k for all models, with most of the gain realised by k=5 .
Figure 10: Slide-level image retrieval performance of the vision encoders. Retrieval uses each model’s vision encoder only (no text encoder at inference). Each whole-slide image was encoded into a single slide embedding and matched against the remaining slides by cosine similarity; the label of the retrieved neighbour(s) was transferred to the query. Columns give the label-transfer rule (Top-1, or majority vote over the k=3,5,7 nearest neighbours) and rows the resulting metric (accuracy, balanced accuracy, macro-averaged F1-score). Axes are the four evaluation cohorts (BCTherapy, TCGA-BRCA, Mayo Skin, Mayo Breast Diagnosis); the radial scale spans 0 – 1 with gridlines every 0.2 . Six slide encoders are compared: GigaPath, TITAN, PRISM, and three WILSON variants differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive). The WILSON variants match or exceed the published foundation models across cohorts, metrics and values of k , with the clearest separation on Mayo Skin and Mayo Breast Diagnosis; GigaPath trails on every axis. Chance level differs per axis because class counts and prevalences differ between cohorts.
Figure 11: Slide-level image retrieval accuracy by cohort, macro-averaged F1-score. Per-cohort breakdown of the slide-level retrieval results summarized in Fig. 10 , using the vision encoder of each model only. Panels correspond to the label-transfer rule (majority vote over the k=1,3,5,7 nearest neighbours); groups within each panel are the four evaluation cohorts (TCGA-BRCA, BCTherapy, Mayo Breast Diagnosis, Mayo Skin). Within each group, bars are ordered GigaPath, TITAN, PRISM, WILSON-distill, WILSON-KW, WILSON-CLIP, the last three differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive); error bars denote [s.d. across n folds / seeds] and horizontal dashed lines mark each model’s mean across the four cohorts. Note the truncated vertical axis. The WILSON encoders lead on Mayo Skin at every k (up to 0.69 for WILSON-distill at k=7 vs. 0.63 for PRISM and 0.57 for TITAN) and are comparable to TITAN and PRISM on BCTherapy and Mayo Breast Diagnosis, while TITAN and PRISM retain a small advantage on TCGA-BRCA at k=1 . GigaPath performs worst in every cohort and at every k , most markedly on Mayo Breast Diagnosis and Mayo Skin.
Figure 12: Complete slide-level image retrieval results. Numerical values underlying Figs. 10 and 11 , using the vision encoder of each model only. Panels report accuracy (left), balanced accuracy (centre) and macro-averaged F1-score (right); note that each panel has its own color scale, so shading is comparable within but not across panels. Columns are the four evaluation cohorts and rows every combination of model (GigaPath, TITAN, PRISM, WILSON-distill, WILSON-KW, WILSON-CLIP) and label-transfer rule (majority vote over the k=1,3,5,7 nearest neighbours), grouped by k . Three patterns are visible. First, no single encoder leads everywhere: WILSON-distill is strongest on Mayo Skin at every k (0.69 accuracy and 0.69 macro F1 at k=7 , vs. 0.64 and 0.63 for PRISM), WILSON-CLIP on BCTherapy (0.77 accuracy at k=5 ), PRISM on Mayo Breast Diagnosis (0.72 accuracy at k=7 ) and TITAN on TCGA-BRCA (0.88 accuracy at k=7 ). Second, the gap between accuracy and balanced accuracy is pronounced on TCGA-BRCA for all six encoders (e.g. 0.88 vs. 0.43 for TITAN at k=7 ), indicating that accuracy in that cohort is dominated by class imbalance and that the balanced-accuracy and macro-F1 panels are the informative ones. Third, GigaPath is the weakest encoder throughout, and its deficit is largest on Mayo Breast Diagnosis and Mayo Skin (balanced accuracy 0.25–0.34) while remaining competitive on TCGA-BRCA accuracy (0.73–0.79), a pattern consistent with majority-class prediction rather than discriminative retrieval.
Figure 13: Comprehensive Performance Comparison of WILSON and PRISM Models on Cross-Modal Retrieval Tasks. The figure presents recall metrics (R@1, R@3, R@5, R@10) evaluated across Mayo Clinic (Diagnoses, Duodenum, Pancreas, Stomach) and TCGA (Brain, Kidney) datasets. Top section (Bar charts): Quantitative comparison showing exact performance values for each metric and dataset. Bottom section (Radar charts): Multi-dimensional performance visualization enabling visual assessment of relative strengths across all metrics and datasets simultaneously. Each row represents a cross-modal retrieval direction: Image-to-Text (I2T) retrieves relevant diagnostic text reports given medical images; Text-to-Image (T2I) retrieves relevant medical images given text descriptions. Blue elements (bars and lines) represent WILSON model performance, purple elements represent PRISM model performance. All metrics are reported as percentages, with bar chart values displayed above each bar for precise quantitative reference.
Figure 14: Additional qualitative comparison of generated slide-level reports. Seven further cases (colon, uterus, urinary bladder, liver, stomach, prostate, breast), showing the whole-slide image and its clinical report, the composite input, and the descriptions generated by WILSON, PRISM and PRISM2. Conventions as in Fig. 4 .
Figure 15: Caption-quality evaluation across cohorts, models, and metrics. Grouped bar charts of eight caption-quality metrics (columns) for PRISM, PRISM2 and WILSON (ours) on three cohorts (rows): held-out Mayo Clinic, HistAI and TCGA. Bars show the mean over caption pairs and error bars one standard deviation; annotations give the mean value. Metrics are BLEU-4 ( n -gram precision), CIDEr (consensus-based TF–IDF-weighted n -gram similarity), ROUGE-1 (unigram overlap), ROUGE-L (longest common subsequence), BERTScore precision/recall/F1 (contextual similarity from BERT embeddings) and CLIPScore (image–text similarity in a contrastive embedding space). CLIPScore is computed within each model’s own embedding space and is therefore not comparable across models; PRISM2 exposes no general-purpose text encoder for arbitrary strings, so no PRISM2 bar is shown. The rightmost panel (“Avg”) summarizes each model as the mean of its metric values after min–max normalization within metric and cohort, rescaled to 0–100 [excluding CLIPScore]. Note that y -axis ranges differ between panels: bar heights are comparable within a panel but not across panels.
Dataset
Task
Yottixel + UNI
Composite + UNI
BCTherapy
Treatment response
0.51
0.56
MayoSkin
Diagnostic classification
0.33
0.34
MayoBreast
Diagnostic classification
0.31
0.35
TCGA-Kidney
Diagnostic classification
0.89
0.88
Appendix
Table 1: Composite images vs. Yottixel patches as WSI representations, both encoded with the pretrained UNI foundation model. Values are top-1 macro-F1 retrieval performance (higher is better) across two public and two internal Mayo Clinic cohorts.
Base zero-shot
Fine-tuned zero-shot
Fine-tuned classification
Task
acc
Macro avg
acc
Macro avg
acc
Macro avg
Histology
0.617
0.401
0.637
0.428
0.690
0.557
sTILs
0.412
0.425
0.432
0.456
0.541
0.531
Appendix
Table 2: End-to-end fine-tuning of WILSON (dense embedding) on MayoTNBC dataset.Accuracy (acc) and macro-averaged F1 (Macro avg) are reported for histologic subtyping and sTILs grading under three settings: the pretrained backbone evaluated zero-shot, the same backbone after end-to-end fine-tuning evaluated zero-shot, and the fine-tuned backbone with a jointly trained classification head.
Direction
Model
R@1 (%)
R@3 (%)
R@5 (%)
R@10 (%)
Image-to-Text
WILSON
78.29
85.75
88.82
92.11
PRISM
60.09
66.45
69.74
75.66
Δ (W − P)
+18.20
+19.30
+19.08
+16.45
Text-to-Image
WILSON
81.14
94.08
96.49
98.03
PRISM
56.36
77.85
82.89
87.72
Δ (W − P)
+24.78
+16.23
+13.60
+10.31
Appendix
Table 3: Cross-modal retrieval on the MayoCaption dataset, horizontal evaluation across all diagnoses combined. Δ denotes WILSON minus PRISM.
Direction
Model
R@1 (%)
R@3 (%)
R@5 (%)
R@10 (%)
Image-to-Text
WILSON
77.78
83.33
86.36
89.90
PRISM
64.14
65.15
67.17
68.69
Δ (W − P)
+13.64
+18.18
+19.19
+21.21
Text-to-Image
WILSON
80.81
94.95
97.98
98.48
PRISM
55.56
67.68
74.24
78.79
Δ (W − P)
+25.25
+27.27
+23.74
+19.70
Appendix
Table 4: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Duodenum. Δ denotes WILSON minus PRISM.
Direction
Model
R@1 (%)
R@3 (%)
R@5 (%)
R@10 (%)
Image-to-Text
WILSON
78.18
89.09
90.91
93.64
PRISM
59.09
63.64
69.09
80.00
Δ (W − P)
+19.09
+25.45
+21.82
+13.64
Text-to-Image
WILSON
83.64
92.73
95.45
99.09
PRISM
70.91
95.45
98.18
98.18
Δ (W − P)
+12.73
-2.73
-2.73
+0.91
Appendix
Table 5: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Pancreas. Δ denotes WILSON minus PRISM.
Direction
Model
R@1 (%)
R@3 (%)
R@5 (%)
R@10 (%)
Image-to-Text
WILSON
85.81
91.89
93.92
96.62
PRISM
81.76
87.84
90.54
95.95
Δ (W − P)
+4.05
+4.05
+3.38
+0.68
Text-to-Image
WILSON
85.14
95.95
96.62
97.97
PRISM
65.54
92.57
95.27
96.62
Δ (W − P)
+19.59
+3.38
+1.35
+1.35
Appendix
Table 6: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Stomach. Δ denotes WILSON minus PRISM.
Direction
Model
R@1 (%)
R@3 (%)
R@5 (%)
R@10 (%)
Image-to-Text
WILSON
41.94
58.87
65.32
78.23
PRISM
20.16
40.32
55.65
81.45
Δ (W − P)
+21.77
+18.55
+9.68
-3.23
Text-to-Image
WILSON
54.03
70.97
82.26
88.71
PRISM
41.13
62.10
71.77
87.10
Δ (W − P)
+12.90
+8.87
+10.48
+1.61
Appendix
Table 7: Cross-modal retrieval on the TCGA Brain dataset. Δ denotes WILSON minus PRISM.
Direction
Model
R@1 (%)
R@3 (%)
R@5 (%)
R@10 (%)
Image-to-Text
WILSON
82.56
86.05
93.02
96.51
PRISM
63.22
93.10
96.55
100.00
Δ (W − P)
+19.34
-7.06
-3.53
-3.49
Text-to-Image
WILSON
77.91
90.70
94.19
100.00
PRISM
54.02
59.77
67.82
78.16
Δ (W − P)
+23.88
+30.93
+26.37
+21.84
Appendix
Table 8: Cross-modal retrieval on the TCGA Kidney dataset. Δ denotes WILSON minus PRISM.
Image-to-Text R@1 (%)
Text-to-Image R@1 (%)
Dataset
WILSON
PRISM
WILSON
PRISM
MayoCaption (all)
78.29
60.09
8 1.14
56.36
Mayo – Duodenum
77.78
64.14
80.81
55.56
Mayo – Pancreas
78.18
59.09
83.64
70.91
Mayo – Stomach
85.81
81.76
85.14
65.54
TCGA – Brain
41.94
20.16
54.03
41.13
Appendix
Table 9: Summary of R@1 retrieval performance for WILSON and PRISM across all evaluated datasets, for both image-to-text (I2T) and text-to-image (T2I) retrieval directions.
WILSON (%)
PRISM (%)
Dataset
Direction
R@1
R@3
R@5
R@10
R@1
R@3
R@5
R@10
MayoCaption
I2T
78.29
85.75
88.82
92.11
60.09
66.45
69.74
75.66
T2I
81.14
94.08
96.49
98.03
56.36
77.85
82.89
87.72
Mayo – Duodenum
I2T
77.78
83.33
86.36
89.90
64.14
65.15
67.17
68.69
T2I
80.81
94.95
97.98
98.48
55.56
67.68
74.24
78.79
Mayo – Pancreas
I2T
78.18
89.09
90.91
93.64
59.09
63.64
69.09
80.00
Appendix
Table 10: Comprehensive comparison of WILSON and PRISM cross-modal retrieval recall (R@1, R@3, R@5, R@10) across all used Mayo Clinic and TCGA dataset/organ splits, for both image-to-text (I2T) and text-to-image (T2I) retrieval.
R@1
R@10
Dataset / Direction
PRISM
WILSON
Δ
PRISM
WILSON
Δ
MayoCaption I2T
60.1
78.3
+18.2
75.7
92.1
+16.4
MayoCaption T2I
56.4
81.1
+24.7
87.7
98.0
+10.3
Mayo Duodenum I2T
64.1
77.8
+13.7
68.7
89.9
+21.2
Mayo Duodenum T2I
55.6
80.8
+25.2
78.8
98.5
+19.7
Mayo Pancreas I2T
59.1
78.2
+19.1
80.0
93.6
+13.6
Appendix
Table 11: WILSON vs. PRISM at R@1 and R@10, all cohorts and directions. Δ = WILSON − PRISM; bold negative values indicate PRISM leads.
Model
Architecture
Total params
FLOPs per tile
FLOPs per WSI
Prov-GigaPath
ViT-G/14 (DINOv2) + LongNet
1,220M
555.4G
5.56P
PRISM2
Virchow2 (ViT-H/14) + PerceiverAR
1,251M
413.2G
4.13P
PRISM
Virchow (ViT-H/14) + Perceiver
730.3M
406.7G
4.07P
TITAN
CONCH v1.5 (ViT-L/16) + slide encoder
354.6M
70.0G
703.19T
MOOZY
ViT-S/8 (Lunit DINO) + ViT 6L, 768-d
85.8M
—
25.07T
WILSON
ConvNeXt-Base + extended head (single pass)
133.7M
—
2.58T
Appendix
Table 12: Parameter counts and computational cost of pathology foundation models at a common workload of 10,000 tiles per whole-slide image (WSI). For tile-encoder plus slide-aggregator pipelines, cost per WSI is the tile encoder evaluated N=10,000 times plus one pass of the slide aggregator. Counts cover the embedding path only; text decoders (PRISM’s BioGPT, PRISM2’s Phi-3 Mini, TITAN’s text tower) are excluded. A dash indicates that no per-tile cost is defined: WILSON encodes one 2048×2048 composite of 8×8=64 selected tissue regions in a single forward pass, and the MOOZY figure is reported for its frozen patch encoder together with its slide encoder. Prov-GigaPath’s slide encoder uses LongNet dilated attention, so its actual slide-encoder cost is lower than the dense-attention figure shown. Rows are ordered by cost per WSI.
Mayo Clinic
HistAI
TCGA
Metric
PRISM
PRISM2
WILSON
PRISM
PRISM2
WILSON
PRISM
PRISM2
WILSON
BLEU-4
0.0004
0.0034
0.0155
0.0000
0.0004
0.0017
0.0019
0.0023
0.0085
CIDEr
0.0022
0.0550
0.2236
0.0944
0.0199
0.0236
0.0204
0.0272
0.0903
ROUGE-1
0.1424
0.2307
0.3711
0.1196
0.0948
0.1617
0.1825
0.2148
0.2136
ROUGE-L
0.1189
0.1670
0.2556
0.1023
0.0801
0.1262
0.1497
0.1596
0.1592
BERTScore-P
0.8812
0.8946
0.8627
0.8519
0.8290
0.8383
0.8808
0.8740
0.8609
Appendix
Table 13: Caption-quality evaluation across three models and three cohorts. BLEU-4 ( n -gram overlap), CIDEr (consensus-based), ROUGE-1 (unigram overlap), ROUGE-L (longest common subsequence), BERTScore-P/R/F1 (contextual similarity from BERT embeddings) and CLIPScore (CLIP-based semantic similarity) are reported as raw scores. “Avg” is the mean performance across metrics after min–max normalization within each metric, rescaled to 0–100. Best model per cohort is bold. a CLIPScore is computed in each model’s own contrastive embedding space and is therefore not comparable across models; PRISM2 exposes no general-purpose text encoder for arbitrary strings. Excluded from bolding and from Avg.
Dataset
Organ
Task
Classes
Patients
Slides
Format
Mayo189K
Various
WILSON Training
N/A
112,296
189,291
DICOM
MayoBreast
Breast (tumor, nontumor)
Histological subtype classification
12
1,289
1,339
DICOM
MayoSkin
Skin (tumor, nontumor)
Histological subtype classification
20
677
688
DICOM
MayoTNBC
Breast carcinoma (triple negative)
Fine tuning
4/4
508
508
Philips Tiff
MayoCaption
Three organs
Cross-modal retrieval
18
397
456
DICOM
MayoDuodenum
Duodenum
Cross-modal retrieval
7
172
198
DICOM
Appendix
Table 14: Summary of training and validation WSI datasets. The MayoTNBC dataset contains two tasks, histological subtype classification and stromal tumor-infiltrating lymphocyte (sTIL) prediction, each comprising four classes. The MayoCaption dataset was constructed by combining the MayoDuodenum, MayoStomach, and MayoPancreas datasets for horizontal cross-modal retrieval. Histological subtype annotations for the TCGA-derived datasets were assigned according to Uegami et al. Uegami et al. (2026) .
Dataset
Organ
Task
Classes
Cases
WSIs
Format
MayoCaseBreast
Breast (tumor, nontumor)
Histological subtype classification
12
494
4,361
DICOM
MayoCaseSkin
Skin (tumor, nontumor)
Histological subtype classification
20
500
1,280
DICOM
MayoBreastSubtype
Breast carcinoma
Molecular subtype classification
3
846
2,254
DICOM
CPTAC-BRCA National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020)
Breast carcinoma
PAM50 subtype classification
5
198
653
SVS
Appendix
Table 15: Summary of case-level validation WSI datasets.
Class
Count
Invasive carcinoma of no special type
812
Invasive lobular carcinoma
204
Ductal and lobular carcinoma
27
Mucinous adenocarcinoma
20
Metaplastic carcinoma
13
Medullary carcinoma
7
Appendix
Table 16: Class distribution of the TCGA-BRCA dataset. Cancer Genome Atlas Research Network et al. (2013)
Class
Count
RD
119
pCR
41
Appendix
Table 17: Class distribution of the BCTherapy dataset Sammut et al. (2022) .
Class
Count
Invasive breast carcinoma of no special type (ductal)
350
Invasive lobular carcinoma
70
Mixed ductal and lobular carcinoma
70
Mucinous carcinoma
70
Metaplastic carcinoma
70
Ductal carcinoma in situ
210
Appendix
Table 18: Class distribution of the MayoBRCA dataset.
Class
Count
Lentigo
41
Actinic keratosis
40
Malignant melanoma
40
Blue nevus
39
Dermatofibroma
38
Verrucae
38
Appendix
Table 19: Class distribution of the MayoSkin dataset.
Class
Count
Hormone receptor-positive/HER2-negative
269
HER2-positive
303
Triple-negative
274
Appendix
Table 20: Class distribution of the MayoBreastSubtype dataset.
Class
Count
Luminal A
57
Basal-like
27
Luminal B
17
HER2-enriched
14
Normal-like
5
Appendix
Table 21: Class distribution of the CPTAC-BRCA dataset National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020) .
Class
Count
Invasive carcinoma of no special type (ductal)
44
Invasive lobular carcinoma
42
Mixed ductal and lobular carcinoma
40
Mucinous carcinoma
41
Metaplastic carcinoma
38
Ductal carcinoma in situ
40
Appendix
Table 22: Class distribution of the MayoCaseBreast dataset.
Class
Count
Sebaceous gland hyperplasia
31
Dermatofibroma
31
Malignant melanoma
31
Skin tag
29
Nevus with atypia
28
Seborrheic keratosis
27
Appendix
Table 23: Class distribution of the MayoCaseSkin dataset.
WSI ID
Organ
case_00760/slide_H&E_0.tiff
Bone
case_03070/slide_H&E_0.tiff
Bone
case_09511/slide_H&E_0.tiff
Bone
case_11368/slide_H&E_0.tiff
Bone
case_11593/slide_H&E_0.tiff
Bone
case_01871/slide_H&E_0.tiff
Breast
Appendix
Table 24: HistAI Nechaev et al. (2025) whole-slide images (WSIs) and corresponding organ categories.
Table 25: TCGA Weinstein et al. (2013) whole-slide images (WSIs) and corresponding organ categories.
Stage
Paired WSIs
Composites
Val. frac.
Epochs
GPUs
Eff. batch
WILSON-distill
178,020
1,759,787
0.02
20
32 (4 × 8)
2,048
WILSON-KW (keyword regr.)
178,088
1,760,461
0.10
7
8 (1 × 8)
512
WILSON-CLIP, Phase 2
1,760,461 *
1,760,461 *
0.199 *
20
32 (4 × 8)
576
WILSON-CLIP, Phase 3
1,760,461 *
1,760,461 *
0.199 *
15
32 (4 × 8)
768
WILSON-CoCa captioning
178,020
1,759,787
0.02
20
32 (4 × 8)
1,024
Appendix
Table 26: Training corpus and compute per vision–language stage. * WILSON-CLIP uses a fixed 1,410,600/349,861 train/val composite manifest (§ 4 ); all other rows are approximate WSI-level splits computed from each script’s val_frac argument applied to the paired corpus above, since exact per-sample assignment is determined at run time by the data loader. “Eff. batch” is GPUs × per-GPU batch size × gradient-accumulation steps.
Figure 16: Example of composite image (single WSI). MayoBreast dataset.
Figure 17: WILSON vision encoder ( ConvNeXtV3Extended ). a , Stage-by-stage architecture for the 2048×2048 reference input (an 8×8 grid of 256-px tiles). Tensor glyphs encode each feature map (face ∝log2 spatial size, depth ∝log2 channels), making the progression from a broad, shallow 128×512×512 map to a narrow, deep 3072×8×8 map visible at a glance. A DINOv3-pretrained ConvNeXt-Base backbone (depths 3,3,27,3 ; widths 128,256,512,1024 ; stride 32; 87.57M params) is followed by three stride-2 downsampling blocks (extra_blocks ×3 ; 46.15M params) that extend the width to 3072 (stride 256) and global average pooling, for 133.7M parameters. Column-aligned bars show where parameters and compute (conv/linear MACs) reside: Stage 3 dominates both (57.4M params, 939 GMACs), whereas the extended head holds 34.5% of the parameters but only 0.6% of the compute. b , Anatomy of the two block types with symbolic parameter counts; the first extended block omits its LayerNorm (its input is already normalized by the backbone), and a 1×1 convolutional fallback replaces the stride-2 convolution only when a block’s input is already 1×1 (e.g. 224-px inputs). c , The three operating resolutions. At 224–256 px (tile-based SSL) only the backbone is used and its 1024-d GAP output is taken directly (20 GMACs); at 512×512 (composite-based SSL pretraining, § 4 ) the final map is 3072×2×2 (81 GMACs); at 2048×2048 (composite-based vision–language training, § 4 ) the final 3072×8×8 map provides 64 spatial tokens to the CoCa attentional pooler (§ 4 ) (1,291 GMACs). d , Warm-start of the extra-block convolutions when the backbone is loaded from a domain-specific SSL checkpoint: the 1024×512×2×2 filters of the last backbone downsampling layer are tiled to each extra-block shape and rescaled to He-initialization magnitude ( std=2/fan_in ), preserving filter directions while keeping activations within fp16 range; biases are tiled likewise and the extra-block LayerNorms start at (1,0) .
Table 27: WILSON vision encoder ( ConvNeXtV3Extended ) — parameter and compute breakdown by component. Output shapes, cumulative stride and MACs are given for the 2048×2048 reference input (an 8×8 grid of 256-px tiles), the default resolution for composite-based vision–language training. Each “Block” is a standard ConvNeXt residual block (Fig. 17 b). Parameter counts were verified by instantiating the model. The extended head holds 34.5% of the parameters but accounts for only 0.6% of the compute, because it operates on ≤32×32 maps; conversely Stage 3 alone accounts for 72.7% of the MACs. † Blocks 2–3 also instantiate a 1×1 convolutional fallback (2,099,200 and 6,294,528 params, included in the totals) that is used only when the block’s input is already 1×1 ; it is dormant for all inputs ≥256 px, leaving 125,327,488 active parameters ( ≈ 125.3M). ‡ Multiply–accumulates of convolution and linear layers only (LayerNorm, GELU, layer scale and pooling omitted); 1 MAC = 2 FLOPs. At 512×512 the total is 80.7 GMACs and at 256×256 (backbone only) 20.1 GMACs.
Hyperparameter
Value
Input resolution
2048×2048 px
Vision spatial tokens
8×8=64 , 3072-dim
Vision-encoder trainable params
133.72M
Attentional-pooler queries / heads
256 / 8
Joint embedding dim
768
Decoder width
768
Appendix
Table 28: WILSON + CoCa vision-language training hyperparameters.
Computational pathology needs whole-slide image (WSI) foundation models that transfer across diverse clinical tasks, yet current approaches remain largely slide-centric, often depend on private data and expensive paired-report supervision, and do not explicitly model relationships among multiple slides from the same patient. We present MOOZY, a patient-first pathology foundation model in which the patient case, not the individual slide, is the core unit of representation. MOOZY explicitly models dependencies across all slides from the same patient via a case transformer during pretraining, combining multi-stage self-supervision with scaled low-cost task supervision. In Stage 1, we pretrain a vision-only slide encoder on 77,134 public slide feature grids using masked self-distillation. In Stage 2, we align these representations with clinical semantics using a case transformer and multi-task supervision over 333 tasks from 56 public datasets, including 205 classification and 128 survival tasks across four endpoints. Across sixteen held-out tasks, MOOZY improves macro weighted F1, balanced accuracy, and macro weighted ROC-AUC relative to PRISM by +4.19%, +7.93%, and +6.95%, respectively. MOOZY is also parameter efficient with 85.77M parameters, 14× smaller than GigaPath. These results suggest that patient-level pretraining yields transferable embeddings, providing a path toward scalable patient-first histopathology foundation models.
Yousef Kotp, Vincent Quoc-Huy Trinh, Christopher Pal +1
CSSE, Concordia University, Montréal, Canada · Mila – Québec AI Institute, Montréal, Canada · CHUM, Université de Montréal, Montréal, Canada +4
Foundation models are reshaping computational pathology, yet their capabilities remain shaped by pretraining objectives, data sources, and spatial scales, fragmenting complementary expertise across separate backbones. Here we present ALICE, a unified foundation model trained through multi-stage agglomerative distillation that sequentially distills eight vision-only, vision-language, and slide-level teacher models into dedicated modules of a single backbone. ALICE is pretrained on 24,985,184 tile-level pathology images and 155,604 high-resolution images, and evaluated across 21 task scenarios, 96 downstream tasks, and 48 data sources, spanning region-of-interest tissue analysis, vision-language multimodal evaluation, and whole-slide clinical assessment. In all three evaluation settings, ALICE achieved the best average rank among task-matched pathology foundation models. These results demonstrate that agglomerative distillation can consolidate complementary capabilities from specialized models into a unified backbone for broad computational pathology applications. The model is available at https://github.com/WonderLandxD/ALICE.
Jiawen Li, Tian Guan, Huijuan Shi +5
Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China · Department of Engineering Science, University of Oxford, Oxford, UK +1
Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating multi-scale evidence. However, most existing pathology benchmarks evaluate models on pre-cropped patches or pre-extracted slide features, leaving their ability to acquire evidence directly from gigapixel WSIs largely untested. We introduce PathAgentBench, a benchmark for evaluating evidence-seeking vision-language models (VLMs) across four complementary capabilities: image-to-text matching for evidence interpretation, text-to-image retrieval for evidence verification, diagnostic-region localization for evidence acquisition, and multi-scale reasoning for evidence integration. The benchmark is organized as a diagnostic tree that links nested regions across magnifications with scale-specific findings and path-level diagnoses. It contains 1,822 TCGA WSIs and 17,135 diagnostic paths annotated by ten board-certified pathologists. An additional private cohort of 190 breast cancer WSIs with detailed annotations is used to evaluate autonomous whole-slide exploration. We evaluate 20 general-purpose, medical, and pathology-specialized models. Leading open-weight models achieve over 93% accuracy in multi-scale reasoning and over 50% accuracy in both cross-modal matching tasks. In contrast, diagnostic-region localization remains challenging: the best text-guided mean intersection-over-union is below 0.09, underperforming a simple center-based heuristic. During autonomous exploration, the unconditional hit rate decreases from 0.522 at low magnification to 0.185 at intermediate magnification and 0.020 at high magnification. These results reveal a pronounced gap between reasoning over curated evidence and acquiring that evidence directly from WSIs. PathAgentBench provides a unified framework for measuring and improving evidence-seeking pathology models.
Dankai Liao, Tianyi Zhang, Yufeng Wu +6
Department of Biomedical Engineering, National University of Singapore, Singapore 117417, Singapore