q-bio.QMSep 20, 2026

WILSON - a pathology foundation model framework for patient-level analysis and diagnostic text generation

Authors: Saghir Alfasly, Wataru Uegami, Sobhan Hemati, Wenchao Han, Xiaojia Tang, Kevin Thompson, Daniel Stone, Ghazal Alabtah, +8 more

Organizations: KIMIA Lab, Department of AI & Informatics, Mayo Clinic, Rochester, MN, USA. · Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA. · Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, USA. · Mayo Clinic Comprehensive Cancer Center, Mayo Clinic, Rochester, MN, USA. · Division of Breast and Melanoma Surgical Oncology, Department of Surgery, Mayo Clinic, Rochester, MN, USA. · Department of Oncology, Mayo Clinic, Rochester, MN, USA.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Pathologists integrate morphology across magnifications and across the slides of a patient case, whereas pathology foundation models encode thousands of tiles from single slides and aggregate their features. Here we present WILSON, a vision--language foundation model that represents whole-slide images and multi-slide cases as single multi-magnification composite images, trained on approximately 189k slides from Mayo Clinic spanning 42 organs and 829 diagnostic entities using pathology reports as supervision. Without task-specific training, WILSON exceeded a dedicated case-level model on all internal cohorts (macro-F1 0.52 versus 0.38) and matched slide-level models up to 9.4 times larger at 272- to 2,155-fold lower compute. End-to-end fine-tuning on 508 triple-negative breast cancer cases improved histologic subtyping and stromal tumor-infiltrating lymphocyte grading by 0.16 and 0.11 macro-F1. WILSON retrieved matching diagnostic text at 75.6% recall@1 (PRISM, 58.1%) and generated captions closer to report-derived references than PRISM and PRISM2 on the internal cohort and on most external comparisons. Composite images thus offer a compact, clinically aligned computational unit for pathology.

Figures & tables

Figure 1: Overview of WILSON. a , Slides and cases per organ (bars), with the cohort further grouped by organ and by diagnostic category (donuts). b , An LLM cleans each pathology report and generates three paraphrased caption variants per report. c , Composites are tiled from selected WSIs; 178,020 WSIs have both a rendered composite and a caption record, giving 1,759,787 composites ( ≈\approx 9.9/WSI) and 1,759,787×3=5,279,3611{,}759{,}787\times 3=5{,}279{,}361 ( ≈\approx 5.28M) image–caption pairs. d , WILSON pretraining and training: tile-based SSL on 256×256256\times 256 patches at 2.5×/10×/20×2.5\times/10\times/20\times →\rightarrow composite-based SSL on 512×512512\times 512 four-tile composites →\rightarrow vision–language alignment of 2048×20482048\times 2048 WSI composites against report text (SigLIP distillation from Gemini embeddings, CLIP, and keyword regression) →\rightarrow CoCa-style report generator, a multimodal decoder that cross-attends over pooled WILSON visual tokens to generate a caption autoregressively. e , Vision-encoder size, pretraining data (WSIs) and cost (FLOPs per 10k-tile WSI) for WILSON (133.7M; ≈\approx 189k; 2.6 T) versus MOOZY (85.8M; ≈\approx 77k; 25.1 T), TITAN (354.6M; ≈\approx 335k; 703.2 T), PRISM (730.3M; ≈\approx 587k; 4.07 P), PRISM2 (1,251M; ≈\approx 2.3M; 4.13 P) and Prov-GigaPath (1,220M; ≈\approx 171k; 5.56 P). f , Report-generation performance averaged over three cohorts (Mayo, HistAI, TCGA), as a normalized score: PRISM 27.1, PRISM2 34.9, WILSON 58.1, WILSON exceeds PRISM2 by 23.2 points with ≈\approx 9 ×\times fewer parameters and ≈\approx 1,600 ×\times fewer FLOPs per WSI.
Figure 2: WILSON matches or exceeds specialized case- and WSI-level baselines and benefits from efficient end-to-end fine-tuning. a , Multi-magnification composites versus Yottixel tile selection as alternative WSI representations, both encoded with the pretrained UNI foundation model. Top-1 macro-averaged F1 retrieval is shown for four WSI-level benchmarks, split across two sub-axes with different yy -ranges: BCTherapy and TCGA-Kidney (left, 00 – 11 ) and MayoSkin and MayoBreast (right, 00 – 0.50.5 ). Bars are single point estimates. b , Case-level zero-shot retrieval performance (macro-averaged F1 under top-5 majority voting, MV@5) of three WILSON variants (WILSON-distill, WILSON-KW, WILSON-CLIP) against MOOZY, a case-level pathology foundation model, on four case-level benchmarks: CPTAC-BRCA, MayoBreastSubtype, MayoCaseBreast and MayoCaseSkin. Error bars indicate the standard deviation across the 10 composite images generated per case; MOOZY operates directly on the WSI and therefore has no associated variance. Color-matched horizontal dashed lines mark each model’s mean across the four benchmarks. c , The same values as in b shown as a model ×\times benchmark heatmap (rows, models; columns, benchmarks; shading, macro-averaged F1). d , WSI-level zero-shot MV@5 retrieval performance of the same three WILSON variants against three WSI-level foundation models (GigaPath, TITAN, PRISM) on four WSI-level benchmarks: TCGA-BRCA, BCTherapy, MayoBreast and MayoSkin. Error bars indicate the standard deviation across the 10 composite images generated per WSI; the three baselines operate on frozen tile-level features and therefore have no associated variance. Color-matched dashed lines mark each model’s mean across the four benchmarks. e , The same values as in d shown as a model ×\times benchmark heatmap. f , The same six models and four WSI-level benchmarks shown as a radar plot (BCTherapy, top; TCGA-BRCA, right; MayoSkin, bottom; MayoBreast, left; radial axis, macro-averaged F1 from 0.00.0 to 0.80.8 ), with colors as in d . g , Effect of end-to-end fine-tuning of WILSON on MayoTNBC, a cohort of 508 triple-negative breast cancer cases, evaluated on histological subtyping (Histology) and stromal tumor-infiltrating lymphocyte (sTILs) grading. Accuracy (left) and macro-averaged F1 (right) are shown for three settings: the pretrained backbone evaluated zero-shot (Base Zero-shot), the same backbone after end-to-end fine-tuning and evaluated zero-shot with no task-specific head (FT-Zero-shot), and the fine-tuned backbone trained jointly with a task-specific classification head (FT-classification).
Figure 3: WILSON vs. PRISM across all six cohorts—MayoCaption, MayoDuodenum, MayoPancreas, MayoStomach, TCGA-Brain, TCGA-Kidney—at K∈{1,3,5,10}K\in\{1,3,5,10\} . (a) I2T recall as radar plots (one axis per cohort, R@1–R@10 left to right); (b) the same I2T values as grouped bars, with dashed lines marking each model’s across-cohort mean (WILSON ≈\approx 74.1%, PRISM ≈\approx 58.1% at R@1). (c,d) mirror (a,b) for T2I. The WILSON envelope contains the PRISM envelope almost everywhere and both narrow toward TCGA-Brain; the orange curve extends past blue only on the TCGA-Kidney axis (I2T, R@3–R@10) and the MayoPancreas axis (T2I, R@3/R@5), the two crossover cases detailed in Table 11 .
Figure 4: WILSON-CoCa generates pathology captions that more closely match reference reports than PRISM and PRISM2, on an internal held-out set and on two external cohorts. WILSON-CoCa is a CoCa-style Yu et al. (2022) generative extension of WILSON that decodes a free-text diagnostic caption directly from a composite image, compared against two general-purpose pathology vision–language foundation models capable of report generation, PRISM Shaikovski et al. (2024) and PRISM2 Shaikovski and others (2025) . a , Evaluation pipeline: a WSI is rendered as a multi-magnification composite and encoded by the model’s vision tower; the text decoder generates a caption autoregressively, which is scored against the report-derived reference using nn -gram overlap (BLEU-4, CIDEr, ROUGE-1, ROUGE-L), embedding-based similarity to the reference (BERTScore precision/recall/F1) and CLIPScore. CLIPScore is computed between the slide’s image embedding and the generated caption’s text embedding in each model’s own image–text contrastive space, and therefore measures whether the caption is consistent with what the model encodes from the slide rather than whether it resembles the reference; because each contrastive space has its own scale, CLIPScore is interpretable within a model but not across models. It is not reported for PRISM2, which exposes no general-purpose text encoder for arbitrary strings; BERTScore provides an encoder-agnostic reference-similarity measure for all three models. b , BLEU-4, ROUGE-L and CLIPScore on three cohorts (rows): 1,000 image–caption pairs from a held-out subset of the Mayo Clinic cohort spanning multiple organs and diagnoses; HistAI (48 pairs); and TCGA (50), neither external cohort seen during training. Bars show the mean over pairs, error bars one standard deviation; yy -axis ranges differ between panels, so heights are comparable within but not across panels. WILSON-CoCa leads both baselines on the overlap metrics, by roughly 60% in relative terms on the internal cohort and by smaller margins on the external cohorts, and achieves the highest BERTScore recall while BERTScore precision and F1 remain comparable across models. Per-metric values are given in Table 13 and metric definitions and interpretation caveats in Supplementary Note 9 . c , Qualitative examples (placenta, gallbladder) showing the WSI, its clinical report, the composite input and the captions generated by each model; further cases in Fig. 14
Figure 5: Data curation strategy for WILSON. a, Mayo Clinic pathology reports were segmented using an LLM. b, To maximize diagnostic diversity, the number of whole-slide images (WSIs) selected from each disease category was capped, and representative specimens were chosen from WSIs identified as key slides. c, Natural institutional distribution of WSIs before sampling. Colors indicate distinct disease groups, illustrating the strong imbalance typically observed in routine clinical archives. d, Distribution of the WILSON training dataset after the curation procedures described in panels a–d . Diversity-aware sampling substantially reduced organ- and disease-level imbalance while preserving broad diagnostic coverage. e, Overview of the WILSON-DS training dataset. f, Number of patients per diagnostic category (log scale).
Figure 6: COMPOSITE: Clustering-Oriented Composite Assembly Pipeline. a, Phase 1 builds a normal-tissue reference. Tissue regions in healthy whole-slide images (WSIs) are detected by Hue–Saturation–Density thresholding Bejnordi et al. (2016) and randomly tiled; a DINO pre-trained Vision Transformer (ViT) extracts 384-dimensional CLS-token embeddings in a single forward pass. Pooled embeddings are clustered with MiniBatch kk -means ( k=200k=200 ), and each cluster mean is replaced by its medoid so that centroids correspond to genuine tissue appearances. The 200 prototypes form a reusable organ-specific reference. b, Phase 2 scores tiles in disease slides. The same ViT emits both the CLS embedding and quality logits (valid/invalid) in one pass; sub-threshold tiles (marked x) are discarded as artifacts. Remaining tiles receive an abnormality score equal to their minimum Euclidean distance to the 200 normal centroids, higher values indicating greater deviation from normal tissue. c, Phase 3 selects tiles with spatial diversity. Naive top- kk selection (red x) concentrates tiles in a single lesion, sacrificing representativeness. Instead, the tissue bounding box is divided into a spatial grid, the highest-scoring tile per cell is selected, and globally best unused tiles fill empty cells—guaranteeing coverage of all tissue zones while biasing toward diagnostically informative locations, yielding 60 positions per slide. Ranked locations are distributed round-robin across the 10 composites in order of abnormality score, so each composite spans the full abnormality spectrum and all spatial zones rather than concentrating the most atypical locations in one image. Locations per composite is configurable: the schematic shows a six-location layout ( 6×66\times 6 grid), whereas all experiments used eight locations ( 8×88\times 8 grid; panel d). Within each composite, rows are tissue locations ordered by abnormality score and columns encode multi-scale detail, from a 1.25 ×\times context view through 5 ×\times and 20 ×\times magnifications. d, Example 8×88\times 8 composite. Eight low-power regions (A–H; 256×256256\times 256 px, 8 μ8~\mu m/pixel) were drawn from four WSIs in the CaseLuminal dataset, each contributing three mid-power (green boxes; 256×256256\times 256 px, 2 μ2~\mu m/pixel) and four high-power regions (blue boxes; 256×256256\times 256 px, 0.5 μ0.5~\mu m/pixel), assembled into a single 2048×20482048\times 2048 px composite.
Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Case-level image retrieval performance of the vision encoders. Retrieval uses each model’s vision encoder only (no text encoder at inference). All whole-slide images of a case were represented in a single case-level composite before getting its corresponding embedding, and each query case was matched against the remaining cases by cosine similarity; the label of the retrieved neighbour(s) was transferred to the query. Columns give the label-transfer rule (Top-1, or majority vote over the k=3,5,7k=3,5,7 nearest neighbours) and rows the resulting metric (accuracy, balanced accuracy, macro-averaged F1-score). Axes are the four evaluation cohorts; the radial scale spans 00 – 11 with gridlines every 0.20.2 . All three WILSON variants — which differ only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive) — exceed MOOZY on every cohort, metric and kk , with the largest margins on balanced accuracy and macro-averaged F1-score. Chance level differs per axis because class counts and prevalences differ between cohorts.
Figure 8: Case-level image retrieval accuracy by cohort, macro-averaged F1-score. Per-cohort breakdown of the case-level retrieval results summarized in Fig. 7 , using the vision encoder of each model only. Panels correspond to the label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours); groups within each panel are the four evaluation cohorts (CPTAC-BRCA, Mayo Breast Subtyping, Mayo Breast Diagnosis, Mayo Skin). Bars show macro-averaged F1-score for MOOZY and three WILSON variants differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive); error bars denote [s.d. across n folds / seeds][\text{s.d.\ across }n\text{ folds / seeds}] and horizontal dashed lines mark each model’s mean across the four cohorts. Note the truncated vertical axis. WILSON-distill achieves the highest cohort mean at every kk and the largest single-cohort margin on Mayo Skin (0.62 vs. 0.28 for MOOZY at k=1k=1 ; 0.66 vs. 0.27 at k=7k=7 ). Gains are smaller on the breast cohorts, where MOOZY is comparable to WILSON-CLIP on Mayo Breast Subtyping and exceeds it on CPTAC-BRCA at every kk , indicating that the advantage is cohort-dependent rather than uniform.
Figure 9: Complete case-level image retrieval results. Numerical values underlying Figs. 7 and 8 , using the vision encoder of each model only. Panels report accuracy (left), balanced accuracy (centre) and macro-averaged F1-score (right) on a common color scale; columns are the four evaluation cohorts and rows every combination of model (MOOZY, WILSON-distill, WILSON-KW, WILSON-CLIP) and label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours), grouped by kk . Three patterns are visible. First, the WILSON encoders’ advantage is largest on Mayo Skin, where all three exceed MOOZY by roughly 0.3 on every metric and at every kk (e.g. 0.67 vs. 0.29 accuracy at k=7k=7 ). Second, on CPTAC-BRCA the accuracy of all models (0.42–0.58) substantially exceeds their balanced accuracy and macro-averaged F1-score (0.29–0.40), indicating that raw accuracy there is inflated by class imbalance; MOOZY shows the widest gap (0.57 vs. 0.35 at k=3k=3 ), consistent with a bias toward the majority class. Third, performance generally improves with increasing kk for all models, with most of the gain realised by k=5k=5 .
Figure 10: Slide-level image retrieval performance of the vision encoders. Retrieval uses each model’s vision encoder only (no text encoder at inference). Each whole-slide image was encoded into a single slide embedding and matched against the remaining slides by cosine similarity; the label of the retrieved neighbour(s) was transferred to the query. Columns give the label-transfer rule (Top-1, or majority vote over the k=3,5,7k=3,5,7 nearest neighbours) and rows the resulting metric (accuracy, balanced accuracy, macro-averaged F1-score). Axes are the four evaluation cohorts (BCTherapy, TCGA-BRCA, Mayo Skin, Mayo Breast Diagnosis); the radial scale spans 00 – 11 with gridlines every 0.20.2 . Six slide encoders are compared: GigaPath, TITAN, PRISM, and three WILSON variants differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive). The WILSON variants match or exceed the published foundation models across cohorts, metrics and values of kk , with the clearest separation on Mayo Skin and Mayo Breast Diagnosis; GigaPath trails on every axis. Chance level differs per axis because class counts and prevalences differ between cohorts.
Figure 11: Slide-level image retrieval accuracy by cohort, macro-averaged F1-score. Per-cohort breakdown of the slide-level retrieval results summarized in Fig. 10 , using the vision encoder of each model only. Panels correspond to the label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours); groups within each panel are the four evaluation cohorts (TCGA-BRCA, BCTherapy, Mayo Breast Diagnosis, Mayo Skin). Within each group, bars are ordered GigaPath, TITAN, PRISM, WILSON-distill, WILSON-KW, WILSON-CLIP, the last three differing only in the text supervision used during alignment (report distillation, keywords, CLIP-style contrastive); error bars denote [s.d. across n folds / seeds][\text{s.d.\ across }n\text{ folds / seeds}] and horizontal dashed lines mark each model’s mean across the four cohorts. Note the truncated vertical axis. The WILSON encoders lead on Mayo Skin at every kk (up to 0.69 for WILSON-distill at k=7k=7 vs. 0.63 for PRISM and 0.57 for TITAN) and are comparable to TITAN and PRISM on BCTherapy and Mayo Breast Diagnosis, while TITAN and PRISM retain a small advantage on TCGA-BRCA at k=1k=1 . GigaPath performs worst in every cohort and at every kk , most markedly on Mayo Breast Diagnosis and Mayo Skin.
Figure 12: Complete slide-level image retrieval results. Numerical values underlying Figs. 10 and 11 , using the vision encoder of each model only. Panels report accuracy (left), balanced accuracy (centre) and macro-averaged F1-score (right); note that each panel has its own color scale, so shading is comparable within but not across panels. Columns are the four evaluation cohorts and rows every combination of model (GigaPath, TITAN, PRISM, WILSON-distill, WILSON-KW, WILSON-CLIP) and label-transfer rule (majority vote over the k=1,3,5,7k=1,3,5,7 nearest neighbours), grouped by kk . Three patterns are visible. First, no single encoder leads everywhere: WILSON-distill is strongest on Mayo Skin at every kk (0.69 accuracy and 0.69 macro F1 at k=7k=7 , vs. 0.64 and 0.63 for PRISM), WILSON-CLIP on BCTherapy (0.77 accuracy at k=5k=5 ), PRISM on Mayo Breast Diagnosis (0.72 accuracy at k=7k=7 ) and TITAN on TCGA-BRCA (0.88 accuracy at k=7k=7 ). Second, the gap between accuracy and balanced accuracy is pronounced on TCGA-BRCA for all six encoders (e.g. 0.88 vs. 0.43 for TITAN at k=7k=7 ), indicating that accuracy in that cohort is dominated by class imbalance and that the balanced-accuracy and macro-F1 panels are the informative ones. Third, GigaPath is the weakest encoder throughout, and its deficit is largest on Mayo Breast Diagnosis and Mayo Skin (balanced accuracy 0.25–0.34) while remaining competitive on TCGA-BRCA accuracy (0.73–0.79), a pattern consistent with majority-class prediction rather than discriminative retrieval.
Figure 13: Comprehensive Performance Comparison of WILSON and PRISM Models on Cross-Modal Retrieval Tasks. The figure presents recall metrics (R@1, R@3, R@5, R@10) evaluated across Mayo Clinic (Diagnoses, Duodenum, Pancreas, Stomach) and TCGA (Brain, Kidney) datasets. Top section (Bar charts): Quantitative comparison showing exact performance values for each metric and dataset. Bottom section (Radar charts): Multi-dimensional performance visualization enabling visual assessment of relative strengths across all metrics and datasets simultaneously. Each row represents a cross-modal retrieval direction: Image-to-Text (I2T) retrieves relevant diagnostic text reports given medical images; Text-to-Image (T2I) retrieves relevant medical images given text descriptions. Blue elements (bars and lines) represent WILSON model performance, purple elements represent PRISM model performance. All metrics are reported as percentages, with bar chart values displayed above each bar for precise quantitative reference.
Figure 14: Additional qualitative comparison of generated slide-level reports. Seven further cases (colon, uterus, urinary bladder, liver, stomach, prostate, breast), showing the whole-slide image and its clinical report, the composite input, and the descriptions generated by WILSON, PRISM and PRISM2. Conventions as in Fig. 4 .
Figure 15: Caption-quality evaluation across cohorts, models, and metrics. Grouped bar charts of eight caption-quality metrics (columns) for PRISM, PRISM2 and WILSON (ours) on three cohorts (rows): held-out Mayo Clinic, HistAI and TCGA. Bars show the mean over caption pairs and error bars one standard deviation; annotations give the mean value. Metrics are BLEU-4 ( nn -gram precision), CIDEr (consensus-based TF–IDF-weighted nn -gram similarity), ROUGE-1 (unigram overlap), ROUGE-L (longest common subsequence), BERTScore precision/recall/F1 (contextual similarity from BERT embeddings) and CLIPScore (image–text similarity in a contrastive embedding space). CLIPScore is computed within each model’s own embedding space and is therefore not comparable across models; PRISM2 exposes no general-purpose text encoder for arbitrary strings, so no PRISM2 bar is shown. The rightmost panel (“Avg”) summarizes each model as the mean of its metric values after min–max normalization within metric and cohort, rescaled to 0–100 [excluding CLIPScore]. Note that yy -axis ranges differ between panels: bar heights are comparable within a panel but not across panels.
DatasetTaskYottixel + UNIComposite + UNI
BCTherapyTreatment response0.510.56
MayoSkinDiagnostic classification0.330.34
MayoBreastDiagnostic classification0.310.35
TCGA-KidneyDiagnostic classification0.890.88
Appendix
Table 1: Composite images vs. Yottixel patches as WSI representations, both encoded with the pretrained UNI foundation model. Values are top-1 macro-F1 retrieval performance (higher is better) across two public and two internal Mayo Clinic cohorts.
 Base zero-shotFine-tuned zero-shotFine-tuned classification
TaskaccMacro avgaccMacro avgaccMacro avg
Histology0.6170.4010.6370.4280.6900.557
sTILs0.4120.4250.4320.4560.5410.531
Appendix
Table 2: End-to-end fine-tuning of WILSON (dense embedding) on MayoTNBC dataset.Accuracy (acc) and macro-averaged F1 (Macro avg) are reported for histologic subtyping and sTILs grading under three settings: the pretrained backbone evaluated zero-shot, the same backbone after end-to-end fine-tuning evaluated zero-shot, and the fine-tuned backbone with a jointly trained classification head.
DirectionModelR@1 (%)R@3 (%)R@5 (%)R@10 (%)
Image-to-TextWILSON78.2985.7588.8292.11
PRISM60.0966.4569.7475.66
Δ\Delta (W −- P)+18.20+19.30+19.08+16.45
Text-to-ImageWILSON81.1494.0896.4998.03
PRISM56.3677.8582.8987.72
Δ\Delta (W −- P)+24.78+16.23+13.60+10.31
Appendix
Table 3: Cross-modal retrieval on the MayoCaption dataset, horizontal evaluation across all diagnoses combined. Δ\Delta denotes WILSON minus PRISM.
DirectionModelR@1 (%)R@3 (%)R@5 (%)R@10 (%)
Image-to-TextWILSON77.7883.3386.3689.90
PRISM64.1465.1567.1768.69
Δ\Delta (W −- P)+13.64+18.18+19.19+21.21
Text-to-ImageWILSON80.8194.9597.9898.48
PRISM55.5667.6874.2478.79
Δ\Delta (W −- P)+25.25+27.27+23.74+19.70
Appendix
Table 4: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Duodenum. Δ\Delta denotes WILSON minus PRISM.
DirectionModelR@1 (%)R@3 (%)R@5 (%)R@10 (%)
Image-to-TextWILSON78.1889.0990.9193.64
PRISM59.0963.6469.0980.00
Δ\Delta (W −- P)+19.09+25.45+21.82+13.64
Text-to-ImageWILSON83.6492.7395.4599.09
PRISM70.9195.4598.1898.18
Δ\Delta (W −- P)+12.73-2.73-2.73+0.91
Appendix
Table 5: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Pancreas. Δ\Delta denotes WILSON minus PRISM.
DirectionModelR@1 (%)R@3 (%)R@5 (%)R@10 (%)
Image-to-TextWILSON85.8191.8993.9296.62
PRISM81.7687.8490.5495.95
Δ\Delta (W −- P)+4.05+4.05+3.38+0.68
Text-to-ImageWILSON85.1495.9596.6297.97
PRISM65.5492.5795.2796.62
Δ\Delta (W −- P)+19.59+3.38+1.35+1.35
Appendix
Table 6: Cross-modal retrieval on the MayoCaption dataset, per-organ vertical evaluation for Stomach. Δ\Delta denotes WILSON minus PRISM.
DirectionModelR@1 (%)R@3 (%)R@5 (%)R@10 (%)
Image-to-TextWILSON41.9458.8765.3278.23
PRISM20.1640.3255.6581.45
Δ\Delta (W −- P)+21.77+18.55+9.68-3.23
Text-to-ImageWILSON54.0370.9782.2688.71
PRISM41.1362.1071.7787.10
Δ\Delta (W −- P)+12.90+8.87+10.48+1.61
Appendix
Table 7: Cross-modal retrieval on the TCGA Brain dataset. Δ\Delta denotes WILSON minus PRISM.
DirectionModelR@1 (%)R@3 (%)R@5 (%)R@10 (%)
Image-to-TextWILSON82.5686.0593.0296.51
PRISM63.2293.1096.55100.00
Δ\Delta (W −- P)+19.34-7.06-3.53-3.49
Text-to-ImageWILSON77.9190.7094.19100.00
PRISM54.0259.7767.8278.16
Δ\Delta (W −- P)+23.88+30.93+26.37+21.84
Appendix
Table 8: Cross-modal retrieval on the TCGA Kidney dataset. Δ\Delta denotes WILSON minus PRISM.
 Image-to-Text R@1 (%)Text-to-Image R@1 (%)
DatasetWILSONPRISMWILSONPRISM
MayoCaption (all)78.2960.098 1.1456.36
Mayo – Duodenum77.7864.1480.8155.56
Mayo – Pancreas78.1859.0983.6470.91
Mayo – Stomach85.8181.7685.1465.54
TCGA – Brain41.9420.1654.0341.13
Appendix
Table 9: Summary of R@1 retrieval performance for WILSON and PRISM across all evaluated datasets, for both image-to-text (I2T) and text-to-image (T2I) retrieval directions.
  WILSON (%)PRISM (%)
DatasetDirectionR@1R@3R@5R@10R@1R@3R@5R@10
MayoCaptionI2T78.2985.7588.8292.1160.0966.4569.7475.66
T2I81.1494.0896.4998.0356.3677.8582.8987.72
Mayo – DuodenumI2T77.7883.3386.3689.9064.1465.1567.1768.69
T2I80.8194.9597.9898.4855.5667.6874.2478.79
Mayo – PancreasI2T78.1889.0990.9193.6459.0963.6469.0980.00
Appendix
Table 10: Comprehensive comparison of WILSON and PRISM cross-modal retrieval recall (R@1, R@3, R@5, R@10) across all used Mayo Clinic and TCGA dataset/organ splits, for both image-to-text (I2T) and text-to-image (T2I) retrieval.
 R@1R@10
Dataset / DirectionPRISMWILSONΔ\DeltaPRISMWILSONΔ\Delta
MayoCaption I2T60.178.3+18.275.792.1+16.4
MayoCaption T2I56.481.1+24.787.798.0+10.3
Mayo Duodenum I2T64.177.8+13.768.789.9+21.2
Mayo Duodenum T2I55.680.8+25.278.898.5+19.7
Mayo Pancreas I2T59.178.2+19.180.093.6+13.6
Appendix
Table 11: WILSON vs. PRISM at R@1 and R@10, all cohorts and directions. Δ\Delta = WILSON −- PRISM; bold negative values indicate PRISM leads.
ModelArchitectureTotal paramsFLOPs per tileFLOPs per WSI
Prov-GigaPathViT-G/14 (DINOv2) + LongNet1,220M555.4G5.56P
PRISM2Virchow2 (ViT-H/14) + PerceiverAR1,251M413.2G4.13P
PRISMVirchow (ViT-H/14) + Perceiver730.3M406.7G4.07P
TITANCONCH v1.5 (ViT-L/16) + slide encoder354.6M70.0G703.19T
MOOZYViT-S/8 (Lunit DINO) + ViT 6L, 768-d85.8M—25.07T
WILSONConvNeXt-Base + extended head (single pass)133.7M—2.58T
Appendix
Table 12: Parameter counts and computational cost of pathology foundation models at a common workload of 10,000 tiles per whole-slide image (WSI). For tile-encoder plus slide-aggregator pipelines, cost per WSI is the tile encoder evaluated N=10,000N=10{,}000 times plus one pass of the slide aggregator. Counts cover the embedding path only; text decoders (PRISM’s BioGPT, PRISM2’s Phi-3 Mini, TITAN’s text tower) are excluded. A dash indicates that no per-tile cost is defined: WILSON encodes one 2048×20482048\times 2048 composite of 8×8=648\times 8=64 selected tissue regions in a single forward pass, and the MOOZY figure is reported for its frozen patch encoder together with its slide encoder. Prov-GigaPath’s slide encoder uses LongNet dilated attention, so its actual slide-encoder cost is lower than the dense-attention figure shown. Rows are ordered by cost per WSI.
 Mayo ClinicHistAITCGA
MetricPRISMPRISM2WILSONPRISMPRISM2WILSONPRISMPRISM2WILSON
BLEU-40.00040.00340.01550.00000.00040.00170.00190.00230.0085
CIDEr0.00220.05500.22360.09440.01990.02360.02040.02720.0903
ROUGE-10.14240.23070.37110.11960.09480.16170.18250.21480.2136
ROUGE-L0.11890.16700.25560.10230.08010.12620.14970.15960.1592
BERTScore-P0.88120.89460.86270.85190.82900.83830.88080.87400.8609
Appendix
Table 13: Caption-quality evaluation across three models and three cohorts. BLEU-4 ( nn -gram overlap), CIDEr (consensus-based), ROUGE-1 (unigram overlap), ROUGE-L (longest common subsequence), BERTScore-P/R/F1 (contextual similarity from BERT embeddings) and CLIPScore (CLIP-based semantic similarity) are reported as raw scores. “Avg” is the mean performance across metrics after min–max normalization within each metric, rescaled to 0–100. Best model per cohort is bold. a CLIPScore is computed in each model’s own contrastive embedding space and is therefore not comparable across models; PRISM2 exposes no general-purpose text encoder for arbitrary strings. Excluded from bolding and from Avg.
DatasetOrganTaskClassesPatientsSlidesFormat
Mayo189KVariousWILSON TrainingN/A112,296189,291DICOM
MayoBreastBreast (tumor, nontumor)Histological subtype classification121,2891,339DICOM
MayoSkinSkin (tumor, nontumor)Histological subtype classification20677688DICOM
MayoTNBCBreast carcinoma (triple negative)Fine tuning4/4508508Philips Tiff
MayoCaptionThree organsCross-modal retrieval18397456DICOM
MayoDuodenumDuodenumCross-modal retrieval7172198DICOM
Appendix
Table 14: Summary of training and validation WSI datasets. The MayoTNBC dataset contains two tasks, histological subtype classification and stromal tumor-infiltrating lymphocyte (sTIL) prediction, each comprising four classes. The MayoCaption dataset was constructed by combining the MayoDuodenum, MayoStomach, and MayoPancreas datasets for horizontal cross-modal retrieval. Histological subtype annotations for the TCGA-derived datasets were assigned according to Uegami et al. Uegami et al. (2026) .
DatasetOrganTaskClassesCasesWSIsFormat
MayoCaseBreastBreast (tumor, nontumor)Histological subtype classification124944,361DICOM
MayoCaseSkinSkin (tumor, nontumor)Histological subtype classification205001,280DICOM
MayoBreastSubtypeBreast carcinomaMolecular subtype classification38462,254DICOM
CPTAC-BRCA National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020)Breast carcinomaPAM50 subtype classification5198653SVS
Appendix
Table 15: Summary of case-level validation WSI datasets.
ClassCount
Invasive carcinoma of no special type812
Invasive lobular carcinoma204
Ductal and lobular carcinoma27
Mucinous adenocarcinoma20
Metaplastic carcinoma13
Medullary carcinoma7
Appendix
Table 16: Class distribution of the TCGA-BRCA dataset. Cancer Genome Atlas Research Network et al. (2013)
ClassCount
RD119
pCR41
Appendix
Table 17: Class distribution of the BCTherapy dataset Sammut et al. (2022) .
ClassCount
Invasive breast carcinoma of no special type (ductal)350
Invasive lobular carcinoma70
Mixed ductal and lobular carcinoma70
Mucinous carcinoma70
Metaplastic carcinoma70
Ductal carcinoma in situ210
Appendix
Table 18: Class distribution of the MayoBRCA dataset.
ClassCount
Lentigo41
Actinic keratosis40
Malignant melanoma40
Blue nevus39
Dermatofibroma38
Verrucae38
Appendix
Table 19: Class distribution of the MayoSkin dataset.
ClassCount
Hormone receptor-positive/HER2-negative269
HER2-positive303
Triple-negative274
Appendix
Table 20: Class distribution of the MayoBreastSubtype dataset.
ClassCount
Luminal A57
Basal-like27
Luminal B17
HER2-enriched14
Normal-like5
Appendix
Table 21: Class distribution of the CPTAC-BRCA dataset National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC) (2020) .
ClassCount
Invasive carcinoma of no special type (ductal)44
Invasive lobular carcinoma42
Mixed ductal and lobular carcinoma40
Mucinous carcinoma41
Metaplastic carcinoma38
Ductal carcinoma in situ40
Appendix
Table 22: Class distribution of the MayoCaseBreast dataset.
ClassCount
Sebaceous gland hyperplasia31
Dermatofibroma31
Malignant melanoma31
Skin tag29
Nevus with atypia28
Seborrheic keratosis27
Appendix
Table 23: Class distribution of the MayoCaseSkin dataset.
WSI IDOrgan
case_00760/slide_H&E_0.tiffBone
case_03070/slide_H&E_0.tiffBone
case_09511/slide_H&E_0.tiffBone
case_11368/slide_H&E_0.tiffBone
case_11593/slide_H&E_0.tiffBone
case_01871/slide_H&E_0.tiffBreast
Appendix
Table 24: HistAI Nechaev et al. (2025) whole-slide images (WSIs) and corresponding organ categories.
WSI IDOrgan
TCGA-06-0143-01Z-00-DX3.e9011249-11f6-454b-98f1-7f2bcfea228c.svsBrain
TCGA-06-0154-01Z-00-DX1.e52d5443-e663-451f-8a1f-83ec5bb9a401.svsBrain
TCGA-DB-A75P-01Z-00-DX1.11655479-5A76-4689-B8F6-319F882B5EB7.svsBrain
TCGA-DU-A6S3-01Z-00-DX1.08CE6930-D4E8-4084-A8C0-F2911B069E72.svsBrain
TCGA-HT-7680-01Z-00-DX3.C157242E-F4D2-4246-8016-B0A44C4C4AAD.svsBrain
TCGA-AC-A2QH-01Z-00-DX1.00B8BFFF-F1E2-4F99-A969-8DD7EE4F8E0B.svsBreast
Appendix
Table 25: TCGA Weinstein et al. (2013) whole-slide images (WSIs) and corresponding organ categories.
StagePaired WSIsCompositesVal. frac.EpochsGPUsEff. batch
WILSON-distill178,0201,759,7870.022032 (4 ×\times 8)2,048
WILSON-KW (keyword regr.)178,0881,760,4610.1078 (1 ×\times 8)512
WILSON-CLIP, Phase 21,760,461 *1,760,461 *0.199 *2032 (4 ×\times 8)576
WILSON-CLIP, Phase 31,760,461 *1,760,461 *0.199 *1532 (4 ×\times 8)768
WILSON-CoCa captioning178,0201,759,7870.022032 (4 ×\times 8)1,024
Appendix
Table 26: Training corpus and compute per vision–language stage. * WILSON-CLIP uses a fixed 1,410,600/349,861 train/val composite manifest (§ 4 ); all other rows are approximate WSI-level splits computed from each script’s val_frac argument applied to the paired corpus above, since exact per-sample assignment is determined at run time by the data loader. “Eff. batch” is GPUs ×\times per-GPU batch size ×\times gradient-accumulation steps.
Figure 16: Example of composite image (single WSI). MayoBreast dataset.
Figure 17: WILSON vision encoder ( ConvNeXtV3Extended ). a , Stage-by-stage architecture for the 2048×20482048\times 2048 reference input (an 8×88\times 8 grid of 256-px tiles). Tensor glyphs encode each feature map (face ∝log⁡2\propto\log_{2} spatial size, depth ∝log⁡2\propto\log_{2} channels), making the progression from a broad, shallow 128×512×512128\times 512\times 512 map to a narrow, deep 3072×8×83072\times 8\times 8 map visible at a glance. A DINOv3-pretrained ConvNeXt-Base backbone (depths 3,3,27,33,3,27,3 ; widths 128,256,512,1024128,256,512,1024 ; stride 32; 87.57M params) is followed by three stride-2 downsampling blocks (extra_blocks ×3\times 3 ; 46.15M params) that extend the width to 3072 (stride 256) and global average pooling, for 133.7M parameters. Column-aligned bars show where parameters and compute (conv/linear MACs) reside: Stage 3 dominates both (57.4M params, 939 GMACs), whereas the extended head holds 34.5% of the parameters but only 0.6% of the compute. b , Anatomy of the two block types with symbolic parameter counts; the first extended block omits its LayerNorm (its input is already normalized by the backbone), and a 1×11\times 1 convolutional fallback replaces the stride-2 convolution only when a block’s input is already 1×11\times 1 (e.g. 224-px inputs). c , The three operating resolutions. At 224–256 px (tile-based SSL) only the backbone is used and its 1024-d GAP output is taken directly (20 GMACs); at 512×512512\times 512 (composite-based SSL pretraining, § 4 ) the final map is 3072×2×23072\times 2\times 2 (81 GMACs); at 2048×20482048\times 2048 (composite-based vision–language training, § 4 ) the final 3072×8×83072\times 8\times 8 map provides 64 spatial tokens to the CoCa attentional pooler (§ 4 ) (1,291 GMACs). d , Warm-start of the extra-block convolutions when the backbone is loaded from a domain-specific SSL checkpoint: the 1024×512×2×21024\times 512\times 2\times 2 filters of the last backbone downsampling layer are tiled to each extra-block shape and rescaled to He-initialization magnitude ( std=2/fan_in\mathrm{std}=\sqrt{2/\mathrm{fan\_in}} ), preserving filter directions while keeping activations within fp16 range; biases are tiled likewise and the extra-block LayerNorms start at (1,0)(1,0) .
ComponentOperationIn →\to Out ch.StrideOutput ( 204822048^{2} )Params%GMACs ‡
Backbone — DINOv3 ConvNeXt-Base (87,566,464 params, 65.5%; 1,283 GMACs, 99.4%)
StemConv 4 ⁣× ⁣44\!\times\!4 s4 + LN3→1283\to 1284128 ⁣× ⁣512 ⁣× ⁣512128\!\times\!512\!\times\!5126,5280.01.6
Stage 13 ×\times Block1281284128 ⁣× ⁣512 ⁣× ⁣512128\!\times\!512\!\times\!512415,4880.3108
Downsample 1LN + Conv 2 ⁣× ⁣22\!\times\!2 s2128→256128\to 2568256 ⁣× ⁣256 ⁣× ⁣256256\!\times\!256\!\times\!256131,5840.18.6
Stage 23 ×\times Block2562568256 ⁣× ⁣256 ⁣× ⁣256256\!\times\!256\!\times\!2561,617,4081.2106
Downsample 2LN + Conv 2 ⁣× ⁣22\!\times\!2 s2256→512256\to 51216512 ⁣× ⁣128 ⁣× ⁣128512\!\times\!128\!\times\!128525,3120.48.6
Appendix
Table 27: WILSON vision encoder ( ConvNeXtV3Extended ) — parameter and compute breakdown by component. Output shapes, cumulative stride and MACs are given for the 2048×20482048\times 2048 reference input (an 8×88\times 8 grid of 256-px tiles), the default resolution for composite-based vision–language training. Each “Block” is a standard ConvNeXt residual block (Fig. 17 b). Parameter counts were verified by instantiating the model. The extended head holds 34.5% of the parameters but accounts for only 0.6% of the compute, because it operates on ≤ ⁣32×32\leq\!32\times 32 maps; conversely Stage 3 alone accounts for 72.7% of the MACs. † Blocks 2–3 also instantiate a 1×11\times 1 convolutional fallback (2,099,200 and 6,294,528 params, included in the totals) that is used only when the block’s input is already 1×11\times 1 ; it is dormant for all inputs ≥256\geq 256 px, leaving 125,327,488 active parameters ( ≈\approx 125.3M). ‡ Multiply–accumulates of convolution and linear layers only (LayerNorm, GELU, layer scale and pooling omitted); 1 MAC == 2 FLOPs. At 512×512512\times 512 the total is 80.7 GMACs and at 256×256256\times 256 (backbone only) 20.1 GMACs.
HyperparameterValue
Input resolution2048×20482048\times 2048 px
Vision spatial tokens8×8=648\times 8=64 , 3072-dim
Vision-encoder trainable params133.72M
Attentional-pooler queries / heads256 / 8
Joint embedding dim768
Decoder width768
Appendix
Table 28: WILSON + CoCa vision-language training hyperparameters.

Explore similar work

CardsList

  1. MOOZY: A Patient-First Foundation Model for Computational Pathology

    Mar 27, 2026Yousef Kotp, Vincent Quoc-Huy Trinh, Christopher Pal +1Computational PathologySelf-Supervised Pre-Training

    CSSE, Concordia University, Montréal, Canada · Mila – Québec AI Institute, Montréal, Canada · CHUM, Université de Montréal, Montréal, Canada +4

  2. ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts

    Jul 10, 2026Jiawen Li, Tian Guan, Huijuan Shi +5Computational PathologyUnified Multimodal Models

    Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China · Department of Engineering Science, University of Oxford, Oxford, UK +1

  3. PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image

    Jul 21, 2026Dankai Liao, Tianyi Zhang, Yufeng Wu +6VLM EvaluationMedical VLMs

    Department of Biomedical Engineering, National University of Singapore, Singapore 117417, Singapore