PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology

Fanqi Cheng; Kuo Gong; Shangke Liu; Beidi Zhao; Junchao Zhu; Zheyu Zhu; Leiyue Zhao; Fengbei Liu

cs.CVOct 8, 2026

PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology

Authors: Fanqi Cheng, Kuo Gong, Shangke Liu, Beidi Zhao, Junchao Zhu, Zheyu Zhu, Leiyue Zhao, Fengbei Liu, +10 more

Organizations: New York University, New York, NY, USA · Columbia University, New York, NY, USA · Weill Cornell Medicine, New York, NY, USA · Cornell University, Ithaca, NY, USA · University of British Columbia, Vancouver, BC, Canada · Vanderbilt University, Nashville, TN, USA · University of Pennsylvania, Philadelphia, PA, USA · Johns Hopkins University, Baltimore, MD, USA · New York Medical College, New York, NY, USA · BC Cancer Agency, Vancouver, BC, Canada · The University of Texas MD Anderson Cancer Center, Houston, TX, USA · Cornell Tech, New York, NY, USA

Abstract

Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characterized. Existing pathology VLM benchmarks largely rely on canonical closed-set prompts or perturb only generic templates, treating language as a fixed evaluation component rather than a variable axis of model behavior. In clinical practice, however, diagnostic language varies across reports, institutions, and candidate diagnoses. We introduce PathLang, a language-centered and clinically grounded zero-shot benchmark. PathLang holds the underlying slides, ground-truth labels, and image-text evaluation direction fixed while systematically varying only the diagnostic language, so that performance differences reflect how a diagnosis is phrased rather than what is imaged. The language variation follows how pathologists actually rephrase diagnoses (terminology, specificity, and reporting style), and all prompts and candidate pools are validated by six board-certified pathologists. PathLang covers four task families: (1) zero-shot classification with image-text alignment analysis, (2) cross-modal retrieval, (3) paraphrase robustness, including semantic-equivalence paraphrases, length and reporting-style variation, and prompt ensembling, and (4) open-vocabulary diagnosis retrieval over four candidate pools with distinct forms of semantic competition. Across nine VLMs and five public datasets spanning four organs, we find that performance is highly sensitive to clinically equivalent paraphrases, varies substantially across forms of semantic competition, and that image-text alignment quality does not necessarily translate into inter-class separability. We release the prompt corpus, candidate pools, pre-computed text embeddings, and evaluation code at https://anonymous.4open.science/r/PathLang.

Figures & tables

Fig. 1: Overall framework of PathLang. a. We focus on dual-encoder pathology foundation models; b. We use five datasets including WSIs (20 $\times$ ) from four organs and the pathologist validated prompt pool; c. Our evaluation contains four tasks: zero-shot classification, cross-modal retrieval, paraphrase robustness and open vocabulary retrieval. d. We present a variety of results.

Study	Cls.	I2I	I2T	T2I	Slide	Lang. Sens.	Clin. Valid.
MI-Zero [ 22 ]	✓	–	–	–	✓	–	–
PLIP [ 16 ]	✓	✓	–	✓	–	–	–
Quilt-1M [ 17 ]	✓	–	✓	✓	–	–	–
CONCH [ 21 ]	✓	–	✓	✓	✓	–	–
MUSK [ 32 ]	✓	✓	✓	✓	✓	–	–
Histo-VL [ 2 ]	✓	–	–	–	–	✓	–

Table 1: Positioning against representative pathology foundation-model and VLM studies. Rows are ordered roughly chronologically. A check mark indicates that the work explicitly evaluates the corresponding axis. Cls. : classification or downstream diagnostic performance; I2I : image-to-image retrieval; I2T/T2I : image-to-text and text-to-image retrieval; Lang. Sens. : explicit evaluation of sensitivity to textual variation; Clin. Valid. : clinical validation of diagnostic text prompts or candidate descriptions.

Fig. 2: Zero-shot feature space alignment (left) and discriminative capacity (right) across nine pathology VLMs. Blue indicates positive alignment and strong class separability, while red denotes negative values and poor alignment or semantic confusion. Results are reported under the upper-bound aggregation protocol (Section 3.4 ); per-setting results and rank-stability analysis are in A.1.1 .

Fig. 3: MRR of cross-modal retrieval (T2). a. I2I, b. I2T, and c. T2I retrieval. Detailed quantitative results are shown in C.2 (I2I: Table 2 , I2T: Table 3 , T2I: Table 4 ). Results are reported under the upper-bound aggregation protocol (Section 3.4 ); per-setting results and rank-stability analysis are in A.1.1 .

Fig. 4: Results of paraphrase robustness (T3). a. Variant 1: accuracy across models and datasets under semantic-equivalence paraphrases. Detailed quantitative results are reported in C.3 , Table 7 . b. Variant 2: accuracy stratified by prompt length and reporting style (short, medium, long, and clinical). Detailed quantitative results are reported in C.3 , Table 8 .

Fig. 5: Recall@K of open-vocabulary diagnosis retrieval (T4). Each subplot shows retrieval performance of nine VLMs across five pathology datasets (columns) under four candidate pool settings (rows) that impose distinct forms of semantic competition: open-vocabulary, label-specific mixed, cross-organ, and a global all-datasets pool. Each line represents an individual model; the best and second-best models are highlighted with the thickest and second-thickest curves, respectively. For each dataset and candidate set, models are plotted using their best-performing aggregation setting. Full per-setting results and rank-stability analysis are given in A.1.1 , where model rankings are stable across settings (Spearman

\rho\geq 0.967

for all four candidate sets). Detailed quantitative results are reported in C.4 (Set 1: Table 12 , Set 2: Table 13 , Set 3: Table 14 , Set 4: Table 15 ).

Model	CAMELYON16			CAMELYON17			PANDA			TCGA-GBMLGG			UniToPatho
Model	Acc	AUC	mF1	Acc	AUC	mF1	Acc	AUC	mF1	Acc	AUC	mF1	Acc	AUC	mF1
QuiltNet [ 17 ]	0.402	0.561	0.287	0.072	0.528	0.034	0.127	0.542	0.109	0.577	0.546	0.389	0.169	0.534	0.106
PLIP [ 16 ]	0.561	0.565	0.550	0.323	0.528	0.212	0.250	0.506	0.088	0.603	0.390	0.505	0.376	0.529	0.199
MI-Zero [ 22 ]	0.407	0.461	0.292	0.176	0.494	0.084	0.160	0.493	0.090	0.444	0.331	0.325	0.152	0.478	0.145
CONCH [ 21 ]	0.578	0.527	0.423	0.629	0.502	0.199	0.140	0.496	0.108	0.458	0.487	0.399	0.097	0.495	0.080
BiomedCLIP [ 35 ]	0.612	0.505	0.451	0.557	0.499	0.198	0.271	0.498	0.071	0.577	0.546	0.389	0.038	0.470	0.017

Table 2: Accuracy, AUC and macro-F1 of zero-shot classification (T1). Bold = best; underline = second best; ties at the reported precision are both marked. For each (model, dataset) pair we report the aggregation setting that maximizes that model’s accuracy on that dataset, so all three metrics within a cell come from a single configuration (Section 3.4 ). Per-setting results and rank-stability analysis are in A.1.1 .

Models	CAMELYON16									CAMELYON17									PANDA
Models	L1	L2	L3	L4	L5	L6	L7	L8	L9	L1	L2	L3	L4	L5	L6	L7	L8	L9	L1	L2	L3	L4	L5	L6	L7	L8	L9
QuiltNet [ 17 ]	0.472	-00.1	-07.6	-07.3	-10.2	-12.7	-14.5	-14.8	-14.8	0.165	-27.2	-32.0	-36.9	-42.4	-47.8	-48.6	-53.3	-56.2	0.185	+03.4	+00.7	+06.2	+05.2	+13.2	+17.6	+19.3	+24.9
PLIP [ 16 ]	0.543	-05.0	-04.7	-08.6	-07.2	-08.6	-05.2	-07.7	-10.1	0.229	+04.1	-02.4	-09.5	-14.9	-18.8	-24.7	-24.9	-28.3	0.265	+03.7	+06.2	+07.1	+06.3	+05.9	+05.2	+04.1	+03.5
MI-Zero [ 22 ]	0.481	-00.1	-04.3	-04.9	-07.3	-09.3	-10.7	-11.7	-11.9	0.189	-23.0	-19.3	-31.9	-34.0	-36.4	-37.3	-37.8	-37.4	0.158	+04.4	+03.7	-03.7	-08.4	-08.9	-14.9	-12.7	-17.4
CONCH [ 21 ]	0.431	-00.8	-01.0	-04.6	-06.5	-06.8	-06.8	-06.7	-06.7	0.323	-08.9	-25.1	-18.4	-22.6	-26.4	-27.7	-34.6	-60.3	0.179	-01.1	-07.5	-09.8	-08.2	-11.5	-18.9	-10.3	-10.4
BiomedCLIP [ 35 ]	0.520	+04.1	+03.5	+03.8	+07.4	+11.3	+14.4	+15.1	+15.1	0.363	+15.7	+31.9	+34.9	+57.7	+63.2	+72.5	+75.0	+75.0	0.162	-00.5	+00.7	+00.3	-03.9	-05.4	-06.0	-08.7	-14.2

Table 3: Prompt ensembling gain (T3 Variant 3).

L_{n}

denotes an ensemble of

n

prompts drawn uniformly without replacement from the joint pool of nine clinically validated variants (five semantic-equivalence paraphrases and four length or reporting-style variants; Section 3.3 ), with the selected text embeddings averaged and re-normalized before inference. For

n<9

we report the mean over 100 random draws (seed 42);

n=9

is deterministic.

L_{1}

reports absolute accuracy (range 0–1);

L_{2}

–

L_{9}

report the relative change (%) with respect to

L_{1}

, with percentage signs omitted for compactness. Positive and negative changes are highlighted in green and red, respectively.

L_{1}

is the expected accuracy of a single randomly drawn prompt and therefore differs from the canonical-prompt accuracy in Table 2 ; using it as the reference avoids attributing to ensembling any gain that merely reflects a favorable hand-chosen baseline. Sampling intervals and interquartile ranges across draws are reported in Table 9 . Results are reported under the upper-bound aggregation protocol (Section 3.4 ); per-setting results and rank-stability analysis are in A.1.1 .

Appendix figures & tables42 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Top 1%	Top 5%	Top 10%	Top 50%	Mean
BiomedCLIP	0.214	0.220	0.252	0.176	0.218
CONCH	0.235	0.234	0.229	0.233	0.215
KEEP	0.277	0.300	0.314	0.270	0.268
MI-Zero	0.184	0.183	0.182	0.182	0.182
MUSK	0.285	0.281	0.277	0.261	0.284
PathGen-CLIP	0.248	0.241	0.242	0.245	0.245

Appendix

Table 1: Sensitivity of zero-shot classification performance to aggregation. Each entry is macro-F1 averaged over PANDA, UniToPatho, CAMELYON16, CAMELYON17, and TCGA-GBMLGG. The best aggregation within each model is bolded.

	Top 1%	Top 5%	Top 10%	Top 50%	Mean
Top 1%	1.000	0.933	0.883	0.933	0.967
Top 5%	0.933	1.000	0.950	0.967	0.933
Top 10%	0.883	0.950	1.000	0.883	0.933
Top 50%	0.933	0.967	0.883	1.000	0.933
Mean	0.967	0.933	0.933	0.933	1.000

Appendix

Table 2: Spearman rank correlation (

\rho

) between model rankings under different aggregation settings, computed from cross-dataset macro-F1 in Table 1 .

Model	Top 1%	Top 5%	Top 10%	Top 50%	Mean
BiomedCLIP	0.665	0.675	0.661	0.657	0.664
CONCH	0.684	0.685	0.689	0.664	0.672
KEEP	0.683	0.683	0.685	0.687	0.688
MI-Zero	0.642	0.638	0.638	0.635	0.640
MUSK	0.649	0.645	0.642	0.652	0.648
PathGen-CLIP	0.671	0.664	0.673	0.666	0.671

Appendix

Table 3: Sensitivity of Image-to-Text retrieval to aggregation. Each entry is nDCG averaged over CAMELYON16, CAMELYON17, PANDA, TCGA-GBMLGG, and UniToPatho. The best aggregation within each model is bolded.

Model	Top 1%	Top 5%	Top 10%	Top 50%	Mean
BiomedCLIP	0.460	0.462	0.463	0.416	0.422
CONCH	0.492	0.449	0.443	0.366	0.347
KEEP	0.633	0.613	0.589	0.599	0.579
MI-Zero	0.413	0.381	0.399	0.403	0.390
MUSK	0.518	0.481	0.462	0.458	0.505
PathGen-CLIP	0.602	0.583	0.612	0.595	0.608

Appendix

Table 4: Sensitivity of Text-to-Image retrieval to aggregation. Each entry is nDCG averaged over CAMELYON16, CAMELYON17, PANDA, TCGA-GBMLGG, and UniToPatho. The best aggregation within each model is bolded.

Model	Top 1%	Top 5%	Top 10%	Top 50%	Mean
BiomedCLIP	0.674	0.687	0.693	0.685	0.692
CONCH	0.715	0.749	0.764	0.774	0.785
KEEP	0.729	0.749	0.759	0.785	0.787
MI-Zero	0.683	0.680	0.682	0.692	0.686
MUSK	0.671	0.705	0.727	0.769	0.778
PathGen-CLIP	0.713	0.741	0.748	0.770	0.778

Appendix

Table 5: Sensitivity of Image-to-Image retrieval to aggregation. Each entry is nDCG averaged over CAMELYON16, CAMELYON17, PANDA, TCGA-GBMLGG, and UniToPatho. The best aggregation within each model is bolded.

	Top 1%	Top 5%	Top 10%	Top 50%	Mean
Top 1%	1.000	0.833	0.933	0.800	0.850
Top 5%	0.833	1.000	0.917	0.767	0.833
Top 10%	0.933	0.917	1.000	0.783	0.833
Top 50%	0.800	0.767	0.783	1.000	0.983
Mean	0.850	0.833	0.833	0.983	1.000

Appendix

Table 6: Spearman rank correlation (

\rho

) between model rankings under different aggregation settings for Image-to-Text retrieval, computed from cross-dataset nDCG.

	Top 1%	Top 5%	Top 10%	Top 50%	Mean
Top 1%	1.000	0.933	0.867	0.950	0.883
Top 5%	0.933	1.000	0.950	0.933	0.967
Top 10%	0.867	0.950	1.000	0.883	0.950
Top 50%	0.950	0.933	0.883	1.000	0.933
Mean	0.883	0.967	0.950	0.933	1.000

Appendix

Table 7: Spearman rank correlation (

\rho

) between model rankings under different aggregation settings for Text-to-Image retrieval, computed from cross-dataset nDCG.

	Top 1%	Top 5%	Top 10%	Top 50%	Mean
Top 1%	1.000	0.933	0.883	0.817	0.633
Top 5%	0.933	1.000	0.950	0.917	0.833
Top 10%	0.883	0.950	1.000	0.900	0.783
Top 50%	0.817	0.917	0.900	1.000	0.917
Mean	0.633	0.833	0.783	0.917	1.000

Appendix

Table 8: Spearman rank correlation (

\rho

) between model rankings under different aggregation settings for Image-to-Image retrieval, computed from cross-dataset nDCG.

Model	Top 1%	Top 5%	Top 10%	Top 50%	Mean
BiomedCLIP	0.338	0.332	0.335	0.334	0.334
CONCH	0.288	0.288	0.288	0.285	0.285
KEEP	0.480	0.482	0.484	0.484	0.482
MI-Zero	0.308	0.306	0.307	0.307	0.306
MUSK	0.347	0.335	0.332	0.321	0.300
PathGen-CLIP	0.394	0.407	0.416	0.422	0.424

Appendix

Table 9: Sensitivity of T3 Variant 1: semantic-equivalence paraphrases to aggregation. Each entry is Mean Accuracy averaged over PANDA, UniToPatho, CAMELYON16, CAMELYON17, and TCGA-GBMLGG. Mean Accuracy is first averaged across the five paraphrase variants (

v_{1}

–

v_{5}

) within each dataset. The best aggregation within each model is bolded.

	Top 1%	Top 5%	Top 10%	Top 50%	Mean
Top 1%	1.000	0.850	0.933	0.883	0.817
Top 5%	0.850	1.000	0.883	0.917	0.883
Top 10%	0.933	0.883	1.000	0.983	0.950
Top 50%	0.883	0.917	0.983	1.000	0.983
Mean	0.817	0.883	0.950	0.983	1.000

Appendix

Table 10: Spearman rank correlation (

\rho

) between model rankings under different aggregation settings for T3 Variant 1: semantic-equivalence paraphrases, computed from cross-dataset Mean Accuracy.

Model	Top 1%	Top 5%	Top 10%	Top 50%	Mean
BiomedCLIP	0.439	0.439	0.439	0.440	0.442
CONCH	0.338	0.344	0.345	0.340	0.332
KEEP	0.417	0.419	0.421	0.423	0.422
MI-Zero	0.295	0.297	0.296	0.299	0.298
MUSK	0.335	0.333	0.335	0.327	0.324
PathGen-CLIP	0.371	0.375	0.377	0.379	0.379

Appendix

Table 11: Sensitivity of paraphrase robustness T3 Variant 2: length and reporting-style variation to aggregation. Each entry is Mean Accuracy averaged over PANDA, UniToPatho, CAMELYON16, CAMELYON17, and TCGA-GBMLGG. Mean Accuracy is first averaged across short, medium, long, and clinical paraphrases within each dataset. The best aggregation within each model is bolded.

	Top 1%	Top 5%	Top 10%	Top 50%	Mean
Top 1%	1.000	1.000	1.000	1.000	1.000
Top 5%	1.000	1.000	1.000	1.000	1.000
Top 10%	1.000	1.000	1.000	1.000	1.000
Top 50%	1.000	1.000	1.000	1.000	1.000
Mean	1.000	1.000	1.000	1.000	1.000

Appendix

Table 12: Spearman rank correlation (

\rho

) between model rankings under different aggregation settings for T3 Variant 2: length and reporting-style variation, computed from cross-dataset Mean Accuracy.

Candidate set	Min. $\rho$	Max. $\rho$	Top- $K$ min. $\rho$	Top- $r$ max. $\rho$
Set 1: Open Vocabulary	0.983	1.000	0.983	1.000
Set 2: Label-Specific Mixture	0.967	1.000	0.967	1.000
Set 3: Cross-Organ	1.000	1.000	1.000	1.000
Set 4: All Datasets	1.000	1.000	1.000	1.000

Appendix

Table 15: Ranking stability across aggregation settings for Task 4, based on model rankings by cross-dataset Recall@1. The top-

r

columns exclude mean aggregation.

Model	Hybrid Top-1% (query-dependent)		Mean Pooling (query-independent)
Model	P.C.	PSS	P.C.	PSS
BiomedCLIP	0.004	-0.118	0.004	-0.124
CONCH	0.147	-0.092	0.153	-0.108
KEEP	0.295	0.142	0.308	0.113
MI-Zero	0.065	-0.018	0.070	-0.028
MUSK	0.128	0.063	0.155	-0.048

Appendix

Table 16: Prediction Consistency (P.C.) and Prompt Stability Score (PSS) under Hybrid Top-1% (query-dependent) versus mean pooling over all patches (query-independent), averaged across five datasets, for semantic-equivalence paraphrases (T3 Variant 1). Instability persists under the query-independent baseline, indicating that it is not an artifact of the text-conditioned aggregation mechanism.

Model	Hybrid Top-1% (query-dependent)		Mean Pooling (query-independent)
Model	P.C.	PSS	P.C.	PSS
BiomedCLIP	0.170	-0.185	0.179	-0.227
CONCH	0.026	-0.167	0.075	-0.165
KEEP	0.261	0.050	0.259	0.029
MI-Zero	0.107	-0.125	0.115	-0.141
MUSK	0.198	0.013	0.042	-0.100

Appendix

Table 17: Prediction Consistency (P.C.) and Prompt Stability Score (PSS) under Hybrid Top-1% (query-dependent) versus mean pooling over all patches (query-independent), averaged across five datasets, for length and reporting-style variation (T3 Variant 2).

Variation Type	Description
Synonym/Paraphrase	Semantically equivalent rewrites of the canonical class description
Length: Short	Concise, keyword-style prompt
Length: Medium	Moderately detailed prompt
Length: Long	Extended prompt with additional clinical context
Clinical Style	Formal clinical reporting style

Appendix

Table 18: Prompt variation types used in the T3 prompt pool.

Stage	Date	Description
Initial prompt set	Nov. 18, 2025	Initial ground-truth/zero-shot prompts were generated and shared.
Distractor revision	Dec. 8, 2025	A distractor-augmented prompt set was received; issues with the initial distractors were identified and revised.
Expert review	Apr. 9, 2026	The submitted prompts were reviewed and revised by the clinical expert(s), and the revised prompt set was returned.
Post-review verification	Apr. 29, 2026	The revised prompts were applied and tested, leading to identification of additional issues for correction.
Final revision	May 3, 2026	A subsequent revised version incorporating the requested changes was received and used as the final prompt set.

Appendix

Table 19: Prompt development and clinical review timeline.

Revision type	Original	Revised
Dysplasia terminology	low-grade cytologic atypia	low-grade cytologic dysplasia
	severe epithelial atypia	severe epithelial dysplasia
	marked cytologic atypia	marked cytologic dysplasia
	mild atypia (colorectal adenoma)	mild dysplasia
	high-grade intraepithelial neoplasia	high-grade epithelial dysplasia
	tubular colorectal adenoma with mild dysplasia	tubular colorectal adenoma with low-grade dysplasia

Appendix

Table 20: Representative entry-specific prompt revisions from the post-review verification round (Apr. 29, 2026). Categories group the listed revisions by rationale; review was conducted entry by entry.

Revision type	Original	Revised
Generic $\rightarrow$ specific entity	reactive tissue	tubular adenoma
	normal tissue	sessile serrated adenoma
	normal tissue	Barrett esophagus with low-grade dysplasia
	dysplasia	squamocolumnar mucosa with focal intestinal metaplasia
	benign tissue	squamocolumnar mucosa with focal intestinal metaplasia
	low-grade dysplasia	inflammatory polyp

Appendix

Table 21: Entry-specific prompt revisions from the final revision round (May 3, 2026). Distractor candidates judged implausible or non-discriminative were replaced with diagnostically plausible alternatives.

Fig. 1: Qualitative UMAP visualization of the joint slide–text embedding space on CAMELYON17. Points are colored by ground-truth diagnostic class; circles denote slide embeddings and stars denote canonical text embeddings. For each model, 36 slides per class were sampled from the subset with verified labels. UMAP was fit independently for each model using L2-normalized embeddings, cosine distance, $n_{\mathrm{neighbors}}=15$ , $\mathrm{min\_dist}=0.15$ , and seed 42. Coordinates are not comparable across panels. This figure illustrates the modality gap between image and text embeddings [ Liang et al., 2022 ] ; it is not evidence of class separability or of diagnostic-language grounding.

Models	CAMELYON16		CAMELYON17		PANDA		TCGA-GBMLGG		UniToPatho
Models	Align.	Gap	Align.	Gap	Align.	Gap	Align.	Gap	Align.	Gap
QuiltNet	0.2332	-0.0028	0.2395	-0.0167	0.2451	-0.0014	0.2718	0.0002	0.2756	0.0068
PLIP	0.2511	0.0011	0.2554	0.0058	0.2603	0.0043	0.2862	-0.0003	0.2685	0.0046
MI-Zero	-0.0044	-0.0022	-0.0232	-0.0055	-0.0500	0.0002	0.0391	-0.0022	0.0510	0.0011
CONCH	0.0831	-0.0074	0.0939	-0.0066	0.0926	-0.0058	0.0767	-0.0067	0.0941	-0.0069
BiomedCLIP	0.0051	0.0014	0.0138	0.0071	0.0111	0.0000	-0.0040	0.0000	-0.0233	-0.0055

Appendix

Table 1: Alignment score and similarity gap across five pathology datasets. Higher alignment scores indicate stronger image–text alignment, while similarity gap measures the separation between matched and unmatched pairs. The best result for each metric within each dataset is highlighted in bold.

Models	CAMELYON16				CAMELYON17				PANDA				TCGA-GBMLGG				UniToPatho
Models	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG
QuiltNet	0.538	0.874	0.680	0.718	0.595	0.872	0.712	0.742	0.661	0.893	0.761	0.786	0.917	0.973	0.943	0.949	0.468	0.806	0.612	0.640
PLIP	0.544	0.862	0.676	0.704	0.593	0.860	0.712	0.739	0.659	0.900	0.763	0.790	0.929	0.976	0.949	0.955	0.485	0.814	0.626	0.658
MI-Zero	0.475	0.879	0.640	0.682	0.511	0.868	0.656	0.694	0.319	0.729	0.495	0.533	0.867	0.950	0.906	0.914	0.494	0.819	0.636	0.666
CONCH	0.508	0.879	0.665	0.710	0.633	0.862	0.737	0.759	0.707	0.913	0.796	0.819	0.921	0.973	0.943	0.949	0.544	0.844	0.668	0.696
BiomedCLIP	0.491	0.882	0.653	0.696	0.505	0.856	0.662	0.701	0.332	0.737	0.508	0.545	0.846	0.952	0.894	0.905	0.447	0.823	0.607	0.648

Appendix

Table 2: Image-to-Image Cross-modal retrieval performance across five pathology datasets. We report Recall@1 (R@1), Recall@5 (R@5), mean reciprocal rank (MRR), and nDCG. The best result for each metric within each dataset is highlighted in bold.

Models	CAMELYON16			CAMELYON17			PANDA			TCGA-GBMLGG			UniToPatho
Models	R@1	MRR	nDCG	R@1	MRR	nDCG	R@1	MRR	nDCG	R@1	MRR	nDCG	R@1	MRR	nDCG
QuiltNet	0.500	0.728	0.798	0.397	0.618	0.713	0.186	0.421	0.559	0.467	0.699	0.775	0.194	0.430	0.567
PLIP	0.519	0.743	0.810	0.196	0.456	0.590	0.157	0.380	0.527	0.492	0.745	0.812	0.148	0.384	0.530
MI-Zero	0.563	0.781	0.838	0.192	0.385	0.527	0.194	0.404	0.543	0.501	0.731	0.801	0.122	0.354	0.506
CONCH	0.608	0.752	0.814	0.285	0.532	0.648	0.260	0.446	0.576	0.563	0.775	0.834	0.228	0.459	0.587
BiomedCLIP	0.509	0.752	0.817	0.303	0.568	0.676	0.114	0.322	0.479	0.524	0.762	0.824	0.198	0.448	0.582

Appendix

Table 3: Image-to-Text Cross-modal retrieval performance across five pathology datasets. We report Recall@1 (R@1), mean reciprocal rank (MRR), and nDCG. The best result for each metric within each dataset is highlighted in bold.

Models	CAMELYON16				CAMELYON17				PANDA				TCGA-GBMLGG				UniToPatho
Models	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG	R@1	R@5	MRR	nDCG
QuiltNet	0.625	1.000	0.781	0.803	0.438	0.813	0.533	0.601	0.250	0.542	0.326	0.384	0.625	0.750	0.688	0.695	0.250	0.667	0.388	0.460
PLIP	0.625	0.875	0.719	0.752	0.250	0.625	0.358	0.429	0.292	0.625	0.422	0.456	0.500	0.750	0.625	0.676	0.333	0.750	0.481	0.532
MI-Zero	0.500	0.500	0.500	0.500	0.250	0.625	0.385	0.438	0.208	0.500	0.291	0.347	0.500	0.500	0.500	0.500	0.208	0.458	0.300	0.330
CONCH	0.125	0.875	0.406	0.530	0.250	0.563	0.328	0.391	0.250	0.708	0.376	0.462	0.625	1.000	0.760	0.813	0.167	0.542	0.283	0.328
BiomedCLIP	0.750	1.000	0.800	0.828	0.188	0.688	0.359	0.443	0.292	0.542	0.375	0.386	0.500	0.875	0.598	0.665	0.208	0.250	0.208	0.205

Appendix

Table 4: Text-to-Image Cross-modal Retrieval performance across five pathology datasets. We report Recall@1 (R@1), Recall@5 (R@5), mean reciprocal rank (MRR), and nDCG. The best result for each metric within each dataset is highlighted in bold.

Model	CAMELYON16			CAMELYON17			PANDA			TCGA-GBMLGG			UniToPatho
Model	Acc.	P.C.	PSS	Acc.	P.C.	PSS	Acc.	P.C.	PSS	Acc.	P.C.	PSS	Acc.	P.C.	PSS
QuiltNet [ Ikezogwo et al., 2023 ]	0.511	0.000	-0.140	0.275	0.000	-0.126	0.179	0.013	-0.073	0.499	0.012	-0.181	0.236	0.000	-0.048
PLIP [ Huang et al., 2023 ]	0.513	0.010	-0.073	0.236	0.002	-0.068	0.198	0.000	-0.055	0.536	0.196	0.080	0.196	0.000	-0.087
MI-Zero [ Lu et al., 2023 ]	0.497	0.000	-0.111	0.255	0.341	0.381	0.166	0.000	-0.144	0.513	0.000	-0.142	0.178	0.000	-0.105
CONCH [ Lu et al., 2024 ]	0.498	0.724	-0.026	0.263	0.000	-0.139	0.198	0.000	-0.082	0.513	0.011	-0.149	0.160	0.017	-0.072
BiomedCLIP [ Zhang et al., 2024 ]	0.495	0.000	-0.184	0.246	0.000	-0.211	0.165	0.000	-0.137	0.492	0.025	-0.115	0.174	0.000	0.058

Appendix

Table 5: Accuracy (Acc), Prediction Consistency (P.C.) and Prompt Stability Score (PSS) under semantic-equivalence paraphrases (T3 Variant 1). Results are reported under the upper-bound aggregation protocol (Section 3.4 ); per-setting results and rank-stability analysis are in A.1.1 .

Models	CAMELYON16					CAMELYON17					PANDA					TCGA-GBMLGG					UniToPatho
Models	v1	v2	v3	v4	v5	v1	v2	v3	v4	v5	v1	v2	v3	v4	v5	v1	v2	v3	v4	v5	v1	v2	v3	v4	v5
QuiltNet	0.402	0.440	0.598	0.585	0.402	0.110	0.148	0.072	0.497	0.122	0.199	0.180	0.276	0.272	0.141	0.664	0.664	0.325	0.658	0.492	0.135	0.384	0.169	0.110	0.498
PLIP	0.619	0.406	0.521	0.579	0.486	0.345	0.188	0.186	0.383	0.150	0.279	0.254	0.301	0.295	0.267	0.445	0.521	0.525	0.542	0.532	0.430	0.494	0.502	0.477	0.144
MI-Zero	0.450	0.546	0.558	0.402	0.518	0.116	0.313	0.194	0.399	0.116	0.255	0.125	0.124	0.115	0.194	0.561	0.437	0.621	0.412	0.515	0.144	0.397	0.173	0.008	0.046
CONCH	0.420	0.402	0.397	0.440	0.402	0.142	0.090	0.122	0.485	0.505	0.132	0.157	0.278	0.117	0.135	0.547	0.454	0.476	0.509	0.535	0.084	0.131	0.089	0.190	0.089
BiomedCLIP	0.426	0.599	0.514	0.414	0.599	0.118	0.635	0.635	0.064	0.134	0.135	0.105	0.273	0.112	0.251	0.561	0.477	0.444	0.561	0.502	0.131	0.169	0.317	0.118	0.181

Appendix

Table 7: Per-variant accuracy under semantic-equivalence paraphrases (T3 Variant 1). We report accuracy under five semantically equivalent prompt variants, Acc-v1 to Acc-v5. The best result for each prompt variant within each dataset is highlighted in bold.

Models	CAMELYON16				CAMELYON17				PANDA				TCGA-GBMLGG				UniToPatho
Models	Short	Medium	Long	Clinical	Short	Medium	Long	Clinical	Short	Medium	Long	Clinical	Short	Medium	Long	Clinical	Short	Medium	Long	Clinical
QuiltNet	0.402	0.402	0.598	0.402	0.074	0.116	0.170	0.072	0.115	0.164	0.190	0.119	0.665	0.459	0.577	0.577	0.152	0.110	0.283	0.160
PLIP	0.609	0.599	0.516	0.602	0.375	0.184	0.086	0.323	0.272	0.268	0.251	0.258	0.538	0.534	0.436	0.475	0.490	0.122	0.485	0.346
MI-Zero	0.452	0.548	0.414	0.407	0.419	0.072	0.118	0.118	0.242	0.124	0.160	0.155	0.568	0.440	0.345	0.570	0.000	0.173	0.152	0.549
CONCH	0.598	0.405	0.400	0.558	0.118	0.172	0.673	0.455	0.229	0.260	0.118	0.231	0.463	0.569	0.504	0.591	0.089	0.101	0.144	0.409
BiomedCLIP	0.599	0.579	0.399	0.599	0.619	0.635	0.072	0.619	0.272	0.126	0.148	0.120	0.561	0.561	0.439	0.556	0.511	0.485	0.473	0.485

Appendix

Table 8: Per-variant accuracy under length and reporting-style variation (T3 Variant 2) across five pathology datasets. We report accuracy under four prompt styles: Short, Medium, Long, and Clinical. The best result for each prompt style within each dataset is highlighted in bold.

Model	CAMELYON16			CAMELYON17			PANDA			TCGA-GBMLGG			UniToPatho
	CI95-L	CI95-U	IQR	CI95-L	CI95-U	IQR	CI95-L	CI95-U	IQR	CI95-L	CI95-U	IQR	CI95-L	CI95-U	IQR
QuiltNet [ Ikezogwo et al., 2023 ]	0.3999	0.5346	0.0413	0.0708	0.2013	0.0359	0.1363	0.2572	0.0616	0.5509	0.6814	0.0482	0.1097	0.4412	0.1298
PLIP [ Huang et al., 2023 ]	0.3951	0.6096	0.1386	0.0913	0.3478	0.1288	0.2608	0.3084	0.0124	0.5274	0.6196	0.0382	0.2096	0.5038	0.1784
MI-Zero [ Lu et al., 2023 ]	0.4029	0.5349	0.0682	0.1047	0.2350	0.0253	0.1180	0.2219	0.0373	0.4108	0.6087	0.1056	0.0145	0.2152	0.0982
CONCH [ Lu et al., 2024 ]	0.3959	0.4756	0.0102	0.1041	0.5812	0.2186	0.1168	0.2455	0.0782	0.4619	0.5767	0.0586	0.0574	0.3534	0.0982
BiomedCLIP [ Zhang et al., 2024 ]	0.4593	0.6016	0.0823	0.2491	0.6353	0.2050	0.1165	0.2416	0.0586	0.4943	0.5614	0.0171	0.1305	0.4441	0.1924

Appendix

Table 9: Variability of mean accuracy under random prompt sampling. For each ensemble size

n\in\{1,\ldots,9\}

n

prompts are drawn without replacement from the pool of nine variants and their embeddings averaged; this is repeated 100 times (seed 42). We report the 2.5th and 97.5th percentiles (CI95-L, CI95-U) and the interquartile range of accuracy across these draws. These intervals describe sensitivity to which prompts are sampled, rather than statistical uncertainty in the underlying accuracy estimate. For each dataset, the smallest interval width (CI95-L

-

CI95-U) is shown in bold.

Model	CAMELYON16		CAMELYON17		PANDA		TCGA-GBMLGG		UniToPatho
Model	P.C.	PSS	P.C.	PSS	P.C.	PSS	P.C.	PSS	P.C.	PSS
QuiltNet	[0.000, 0.000]	[-0.151, -0.131]	[0.000, 0.000]	[-0.134, -0.118]	[0.011, 0.015]	[-0.076, -0.069]	[0.006, 0.019]	[-0.187, -0.174]	[0.000, 0.000]	[-0.070, -0.029]
PLIP	[0.003, 0.020]	[-0.089, -0.056]	[0.000, 0.006]	[-0.079, -0.058]	[0.000, 0.001]	[-0.059, -0.052]	[0.172, 0.218]	[0.059, 0.102]	[0.000, 0.000]	[-0.100, -0.075]
MI-Zero	[0.000, 0.000]	[-0.127, -0.095]	[0.303, 0.379]	[0.348, 0.414]	[0.000, 0.000]	[-0.145, -0.143]	[0.000, 0.000]	[-0.150, -0.135]	[0.000, 0.000]	[-0.131, -0.078]
CONCH	[0.681, 0.764]	[-0.049, -0.005]	[0.000, 0.000]	[-0.146, -0.133]	[0.000, 0.000]	[-0.085, -0.080]	[0.005, 0.017]	[-0.158, -0.142]	[0.004, 0.038]	[-0.092, -0.050]
BiomedCLIP	[0.000, 0.000]	[-0.193, -0.176]	[0.000, 0.000]	[-0.216, -0.207]	[0.000, 0.000]	[-0.138, -0.136]	[0.017, 0.035]	[-0.122, -0.107]	[0.000, 0.000]	[0.025, 0.086]

Appendix

Table 10: Slide-level bootstrap confidence intervals for Prediction Consistency (P.C.) and Prompt Stability Score (PSS) under semantic-equivalence paraphrases (T3 Variant 1). Intervals are 2.5th–97.5th percentiles over 1,000 bootstrap resamples of slides drawn with replacement (seed 42). Point estimates are those reported in Table 5 . Intervals shown in italics contain zero.

Model	CAMELYON16		CAMELYON17		PANDA		TCGA-GBMLGG		UniToPatho
Model	P.C.	PSS	P.C.	PSS	P.C.	PSS	P.C.	PSS	P.C.	PSS
QuiltNet	[0.000, 0.000]	[-0.333, -0.333]	[0.000, 0.000]	[-0.305, -0.290]	[0.000, 0.000]	[-0.205, -0.200]	[0.158, 0.204]	[-0.105, -0.055]	[0.000, 0.029]	[-0.137, -0.057]
PLIP	[0.386, 0.489]	[-0.051, 0.006]	[0.140, 0.202]	[0.076, 0.121]	[0.209, 0.225]	[-0.067, -0.060]	[0.034, 0.054]	[-0.116, -0.086]	[0.000, 0.000]	[-0.124, -0.077]
MI-Zero	[0.010, 0.040]	[-0.296, -0.249]	[0.313, 0.395]	[0.077, 0.145]	[0.000, 0.000]	[-0.127, -0.121]	[0.147, 0.186]	[-0.135, -0.109]	[0.000, 0.000]	[-0.282, -0.257]
CONCH	[0.000, 0.007]	[-0.301, -0.280]	[0.000, 0.000]	[-0.262, -0.250]	[0.000, 0.002]	[-0.114, -0.108]	[0.343, 0.394]	[0.019, 0.065]	[0.000, 0.000]	[-0.113, -0.076]
BiomedCLIP	[0.000, 0.000]	[-0.266, -0.253]	[0.000, 0.000]	[-0.322, -0.302]	[0.000, 0.000]	[-0.264, -0.259]	[0.000, 0.000]	[-0.269, -0.261]	[0.802, 0.894]	[0.046, 0.231]

Appendix

Table 11: Slide-level bootstrap confidence intervals for Prediction Consistency (P.C.) and Prompt Stability Score (PSS) under length and reporting-style variation (T3 Variant 2). Intervals are 2.5th–97.5th percentiles over 1,000 bootstrap resamples of slides drawn with replacement (seed 42). Point estimates are those reported in Table 6 . Intervals shown in italics contain zero.

CAMELYON16
Models	Mean	R@1	R@2	R@3	R@4	R@5	R@6	R@7	R@8	R@9	R@10
Chance	0.275	0.050	0.100	0.150	0.200	0.250	0.300	0.350	0.400	0.450	0.500
QuiltNet	0.602	0.000	0.191	0.352	0.538	0.666	0.756	0.842	0.862	0.892	0.922
PLIP	0.478	0.005	0.007	0.085	0.251	0.451	0.584	0.732	0.830	0.902	0.932
MI-Zero	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
CONCH	0.313	0.073	0.143	0.201	0.347	0.384	0.395	0.395	0.397	0.397	0.397

Appendix

Table 12: Recall performance for Set 1 – Open Vocabulary. We report mean recall and Recall@K across available cutoffs. The best result for each metric within each dataset is highlighted in bold. Chance denotes the expected performance of a uniformly random ranking over the same candidate pool.

CAMELYON16
Models	Mean	R@1	R@2	R@3	R@4	R@5
QuiltNet	0.253	0.000	0.010	0.395	0.402	0.460
PLIP	0.658	0.373	0.404	0.529	0.988	0.998
MI-Zero	0.328	0.000	0.179	0.222	0.502	0.735
CONCH	0.185	0.003	0.038	0.151	0.266	0.467
BiomedCLIP	0.384	0.000	0.123	0.599	0.599	0.599

Appendix

Table 13: Recall performance for Set 2 – Label-Specific Mixed Diagnostic Candidates. We report mean recall and Recall@K across available cutoffs. The best result for each metric within each dataset is highlighted in bold.

CAMELYON16
Models	Mean	R@1	R@2	R@3	R@4	R@5
QuiltNet	0.630	0.191	0.374	0.668	0.915	1.000
PLIP	0.819	0.421	0.769	0.925	0.980	0.998
MI-Zero	0.335	0.000	0.253	0.384	0.432	0.606
CONCH	0.445	0.364	0.395	0.420	0.482	0.563
BiomedCLIP	0.029	0.000	0.000	0.003	0.028	0.115

Appendix

Table 14: Recall performance for Set 3 – Cross-Organ. We report mean recall and Recall@K across available cutoffs. The best result for each metric within each dataset is highlighted in bold.

CAMELYON16
Models	Mean	R@1	R@2	R@3	R@4	R@5	R@6	R@7	R@8	R@9	R@10
QuiltNet	0.604	0.000	0.018	0.384	0.422	0.530	0.686	0.995	1.000	1.000	1.000
PLIP	0.839	0.368	0.494	0.652	0.895	0.990	0.998	0.998	0.998	0.998	1.000
MI-Zero	0.477	0.000	0.091	0.114	0.217	0.543	0.644	0.689	0.760	0.828	0.881
CONCH	0.410	0.000	0.000	0.020	0.111	0.231	0.447	0.711	0.784	0.869	0.922
BiomedCLIP	0.123	0.000	0.000	0.000	0.005	0.015	0.033	0.045	0.118	0.499	0.519

Appendix

Table 15: Recall performance for Set 4 – All Dataset Labels. We report mean recall and Recall@K across available cutoffs. The best result for each metric within each dataset is highlighted in bold.

Label	Prompt
PANDA
Grade 0	Benign prostate tissue with normal gland architecture, uniform small glands, smooth luminal contours, and no nuclear atypia. Stroma is non-reactive and orderly. No evidence of carcinoma.
Grade 1	Low-grade prostate cancer with well-formed glands. Tumor glands are still round, separate, and organized. Minimal infiltration into surrounding tissue. Mild nuclear atypia with recognizable glandular structures. No fused or cribriform patterns.
Grade 2	Predominantly well-formed glands (pattern 3) with a lesser component of poorly formed or fused small glands (pattern 4). Early architectural disorder, some poorly formed glands beginning to appear.
Grade 3	Predominantly poorly formed, fused, or cribriform glands (pattern 4) with a smaller component of well-formed glands (pattern 3). Architectural disorganization more prominent. Increased nuclear atypia.
Grade 4	High-grade prostate cancer composed almost entirely of fused glands, poorly formed glands, or cribriform structures. Marked architectural disruption and significant nuclear atypia. No well-formed glands remain.

Appendix

Table 1: Text prompts used for zero-shot classification across all datasets.

Label	Positive Prompts
PANDA
Grade 0	p1. Benign prostate tissue with normal gland architecture, uniform small glands, smooth luminal contours, and no nuclear atypia. Stroma is non-reactive and orderly. No evidence of carcinoma p2. benign prostate tissue with no evidence of carcinoma p3. non-neoplastic prostate glands with preserved architecture and no atypia p4. normal prostate glands without dysplasia or malignancy
Grade 1	p1. Low-grade prostate cancer with well-formed glands. Tumor glands are still round, separate, and organized. Minimal infiltration into surrounding tissue. Mild nuclear atypia with recognizable glandular structures. No fused or cribriform patterns. p2. ISUP grade 1 prostate adenocarcinoma with well-formed, separate glands p3. Gleason score 3+3=6 prostate adenocarcinoma composed of round, discrete glands p4. low-grade prostate adenocarcinoma with minimal architectural distortion and mild atypia
Grade 2	p1. Predominantly well-formed glands (pattern 3) with a lesser component of poorly formed or fused small glands (pattern 4). Early architectural disorder, some poorly formed glands beginning to appear. p2. ISUP grade 2 prostate adenocarcinoma with primary Gleason pattern 3 and secondary pattern 4 p3. Gleason score 3+4=7 prostate cancer with a minor component of poorly formed or fused glands p4. predominantly pattern 3 prostate adenocarcinoma with an admixed pattern 4 component
Grade 3	p1. Predominantly poorly formed, fused, or cribriform glands (pattern 4) with a lesser component of well-formed glands (pattern 3). Architectural disorganization more prominent. Increased nuclear atypia. p2. ISUP grade 3 prostate adenocarcinoma with primary Gleason pattern 4 and secondary pattern 3 p3. Gleason score 4+3=7 prostate cancer dominated by poorly formed or fused glands p4. high-intermediate grade prostate adenocarcinoma with extensive pattern 4 architecture
Grade 4	p1. High-grade prostate cancer composed almost entirely of fused glands, poorly formed glands, or cribriform structures. Marked architectural disruption and significant nuclear atypia. No well-formed glands remain. p2. ISUP grade 4 prostate adenocarcinoma composed almost entirely of Gleason pattern 4 p3. Gleason score 4+4=8 prostate cancer with fused and cribriform glands and no well-formed glands p4. high-grade prostate adenocarcinoma with diffuse cribriform and fused glandular growth

Appendix

Table 2: Positive prompt bank used for each dataset label. Negatives are drawn from all other classes within the same dataset plus cross-organ distractors from the remaining datasets.

Label	Prompts
PANDA
Grade 0	v1. Normal prostate tissue v2. Non-neoplastic prostatic glands v3. Histologically normal prostate core v4. Prostate tissue without carcinoma v5. Prostatic tissue within normal limits
Grade 1	v1. Well-formed low-grade prostate adenocarcinoma v2. Low-grade acinar carcinoma of prostate with individual discrete well-formed glands v3. Prostate carcinoma with well-formed glands v4. Well-differentiated prostate cancer v5. ISUP Grade 1 prostatic adenocarcinoma
Grade 2	v1. Predominantly well-formed prostate adenocarcinoma with focal poorly formed glands v2. Predominantly well-formed prostate adenocarcinoma with minor component of poorly formed glands v3. Acinar prostate cancer with focal emerging fused glands v4. ISUP Grade 2 prostatic adenocarcinoma v5. Prostate tumor with minor poorly formed glands
Grade 3	v1. Poorly formed gland-predominant prostate adenocarcinoma v2. Prostate carcinoma with predominantly fused and cribriform glands v3. Acinar prostate cancer with predominant poorly formed glands v4. ISUP Grade 3 prostatic adenocarcinoma v5. Prostate tumor with dominant poorly formed gland architecture
Grade 4	v1. High-grade prostate carcinoma with exclusively poorly formed glands v2. Prostatic adenocarcinoma composed of fused and cribriform glands only v3. Poorly formed malignant prostatic glands without lower-grade component v4. High-grade acinar carcinoma in prostate tissue without well-formed glands v5. High-grade gland-forming prostate malignancy with cribriform architecture only

Appendix

Table 3: Variant 1 prompts: five semantic-equivalence paraphrases per label across all datasets.

Label	Length	Prompt
PANDA
Grade 0	short	ISUP Grade 0 (Benign)
	medium	Benign prostatic tissue with preserved glandular architecture and bland cytology.
	long	Prostate core shows orderly non-neoplastic glands with smooth lumina, uniform epithelium, and no infiltrative or atypical features.
	clinical	Benign prostate tissue with normal gland architecture, uniform small glands, smooth luminal contours, and no nuclear atypia. Stroma is non-reactive and orderly. No evidence of carcinoma.
Grade 1	short	ISUP Grade 1

Appendix

Table 4: Variant 2 prompts: four length and reporting-style variations per label across all datasets.

Part I: Shared Candidate Pool
benign tissue; low-grade malignancy; intermediate-grade malignancy; high-grade malignancy; poorly differentiated carcinoma; well-differentiated carcinoma; moderately differentiated carcinoma; adenocarcinoma; metastatic carcinoma; in situ carcinoma; dysplasia; low-grade dysplasia; high-grade dysplasia; invasive carcinoma; neuroendocrine carcinoma; squamous cell carcinoma; small cell carcinoma; undifferentiated carcinoma; tubular adenoma; sessile serrated adenoma.
Part II: Dataset-specific Ground Truth Mapping
Dataset Label	Ground Truth Candidate
PANDA
Grade 0	benign tissue
Grade 1	low-grade malignancy

Appendix

Table 5: Set 1 (Open Vocabulary): shared 20-candidate pool and dataset-specific ground-truth mapping.

Label	Ground Truth	Candidate Pool
PANDA
Grade 0	benign prostate tissue	benign prostate tissue; ISUP Grade 1 prostate cancer; ISUP Grade 2 prostate cancer; normal colon tissue; benign colorectal mucosa; normal lymph node; benign breast tissue; normal liver; normal lung tissue; benign thyroid tissue
Grade 1	ISUP Grade 1 prostate cancer	ISUP Grade 1 prostate cancer; benign prostate tissue; ISUP Grade 2 prostate cancer; ISUP Grade 3 prostate cancer; low-grade tubular adenoma; low-grade glioma; tubular adenoma low-grade; lymph node micrometastasis; low-grade dysplasia; high grade prostatic intraepithelial neoplasia (HGPIN)
Grade 2	ISUP Grade 2 prostate cancer	ISUP Grade 2 prostate cancer; ISUP Grade 1 prostate cancer; ISUP Grade 3 prostate cancer; ISUP Grade 4 prostate cancer; tubular adenoma high-grade; tubular adenoma low-grade; moderately differentiated carcinoma; intermediate grade dysplasia; lymph node micrometastasis; lymph node macrometastasis
Grade 3	ISUP Grade 3 prostate cancer	ISUP Grade 3 prostate cancer; ISUP Grade 2 prostate cancer; ISUP Grade 4 prostate cancer; ISUP Grade 5 prostate cancer; tubular adenoma high-grade; tubulovillous adenoma high-grade; high-grade dysplasia; HGPIN; poorly differentiated carcinoma; lymph node macrometastasis
Grade 4	ISUP Grade 4 prostate cancer	ISUP Grade 4 prostate cancer; ISUP Grade 3 prostate cancer; ISUP Grade 5 prostate cancer; ISUP Grade 2 prostate cancer; tubulovillous adenoma high-grade; glioblastoma; high-grade dysplasia; well differentiated carcinoma; lymph node macrometastasis; HGPIN

Appendix

Table 6: Set 2 (Label-Specific Mixed Diagnostic Candidates): label-specific mixtures of same-organ diagnostic alternatives, cross-organ candidates, and broader pathological distractors. Each pool contains up to 10 entries and its composition varies by target label. A small number of pools contain a repeated string, so total and unique counts differ.

Label	Ground Truth	Label-specific Candidate(s)
PANDA
Shared candidate pool: glioblastoma; low-grade glioma; lymph node metastasis; normal lymph node; colon adenocarcinoma; normal colon tissue; breast carcinoma; normal lung tissue; benign prostate tissue
Grade 0	benign prostate tissue	benign prostate tissue
Grade 1	ISUP Grade 1 prostate cancer	ISUP Grade 1
Grade 2	ISUP Grade 2 prostate cancer	ISUP Grade 2
Grade 3	ISUP Grade 3 prostate cancer	ISUP Grade 3

Appendix

Table 7: Set 3 (Cross-Organ): dataset-level shared cross-organ candidate pools with label-specific candidates.

Part I: Shared Candidate Pool
benign prostate tissue; ISUP Grade 1 prostate cancer; ISUP Grade 2 prostate cancer; ISUP Grade 3 prostate cancer; ISUP Grade 4 prostate cancer; ISUP Grade 5 prostate cancer; normal colon tissue; hyperplastic polyp; tubular adenoma low-grade; tubular adenoma high-grade; tubulovillous adenoma low-grade; tubulovillous adenoma high-grade; normal lymph node; lymph node metastasis; lymph node isolated tumor cells; lymph node micrometastasis; lymph node macrometastasis; low-grade glioma; glioblastoma.
Part II: Dataset-specific Ground Truth Mapping
Label	Ground Truth
PANDA
Grade 0	benign prostate tissue
Grade 1	ISUP Grade 1 prostate cancer

Appendix

Table 8: Set 4 (All Dataset Labels): shared pool of all class labels across the benchmark: 20 entries, of which 19 are unique (CAMELYON16 and CAMELYON17 share the label normal lymph node ).

Explore similar work

Do Pathology Vision-Language Models Truly See Pathology?
Jul 23, 2026Chengyang Zhang, Wenchuan Zhang, Bo Li +10VLM Evaluation Computational Pathology
College of Computer Science, Sichuan University · Department of Pathology and Institute of Clinical Pathology, West China Hospital, Sichuan University · Department of Computer Science, School of Computing, National University of Singapore +2
PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?
Jul 30, 2026Zongyi Chen, Yu Liang, Jie Lin +1Computational Pathology Medical VQA
National Institute for Data Science in Health and Medicine, Xiamen University · Institute of Artificial Intelligence Xiamen University, Xiamen, China · Department of Computer Science School of Informatics Xiamen University, Xiamen, China
Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
Sep 1, 2026Yumi Lee, Harim Oh, Hyoryung Kim +52Computational Pathology Medical VLMs

Jul 23, 2026, cs.CVJ/K move · Enter open · S save

Do Pathology Vision-Language Models Truly See Pathology?

Chengyang Zhang, Wenchuan Zhang, Bo Li, Xinyu Liu+9

College of Computer Science, Sichuan University · Department of Pathology and Institute of Clinical Pathology, West China Hospital, Sichuan University · Department of Computer Science, School of Computing, National University of Singapore+2

VLM Evaluation Computational Pathology Medical VQA Vision-Language Grounding Medical VLMs

Open paper AskPDF arXiv

Pathology vision-language models (VLMs) have recently progressed rapidly and are commonly evaluated by answer accuracy on pathology VQA benchmarks. However, we dig into current evaluations and identify three overlooked issues: 1) Visual evidence is not always necessary. For instance, Gemini-3-Pro achieves 53.5% average accuracy across 5 VQA benchmarks without any visual input. 2) Domain training can improve accuracy without proportional gains in visual binding. Compared with Qwen2.5-VL-7B, Patho-R1-7B exhibits a 5.8-point lower multimodal gain and a 3.7-point lower attention IoU. 3) Entity-level attention is diffuse and weakly query-specific. On PathVG, attention maps remain highly correlated across different entity queries. These issues can lead to substantial misjudgments of pathology VLMs' actual multimodal capabilities. To this end, we present PathBind, a benchmark comprising 2,600 samples: PathBind-VQA with 1,500 questions across six dimensions, PathBind-PTA with 600 questions from a private pathology teaching atlas, and PathBind-Grounding with 500 expert-curated region-level samples. Each component undergoes task-specific automated filtering and expert review to reduce textual shortcuts and improve entity-region correspondence. We evaluate 18 representative VLMs on VQA samples of PathBind and five existing pathology VQA benchmarks, and further evaluate 10 VLMs on PathBind-Grounding and PathVG. Results show that current pathology VLMs still exhibit a substantial gap between answer-side performance and visual-semantic binding.

Abstract

Explore similar work

Do Pathology Vision-Language Models Truly See Pathology?

PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?

Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation