CHARTER: Auditing Reference Substitution in Hierarchical Compact-Evidence Evaluation for Computational Pathology

Hyun Do Jung; Jungwon Choi; Soojung Choi; Yujin Oh; Hwiyoung Kim

cs.CVOct 6, 2026

CHARTER: Auditing Reference Substitution in Hierarchical Compact-Evidence Evaluation for Computational Pathology

Authors: Hyun Do Jung, Jungwon Choi, Soojung Choi, Yujin Oh, Hwiyoung Kim

Organizations: Yonsei University · KAIST · Hallym University Chuncheon Sacred Heart Hospital

Abstract

In digital pathology, compact evidence is often used to explain or audit predictions made by whole-slide image multiple instance learning models. In hierarchical compact-evidence pipelines, candidate filtering introduces a strategy-specific candidate-conditioned prediction alongside the original full-bag prediction. If the evaluation reference changes while the intended target remains the original full-bag prediction, however, not only can the measured fidelity of the same compact evidence change, but comparisons between competing candidate strategies can also change. To make this dependence explicit, we introduce CHARTER, a reference-aware evaluation charter that asks researchers to DECLARE the intended target and reference, QUANTIFY candidate-induced prediction shift, and AUDIT the stability of comparative conclusions. Across the 15 comparisons in our main five-seed Random-K audit, 4 showed determinate reversals; in a matched native-ranking stress test, the ACMIL comparison changed from REVERSED to PRESERVED. CHARTER turns otherwise implicit candidate-filtering and reference choices into an auditable evaluation specification, helping distinguish genuine preservation of the intended prediction from apparent gains induced by changing the prediction being explained.

Figures & tables

Figure 1: Reference substitution in hierarchical compact-evidence evaluation and CHARTER. The same frozen MIL predictor is applied to the full bag $X$ , candidate pool $C$ , and compact subset $S$ , producing $q_{0}$ , $q_{C}$ , and $p$ . Candidate filtering can move the prediction reference before compact-subset fidelity is assessed. CHARTER makes the intended target, reference shift, and comparative stability explicit.

		Candidate-specific quantities				Pairwise outcome
Task	MILs	Candidate	Prediction shift $\delta_{C}$ (min–max)	Fidelity effect $B_{g}$ (min–max)	Sig. $B_{g}$ MILs	R / P / I
LUAD-TP53	3	$C_{\mathrm{L2}}$	$0.1585$ – $0.2276$	$0.1612$ – $0.4537$	3/3	2 / 0 / 1
LUAD-TP53	3	$C_{\mathrm{RAND}}$	$0.0207$ – $0.0287$	$0.006$ – $0.0179$	0/3	2 / 0 / 1
CRC-MSI	3	$C_{\mathrm{L2}}$	$0.0124$ – $0.018$	$0.0123$ – $0.037$	1/3	0 / 1 / 2
CRC-MSI	3	$C_{\mathrm{RAND}}$	$0.0066$ – $0.0157$	$0.0000$ – $0.0173$	1/3	0 / 1 / 2
Clean NSCLC	9	$C_{\mathrm{L2}}$	$0.0179$ – $0.0636$	$0.0232$ – $0.2428$	9/9	2 / 1 / 6

Table 1: Reference-substitution regimes under Random-

K

. Candidate-specific effects and pairwise outcomes under the fixed tolerance rule for

C_{\mathrm{L2}}

versus

C_{\mathrm{RAND}}

Figure 2: Reference substitution can change comparative conclusions, with outcomes depending on the ranking protocol. A , candidate gaps under the original and candidate-conditioned references; error bars denote 95% confidence intervals. Reversed / Preserved outcomes require a determinate gap under both references (95% CI excluding zero and $|\Delta|>\tau=0.05$ ); otherwise the comparison is Ranking-Indeterminate . B , matched comparison of controlled Random- $K$ and the tested predictor-derived native-ranking signals.

Figure 3: Breadth of the reference-substitution effect across tasks, MIL architectures, and pathology encoders. The broader audits contain 70 validated estimates among 108 planned task–MIL cells and 24 validated estimates among 60 planned encoder cells. A , the complete common-coverage subset of 10 tasks $\times$ 3 MIL architectures (ABMIL, TransMIL, and ACMIL; 30 cells). B , all eight task–MIL settings directly matched between UNI2-h and Phikon-v2 (16 estimates). Fill encodes the signed direct pairwise reference effect $R=\Delta_{\mathrm{cond}}-\Delta_{\mathrm{orig}}$ on a shared scale; R/P denote Reversed / Preserved , and cells without a letter are Ranking-Indeterminate . Displayed subsets are defined by common coverage or encoder matching rather than by the observed outcome.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Unit	Evaluation $N$	Positive class
Clean NSCLC	Patient	95 patients (98 slides)	LUSC: 48/95 (50.5%)
LUAD-TP53	Patient	67 of 69 validation patients	TP53 mutant: 34/67 (50.7%)
CRC-MSI	Patient	81	MSI-H: 11/81 (13.6%)
LUAD-EGFR	Patient	46	EGFR mutant: 6/46 (13.0%)
LUAD-KRAS	Patient	46	KRAS mutant: 14/46 (30.4%)
LUAD-STK11	Patient	46	STK11 mutant: 7/46 (15.2%)

Appendix

Table 2: Evaluation cohorts, analysis units, and class composition. Evaluation

N

refers to the validation set used for the reported fidelity analyses. For PANDA and CAMELYON+, no validated estimate is included in the reported analyses, so

N

denotes the planned validation set.

Figure 4: Full breadth-audit inventory underlying the scoped main-text Figure 3. Panel A shows the full task–MIL inventory for the common- $C_{\mathrm{L2}}$ audit, and Panel B shows the full encoder-breadth inventory over the planned anchor task–MIL settings. Colored cells denote validated estimates included in the reported analyses, with fill indicating direct $R=\Delta_{\mathrm{cond}}-\Delta_{\mathrm{orig}}$ and letters indicating the categorical pairwise outcome. Grey cells denote configurations without a validated estimate. The main-text Figure 3 presents complete common-coverage and encoder-matched subsets for compact comparison.

Explore similar work

From Patches to Patients: A study of the tile-to-slide performance transferability in Digital Pathology
Jun 9, 2026Sofiène Boutaj, Leo Fillioux, Maria Vakalopoulou +2Wireless Foundation Models Multiple Instance Learning
Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, IHU PRISM, Cancer Data Science Unit, France · Université Paris-Saclay, CentraleSupélec, MICS Laboratory, France
Reliable Benchmarking of Artifact Detection in Computational Pathology: A Reproducibility and Uncertainty Analysis
Aug 31, 2026Konstantinos Moutselos, Ilias MaglogiannisWhole-Slide Images Computational Pathology
Department of Digital Systems, University of Piraeus, Piraeus, Greece
HERALD: Counterfactual Audits and Minimal Repairs for Proof-of-Retrieval Rewards
Aug 6, 2026Zhuowen Liu, Bohan Cui, YinShang Guo +2Citations Model Auditing
The Chinese University of Hong Kong · Fudan University · Nanjing University +2

Jun 9, 2026, cs.CVJ/K move · Enter open · S save

From Patches to Patients: A study of the tile-to-slide performance transferability in Digital Pathology

Sofiène Boutaj, Leo Fillioux, Maria Vakalopoulou, Stergios Christodoulidis+1

Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, IHU PRISM, Cancer Data Science Unit, France · Université Paris-Saclay, CentraleSupélec, MICS Laboratory, France

Wireless Foundation Models Multiple Instance Learning Cohort Sentence Encoders Patients

Open paper AskPDF arXiv

Foundation Models (FMs) have recently redefined the state-of-the-art in histopathology by providing robust representations for whole-slide image (WSI) analysis. However, selecting the optimal foundation model (FM) for a specific clinical cohort currently requires multiple preprocessing steps, followed by computationally expensive feature extraction and the training of a Multiple Instance Learning (MIL) aggregator for every model. In this work, we investigate whether efficient tile-level linear probing can serve as a reliable proxy for slide-level performance, reducing the need to run full slide-level pipelines for every candidate encoder. We benchmark 19 state-of-the-art FMs on 42 slide-level and 16 tile-level tasks, comparing tile probing metrics against slide-level outcomes using ABMIL and Mean Pooling aggregations. We observe a high correlation between tile and slide performance across varying task difficulties, indicating that encoder representation quality is the primary determinant of WSI success. Sensitivity analyses show that transferability is stable across models and is more influenced by cohort sizes and numbers of tiles per slide than by average task difficulty. We also measure the agreement in best performing models between tile and slide-level tasks, showing tile benchmarks reliably shortlist strong candidates. Overall, our study indicates that tile-level benchmarking provides an efficient and practical first step for narrowing down candidate models, while slide-level evaluation remains essential for final validation on clinical tasks.

Abstract

Explore similar work

From Patches to Patients: A study of the tile-to-slide performance transferability in Digital Pathology

Reliable Benchmarking of Artifact Detection in Computational Pathology: A Reproducibility and Uncertainty Analysis

HERALD: Counterfactual Audits and Minimal Repairs for Proof-of-Retrieval Rewards