Organizations: University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · MIRACLE Lab · The Hong Kong University of Science and Technology (Guangzhou) · Shanghai Jiaotong University · Ant Group · Nantong University · Zhejiang University · Tianjin University · Durham University · Jinan University · Southeast University · The University of Nottingham Ningbo China · University of Electronic Science and Technology of China · University of Leicester
Analyzing whole-slide images (WSIs) requires combining evidence across tissue regions and spatial scales. Multi-agent systems offer a promising approach to this task. However, existing systems make limited use of molecular evidence, often select visually prominent but less representative regions while overlooking global context, and struggle to reuse experience from previous cases. To address these challenges, we present PathLoop, a self-evolving multi-agent framework that integrates omics data for evidence-grounded WSI analysis. Specifically, PathLoop coordinates visual and omics interpreters across slide, patch, and pixel levels, using molecular signals to guide region selection and combining local tissue details with global slide features. A judgment agent then checks the consistency of the collected evidence against Pathology-Wiki, a knowledge base built from 48 entries across 24 organ systems, and triggers re-examination when the evidence is inconsistent. To reuse past experience, a four-level memory stores reasoning trajectories, practical tips, skills, and tools to guide future cases, forming a loop of evidence collection, review, and memory update. We evaluate PathLoop on four TCGA cohorts with biomarker prediction, survival prediction, and subtype classification. The results demonstrate the effectiveness of PathLoop across these tasks and support its use for evidence-grounded WSI analysis.
Figures & tables
Figure 1: Comparison between existing WSI analysis paradigms and PathLoop. (A) WSI-only methods rely primarily on visually salient ROIs and may miss diagnostically informative but visually inconspicuous regions. (B) Existing WSI–omics methods fuse separately encoded visual and molecular features, without using omics to guide ROI selection. (C) PathLoop leverages omics-informed planning to select task-relevant ROIs and introduces a feedback loop from prediction to planning for iterative refinement.
Figure 2: The PathLoop framework comprises four alternating phases. The Planning Agent analyzes the input and formulates a plan by drawing on historical experience. The Execution Agent performs slide-, patch-, and pixel-level processing and integrates omics data. The Judgment Agent conducts a consistency review against Pathology-Wiki. Finally, the Persist Agent builds a four-level memory from past trajectories, which is retrieved to guide reasoning in future cases.
Figure 3: Overview of Pathology-Wiki. (a) Organ coverage of the knowledge base, spanning 24 organ systems and 48 subtype codes. Codes marked with † share content with another code, resulting in 43 distinct pages. (b) Word counts for major diagnostic sections in each page, showing the completeness and depth of the curated content. (c) Cross-organ differential-diagnosis DDx links among 43 pages, highlighting 16 page pairs that reference entities from different organs. (d) IHC profiles of the linked pages, summarizing positive and negative markers that support differential diagnosis.
Method
Modality
ER (AUC)
PR (AUC)
HER2 (AUC)
Subtype (AUC)
Risk (C-index)
▼ Conventional Methods
SNN
O
0.952 ± 0.021
0.885 ± 0.035
0.725 ± 0.045
0.958 ± 0.015
0.620 ± 0.045
SNNTrans
O
0.958 ± 0.018
0.912 ± 0.028
0.765 ± 0.038
0.970 ± 0.012
0.645 ± 0.038
ABMIL
W
0.885 ± 0.032
0.755 ± 0.045
0.655 ± 0.052
0.955 ± 0.022
0.590 ± 0.052
TransMIL
W
0.901 ± 0.028
0.763 ± 0.041
0.687 ± 0.048
0.972 ± 0.015
0.615 ± 0.048
MCAT
O+W
0.975 ± 0.022
0.935 ± 0.031
0.768 ± 0.042
0.975 ± 0.018
0.665 ± 0.042
Table 1: Comparison with SOTA methods on multi-task prediction on TCGA-BRCA. AUC for ER/PR/HER2, macro AUC for Subtype, and C-index for Risk. Modality denotes the input data: O for omics, W for WSI, and O+W for both. The best and second-best results are highlighted in bold and underlined , respectively.
Method
Alignment
NLP Metrics
CLIP-cos
BLEU-1
BLEU-4
METEOR
ROUGE-L
w/ Projector
0.66
0.53
0.14
0.17
0.19
w/o Projector
0.53
0.43
0.02
0.04
0.06
Quilt-LLaVA
0.485
0.47
0.03
0.07
0.06
HistGen
0.51
0.08
0.01
0.10
0.10
MDAgents
0.46
0.01
0.00
0.05
0.06
Table 2: Ablation study of the projector module and comparative experiments on language generation performance.
Method
BLCA
LUAD
BRCA
UCEC
Mean
SNN
0.541 ± 0.016
0.539 ± 0.069
0.620 ± 0.045
0.493 ± 0.069
0.548
SNNTrans
0.646 ± 0.043
0.634 ± 0.049
0.645 ± 0.038
0.632 ± 0.032
0.639
TransMIL
0.576 ± 0.065
0.615 ± 0.034
0.615 ± 0.048
0.549 ± 0.044
0.589
E2E-CONCH
0.653 ± 0.024
0.705 ± 0.029
0.660 ± 0.014
0.614 ± 0.076
0.658
MCAT
0.645 ± 0.053
0.636 ± 0.043
0.665 ± 0.042
0.634 ± 0.018
0.645
MOTCat
0.674 ± 0.024
0.674 ± 0.036
0.685 ± 0.035
0.667 ± 0.051
0.675
Table 3: Comparison of survival risk prediction (C-index) on four TCGA cohorts. The best and second-best results are highlighted in bold and underlined , respectively.
Figure 4: Kaplan-Meier Analysis of predicted high-risk and low-risk groups. Shaded areas refer to the confidence intervals.
Figure 5: Explainability analysis through visualization on case TCGA-C8-A1HK.
Module
Setting
C-index
Planning
w/o CLS token
0.704
Execution
w/o Projector
0.676
Execution
w/o Omics
0.642
Judgment
w/o Pathology-Wiki
0.648
Persist
w/o Memory
0.703
Table 4: Ablation study of the agent modules on TCGA-BRCA. The full model achieves a C-index of 0.735.
Setting
ER
PR
HER2
Subtype
Risk
Wours
0.964
0.958
0.854
0.983
0.735
Wnoise
0.939
0.897
0.578
0.975
0.685
Wdrop
0.935
0.861
0.770
0.900
0.725
Wshuffle
0.912
0.897
0.667
0.989
0.684
Wrandom
0.898
0.764
0.797
0.893
0.686
K=8
0.577
0.745
0.523
0.659
0.563
Table 5: Ablation study of different settings on TCGA-BRCA.
LogReg C
ER
PR
HER2
Subtype
Risk
0.1
0.964
0.958
0.854
0.983
0.735
0.5
0.954
0.958
0.804
0.931
0.691
1.0
0.881
0.958
0.803
0.992
0.698
Table 6: Sensitivity analysis of logistic regression.
Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating multi-scale evidence. However, most existing pathology benchmarks evaluate models on pre-cropped patches or pre-extracted slide features, leaving their ability to acquire evidence directly from gigapixel WSIs largely untested. We introduce PathAgentBench, a benchmark for evaluating evidence-seeking vision-language models (VLMs) across four complementary capabilities: image-to-text matching for evidence interpretation, text-to-image retrieval for evidence verification, diagnostic-region localization for evidence acquisition, and multi-scale reasoning for evidence integration. The benchmark is organized as a diagnostic tree that links nested regions across magnifications with scale-specific findings and path-level diagnoses. It contains 1,822 TCGA WSIs and 17,135 diagnostic paths annotated by ten board-certified pathologists. An additional private cohort of 190 breast cancer WSIs with detailed annotations is used to evaluate autonomous whole-slide exploration. We evaluate 20 general-purpose, medical, and pathology-specialized models. Leading open-weight models achieve over 93% accuracy in multi-scale reasoning and over 50% accuracy in both cross-modal matching tasks. In contrast, diagnostic-region localization remains challenging: the best text-guided mean intersection-over-union is below 0.09, underperforming a simple center-based heuristic. During autonomous exploration, the unconditional hit rate decreases from 0.522 at low magnification to 0.185 at intermediate magnification and 0.020 at high magnification. These results reveal a pronounced gap between reasoning over curated evidence and acquiring that evidence directly from WSIs. PathAgentBench provides a unified framework for measuring and improving evidence-seeking pathology models.
Dankai Liao, Tianyi Zhang, Yufeng Wu +6
Department of Biomedical Engineering, National University of Singapore, Singapore 117417, Singapore
Whole-slide image (WSI) reasoning requires an agent to sequentially acquire visual evidence before answering a diagnostic question. Existing training-free agentic frameworks formulate this process as iterative patch retrieval based on semantic relevance to the question. However, semantic relevance does not necessarily imply diagnostic informativeness in computational pathology, where competing diagnoses often exhibit similar and overlapping morphological patterns, making many patches semantically relevant yet diagnostically non-discriminative. Consequently, relevance-based retrieval may acquire redundant observations and leave diagnostic uncertainty unresolved. We propose BEACON, a plug-and-play agentic framework that reformulates WSI reasoning as a Bayesian evidence acquisition problem. BEACON maintains a probabilistic belief over competing diagnostic hypotheses and sequentially acquires patches by maximizing expected information gain (EIG) to reduce diagnostic uncertainty. An evidence controller then determines whether to answer, acquire additional evidence, or perform higher-resolution inspection. Built entirely from off-the-shelf foundation models, BEACON requires no additional training or fine-tuning. Extensive zero-shot experiments across five WSI-VQA benchmarks demonstrate that BEACON achieves the strongest overall performance among training-free agentic frameworks while substantially improving evidence acquisition efficiency, establishing Bayesian evidence acquisition as a principled paradigm for uncertainty-aware agentic WSI reasoning. The code is available at https://github.com/bryanwong17/BEACON
Bryan Wong, Xun Xu, Huazhu Fu +2
1Korea Advanced Institute of Science and Technology (KAIST) · Institute of Advanced Intelligence and Computing (IAIC), A*STAR · Centre for Frontier AI Research (CFAR), A*STAR
Whole-slide images (WSIs) are challenging for vision-language reasoning because diagnostically relevant morphology is sparse, heterogeneous, and distributed across gigapixel-scale images and multiple spatial resolutions. Existing WSI models and pathology agents can aggregate slide features or actively acquire evidence, but the information retained after exploration is often difficult to access semantically while preserving its connection to the original visual evidence. We introduce SlideBank, a training-free framework that represents each WSI as a persistent, concept-indexed, and spatially grounded evidence bank. SlideBank performs question-independent coarse-to-fine exploration to identify informative regions and multi-scale views, converts them into explicit morphological observations, and grounds pathology signals to their supporting patches and WSI coordinates. At inference time, questions are routed to relevant signals and evidence scales, and the linked global, regional, and patch evidence is integrated through confidence-based cross-level consensus. Experiments on WSI-VQA and SlideBench-BCNB show that with Patho-R1, SlideBank reaches 52.77% on WSI-VQA and with Quilt-LLaVA, it reaches 50.92% average accuracy on SlideBench-BCNB, while structured signal-guided retrieval consistently outperforms random evidence sampling. Reusing the same bank across repeated queries further achieves over 99% rephrasing consistency and substantially reduces amortized inference cost through persistent evidence reuse.
Beidi Zhao, Gexin Huang, Ciro Zhang +6
University of British Columbia · Vector Institute · Harvard University +3