PathPilot: An Adaptive Question-Conditioned Multi-Magnification Navigation Agent for Whole-Slide Image Analysis

Zhengyang Xu; Han Li; Jingsong Liu; Ayako Ura; Shihui Zu; Shaohua Kevin Zhou; Nassir Navab; Peter Schüffler

cs.CVOct 12, 2026

PathPilot: An Adaptive Question-Conditioned Multi-Magnification Navigation Agent for Whole-Slide Image Analysis

Authors: Zhengyang Xu, Han Li, Jingsong Liu, Ayako Ura, Shihui Zu, Shaohua Kevin Zhou, Nassir Navab, Peter Schüffler

Organizations: Institute of Pathology, Technical University of Munich, Germany · Munich Center for Machine Learning, Munich, Germany · Chair of Computer Aided Medical Procedures, Technical University of Munich, Germany · Department of Human Pathology, Juntendo University Graduate School of Medicine, Tokyo, Japan ·

Abstract

The diagnostic capability of WSI agents relies on effective slide navigation to identify diagnostically relevant regions and efficiently integrate evidence across the slide. While in clinical practice, pathologists dynamically examine slides across multiple scales, integrating global and cellular evidence in a sequential manner, existing AI navigation approaches mostly operate at a single fixed magnification or rely on predefined magnification traversal. This mismatch prevents existing methods from modeling cross-magnification interactions and adaptive magnification selection inherent to real diagnostic workflows. We propose PathPilot, a clinically consistent multi-magnification WSI navigation agent that explicitly models cross-magnification localization and evidence-adaptive exploration. Specifically, we introduce a CrossMagLoc that learns magnification-specific diagnostic localization from pathologist viewing behavior and integrates information from adjacent magnifications to identify the diagnostically most relevant region within the current ROI, thereby providing a diagnostically informed starting point for subsequent question-conditioned exploration. We further introduce an EviRegionSearch that starts question-conditioned exploration from this region, progressively accumulating question-relevant evidence and adaptively determining the next navigation action according to the remaining evidence requirement. Their iterative interaction enables PathPilot to sequentially explore across ROIs and magnifications until sufficient evidence is acquired. Extensive QA evaluations across five cohorts demonstrate consistent superiority over competing SOTA methods on inter/external cohorts, respectively, validating question-conditioned evidence acquisition across regions and magnifications that mirrors the sequential, knowledge-driven navigation of pathologists.

Figures & tables

Fig. 1: Overview of PathPilot for adaptive WSI exploration. For each navigation step (e.g., Step $t$ ), CrossMagLoc identifies the diagnostically most relevant region within the current ROI, while EviRegionSearch accumulates question-relevant evidence and adaptively selects the next navigation action (e.g., Step $t+1$ ). Their iterative interaction enables sequential movement across ROIs and magnifications until evidence is sufficient (e.g., Step $t+N$ ), where Stop is selected and the final answer is generated.

Fig. 2: Overview of PathPilot and its two complementary navigation components. Given a WSI and pathology question, PathPilot starts from the thumbnail and iteratively alternates between CrossMagLoc and EviRegionSearch until sufficient evidence has been accumulated for answering the question. CrossMagLoc performs question-agnostic localization of diagnostically relevant candidate ROIs at each magnification by combining magnification-specific representations with spatially aligned information from adjacent scales. EviRegionSearch performs question-conditioned exploration over the ROI identified by CrossMagLoc, integrates morphological observations from the selected ROI with the accumulated evidence, identifies the remaining evidence requirement, and adaptively determines the next navigation action (Move, Zoom In, Zoom Out, or Stop) to select the next region.

Fig. 3: Overview of the datasets and evaluation cohorts used in this study. Top: Three navigation datasets provide pathologist viewing supervision for CrossMagLoc, including the Skin Eye-Tracking dataset and the TUM Liver and TUM Uterine Viewpoint datasets. The Skin dataset additionally provides pixel-level tumor annotations for region-level localization evaluation. Bottom: Five open-ended QA cohorts are used for end-to-end PathPilot evaluation, comprising the internal TUM Liver and TUM-Uteria cohorts and the external TUM Skin, WSI-Bench TCGA-SKCM, and WSI-Bench TCGA-LIHC cohorts. For the TUM cohorts, the circular plots summarize the predefined H&E-visible findings grouped into tissue-level, architectural, and cellular categories; for the WSI-Bench cohorts, they summarize the distributions of question categories.

Fig. 4: QA performance of PathPilot across internal and external validation cohorts. Comparisons include pathology-specialized thumbnail-based VLMs (PathGen-LLaVA and Patho-R1-7B), patch-based WSI VLMs (WSI-LLaVA and ANTONI-Alpha), and pathology agents (PathAgent and PathNavigate). The left radar plot summarizes average performance across all questions on two internal cohorts (TUM-Uteria and TUM Liver) and three external cohorts (TUM Skin, WSI-Bench TCGA-SKCM, and WSI-Bench TCGA-LIHC). The four right-hand plots further assess tissue-level, architectural-level, and cellular-level QA on the three TUM cohorts, and morphology-focused QA on the two WSI-Bench cohorts, distinguishing overall QA performance from performance on specific pathological assessment tasks. Each cohort is evaluated using GACC, PBR, and $1-\mathrm{IAR}$ , with higher values indicating better performance on all axes. Colored outer arcs identify cohorts; solid and dashed arcs denote internal and external validation, respectively. The plots summarize the quantitative results in Tables 2 – 5 .

Fig. 5: Pathologist-in-the-loop interpretation of CrossMagLoc across five magnifications (thumbnail, 1.25 $\times$ , 2.5 $\times$ , 5 $\times$ , and 10 $\times$ ; top to bottom). The columns show the pathology images (column 1), CrossMagLoc predictions (column 2), test-set navigation annotations (column 3), and independent annotations with morphological explanations from pathologists trained in Europe (column 4) and Asia (column 5). Across magnifications, CrossMagLoc consistently prioritizes diagnostically relevant regions identified by the pathologists while deprioritizing non-primary or artifact-limited areas. At 2.5 $\times$ , it assigns higher attention to well-preserved tumor regions than to the crush-artifact region, while at higher magnifications it continues to emphasize morphologically informative tumor regions. Overall, its localization broadly agrees with pathologist viewing behavior while additionally highlighting clinically relevant regions identified during independent review.

Metric	Expected	WSI-Bench TCGA-SKCM	WSI-Bench TCGA-LIHC	Pooled
Variant-level judge scores
Paraphrase ( $S_{P}$ )	High	0.957 [0.923, 0.991]	0.903 [0.855, 0.952]	0.932 [0.900, 0.959]
Omission ( $S_{O}$ )	$\approx S_{P}$	0.829 [0.769, 0.886]	0.845 [0.787, 0.894]	0.836 [0.797, 0.876]
Attribute error ( $S_{A}$ )	Lower	0.611 [0.560, 0.654]	0.348 [0.252, 0.439]	0.488 [0.427, 0.545]
Core contradiction ( $S_{C}$ )	Lowest	0.000 [0.000, 0.000]	0.000 [0.000, 0.000]	0.000 [0.000, 0.000]
Overall ordered sensitivity

Table 6: Validation of the LLM-based evaluator on the morphology subsets of WSI-Bench TCGA-SKCM and TCGA-LIHC using four controlled answer variants: paraphrase (P), omission (O), attribute error (A), and core contradiction (C). The first block reports the evaluator scores for the four variants, the second block evaluates whether the expected ordering (

P\geq O\geq A\geq C

) is preserved, and the final block reports score differences between adjacent variants. Scores are reported as mean [95% CI]. Overall, paraphrasing has little effect on the score, omission causes only a small decrease, factual attribute errors lead to a clear score reduction, and core contradictions are consistently assigned a score of 0.

Fig. 6: Quantitative evaluation of CrossMagLoc navigation performance. The radar plot summarizes pathologist-prioritization alignment ( $\rho^{N}$ , $AP_{5}^{N}$ , and $1-JS^{N}$ ) and tumor-region precision/recall at the thumbnail, 1.25 $\times$ , and 2.5 $\times$ magnifications, with PathFinder [ 14 ] and PathAttFormer [ 6 ] as navigation baselines and pathologist navigation as a clinical reference for tumor-region alignment. CrossMagLoc achieves the strongest agreement with pathologist prioritization while maintaining high tumor recall across magnifications, indicating a similar balance between diagnostic focus and broad tumor coverage.

Fig. 7: Visualization of a representative question-conditioned navigation workflow. CrossMagLoc-based ROI localization and EviRegionSearch-based evidence reasoning are iteratively coupled to support adaptive exploration across regions and magnifications, evidence requirement reasoning, and final answer generation once sufficient evidence is acquired.

Whole-Slide Image Analysis

Whole-slide image analysis uses AI to interpret gigapixel pathology slides for diagnosis, prognosis, tissue mapping, and report generation.

See all 214 papers

MMNavAgent: Multi-Magnification WSI Navigation Agent for Clinically Consistent Whole-Slide Analysis
Mar 2, 2026Zhengyang Xu, Han Li, Jingsong Liu +10Whole-Slide Image Analysis Computational Pathology
Institute of Pathology, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Computer Aided Medical Procedures (CAMP), TU Munich, Munich, Germany +7
PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA
May 22, 2026Chunze Yang, Qidong Liu, Wenjie Zhao +10Healthcare Medical VQA
School of Comp. Science & Technology, Xi’an Jiaotong University · Tencent Jarvis Lab · University of Cambridge
Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception
Aug 9, 2026Jingyun Chen, Fengchun Liu, Linghan Cai +5Active Perception Medical VQA
College of Computer Science and Technology, Harbin Institute of Technology, Shenzhen · School of Computing and Data Science, The University of Hong Kong

Mar 2, 2026J/K move · Enter open · S save

MMNavAgent: Multi-Magnification WSI Navigation Agent for Clinically Consistent Whole-Slide Analysis

Zhengyang Xu, Han Li, Jingsong Liu, Linrui Xie+9

Institute of Pathology, Technical University of Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Computer Aided Medical Procedures (CAMP), TU Munich, Munich, Germany+7

Whole-Slide Image Analysis Computational Pathology

Open paper AskPDF arXiv

Recent AI navigation approaches aim to improve Whole-Slide Image (WSI) diagnosis by modeling spatial exploration and selecting diagnostically relevant regions, yet most operate at a single fixed magnification or rely on predefined magnification traversal. In clinical practice, pathologists examine slides across multiple magnifications and selectively inspect only necessary scales, dynamically integrating global and cellular evidence in a sequential manner. This mismatch prevents existing methods from modeling cross-magnification interactions and adaptive magnification selection inherent to real diagnostic workflows. To these, we propose a clinically consistent Multi-Magnification WSI Navigation Agent (MMNavAgent) that explicitly models multi magnification interaction and adaptive magnification selection. Specifically, we introduce a Cross-Magnification navigation Tool (CMT) that aggregates contextual information from adjacent magnifications to enhance discriminative representations along the navigation path. We further introduce a Magnification Selection Tool (MST) that leverages memory-driven reasoning within the agent framework to enable interactive and adaptive magnification selection, mimicking the sequential decision process of pathologists. Extensive experiments on a public dataset demonstrate improved diagnostic performance, with 1.45% gain of AUC and 2.93% gain of BACC over a non-agent baseline. Code will be public upon acceptance.

Abstract

Whole-Slide Image Analysis

Explore similar work

MMNavAgent: Multi-Magnification WSI Navigation Agent for Clinically Consistent Whole-Slide Analysis

PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA

Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception