PlanPath: Exploration-Guided Orchestration for Patient-Level Pathology Reasoning

Jiusong Ge; Mingzhao He; Qidong Liu; Chunbao Wang; Chunze Yang; Di Zhang; Jiashuai Liu; Mireia Crispin-Ortuzar

cs.CVOct 13, 2026

PlanPath: Exploration-Guided Orchestration for Patient-Level Pathology Reasoning

Authors: Jiusong Ge, Mingzhao He, Qidong Liu, Chunbao Wang, Chunze Yang, Di Zhang, Jiashuai Liu, Mireia Crispin-Ortuzar,

Organizations: Xi’an Jiaotong University · University of Cambridge · A*STAR · Chinese Academy of Medical Sciences

Abstract

A patient's pathology case spans tens of slides from distinct specimens and anatomical sites, each carrying non-redundant, site-specific evidence. Forming a patient-level conclusion therefore requires locating the slides relevant to a given question and integrating complementary findings across them. However, existing pathology agents analyze one slide at a time and describe morphology rather than measure it, leaving evidence acquisition across a patient's specimens and patient-level quantification underexplored. To address this gap, we propose PlanPath, an agentic framework for patient-level pathology visual question answering (VQA). PlanPath turns the question into an exploration plan that specifies the evidence required, routes to the slides likely to carry it, and drives coarse-to-fine acquisition from tissue architecture down to individual cells. Tissue- and cell-level tools supply the counts, proportions, and extents, which PlanPath aggregates across slides into a traceable answer. To enable evaluation, we build two patient-level VQA datasets for renal and gastric cancer, averaging 15 WSIs per patient, with pathologist-verified question-answer pairs derived from diagnostic reports. Experimental results show that PlanPath outperforms the strongest baseline by 9.4% and 12.1% in overall accuracy on the two datasets and further improves existing slide-level agents when plugged into our framework. The dataset and code are available at: https://github.com/JSGe-AI/PlanPath_1.

Figures & tables

Figure 1: Comparison of slide-level and patient-level pathology VQA. (a) A slide-level agent performs well on single-slide questions. (b) Directly applying slide-level agents to multiple slides lacks effective slide selection and cross-slide reasoning. (c) PlanPath addresses patient-level questions through exploration planning, slide ranking, and cross-slide evidence integration.

Figure 2: Overview of PlanPath. The Agent Brain plans and coordinates question-driven slide retrieval and cross-slide multiscale evidence acquisition, followed by iterative evidence integration for prediction, confidence estimation, and traceable reasoning.

Method	Input	PatientVQA-Gastric				PatientVQA-Renal
Method	Input	Total	MCQ	Yes/No	Open	Total	MCQ	Yes/No	Open
Pretrained MLLM Models
GPT-5.5	Slide-T	35.10	39.77	30.97	34.85	40.71	23.97	60.55	22.14
GPT-5.6-Luna	Slide-T	37.70	36.29	50.75	29.80	49.29	47.11	62.08	21.37
Qwen3.5-4B	Slide-T	50.38	42.86	65.30	45.20	55.14	50.00	69.11	29.77
Qwen3.5-9B	Slide-T	47.24	41.70	62.31	40.66	55.57	46.69	71.56	32.06

Table 1: Performance comparison on PatientVQA-Gastric and PatientVQA-Renal. The best result in each column is shown in bold , and the second-best result is underlined .

Table 5

Figure 4: Case study comparing slide-level agents and PlanPath for patient-level VQA.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Construction of PatientVQA-Gastric and PatientVQA-Renal. Pathology reports and WSI collections are linked by pathology identifier. GPT-5.6 Sol generates candidate question–answer pairs from the reports. Two pathologists cross-review the candidates for clinical correctness, report support, and answerability from the available WSIs. Each retained question–answer pair is associated with the corresponding patient’s full WSI collection.

Type	Question and options	Answer
PatientVQA-Gastric
Short answer	What is the deepest level of invasion of the primary tumor in this patient?	Submucosa
Single-choice	What is the Lauren classification of the primary tumor in this patient? Options: A: Diffuse type; B: Indeterminate type; C: Intestinal type; D: Mixed type.	Mixed type
Yes/No	Is tumor identified at any submitted margin in this patient? Options: Yes; No.	No
PatientVQA-Renal
Short answer	Which pT category is assigned to this patient’s primary renal tumor?	pT1

Appendix

Table 5: Examples from the final question files. Each dataset’s three examples belong to one patient. Options are shown in their stored evaluation order. Reference answers are used only for evaluation.

Statistic	Gastric	Renal
Patients	104	109
WSIs	2,227	1,018
Question–answer pairs	923	700
WSIs per patient (mean)	21.41	9.34
WSIs per patient (median)	21	8
WSIs per patient (range)	13–58	2–36

Appendix

Table 6: Statistics of PatientVQA-Gastric and PatientVQA-Renal. Question-type percentages are calculated over all questions within the corresponding dataset.

Figure 6: Patient-level distributions in PatientVQA-Gastric and PatientVQA-Renal. (a) Number of WSIs per patient, grouped into intervals of five. (b) Number of questions per patient. Bar heights and the numbers above nonzero bars indicate patient counts.

Module	Model / Architecture	Key Parameters	Main Function
Agent Brain	Qwen3.5-4B	32 layers; hidden size 2560	Exploration planning, evidence verification, and answer generation.
Morphology Description Tool	Patho-R1-7B (Qwen2.5-VL)	8,192-token context; single-image input	Question-independent morphological description.
Slide Router	TITAN	768-D embeddings	Text–WSI similarity ranking across patient slides.
ROI Retrieval	PLIP	CLIP-based; $224\times 224$ input; 512-D features	Semantic ranking of candidate ROIs at $5\times$ and above.
Tissue-level Analysis Tool	Tissue Feature Prototype Bank	768-D features; softmax temperature 0.04	Localization of target tissues, interfaces, and heterogeneous regions.
Cell-level Quantification Tool	HistoPLUS	$20\times$ / $40\times$ models; $784\times 784$ input; confidence thr=0.5	Cell detection, instance segmentation, classification, and density quantification.

Appendix

Table 7: Implementation details of PlanPath.

Scale	WSI Level-0 Crop (px)	Patho-R1 Input (px)	PLIP Input (px)	Main Purpose	HistoPLUS
$5\times$	$4096\times 4096$	$512\times 512$	$224\times 224$	Tissue architecture, lesion distribution, and tumor interfaces.	Not used.
$20\times$	$1024\times 1024$	$512\times 512$	$224\times 224$	Cellular organization, epithelial lesions, and tumor–vessel/nerve relations.	$0.50~\mu$ m/px; $784\times 784$ input.
$40\times$	$512\times 512$	$512\times 512$	$224\times 224$	Nuclear/cytoplasmic morphology, tumor subtype, mitoses, and subtle foci.	$0.25~\mu$ m/px; $784\times 784$ input.

Appendix

Table 8: Scale-specific settings for hierarchical evidence acquisition in PlanPath.

Cohort	#WSIs	#Patches	$K$	Tissue Category	Dominant Clusters	Prototype Patches	Inference Prototypes
TCGA-STAD	1,051	1,944,093	100	Artifact / background	15	150	15
				Non-tumor mucosa / epithelium	3	30	3
				Gland-forming tumor	21	210	21
				Diffuse / poorly cohesive tumor	16	160	16
				Desmoplastic stroma / tumor bed	19	190	19
				Immune / lymphoid tissue	3	30	3

Appendix

Table 9: Statistics of the tissue feature prototype banks used by the Tissue-level Analysis Tool.

Method	Input	Total	MCQ	Yes/No	Open	BLEU-1	ROUGE-L
Pretrained MLLM Models
GPT-5.5 ( OpenAI, 2026a )	Slide-T	35.10	39.77	30.97	34.85	28.22	36.84
GPT-5.6-Luna ( OpenAI, 2026b )	Slide-T	37.70	36.29	50.75	29.80	18.65	27.70
Qwen3.5-4B ( Qwen Team, 2026 )	Slide-T	50.38	42.86	65.30	45.20	28.68	44.11
Qwen3.5-9B ( Qwen Team, 2026 )	Slide-T	47.24	41.70	62.31	40.66	25.95	44.32
Supervised Pathology Models

Appendix

Table 10: Detailed performance on PatientVQA-Gastric. BLEU-1 and ROUGE-L are additionally reported for open-ended questions. All values are reported as percentages. The best and second-best results in each column are shown in bold and underlined , respectively.

Method	Input	Total	MCQ	Yes/No	Open	BLEU-1	ROUGE-L
Pretrained MLLM Models
GPT-5.5 ( OpenAI, 2026a )	Slide-T	40.71	23.97	60.55	22.14	17.86	19.86
GPT-5.6-Luna ( OpenAI, 2026b )	Slide-T	49.29	47.11	62.08	21.37	24.75	23.85
Qwen3.5-4B ( Qwen Team, 2026 )	Slide-T	55.14	50.00	69.11	29.77	21.67	20.94
Qwen3.5-9B ( Qwen Team, 2026 )	Slide-T	55.57	46.69	71.56	32.06	31.39	27.19
Supervised Pathology Models

Appendix

Table 11: Detailed performance on PatientVQA-Renal. BLEU-1 and ROUGE-L are additionally reported for open-ended questions. All values are reported as percentages. The best and second-best results in each column are shown in bold and underlined , respectively.

Method	Input	Total	MCQ	Yes/No	Open	BLEU-1	ROUGE-L
Pretrained MLLM Models
GPT-5.5 ( OpenAI, 2026a )	Slide-T	45.50	45.17	46.27	45.20	24.32	22.05
GPT-5.6-Luna ( OpenAI, 2026b )	Slide-T	43.45	41.31	64.55	30.56	21.19	26.45
Qwen3.5-4B ( Qwen Team, 2026 )	Slide-T	53.30	47.49	64.18	49.75	34.64	50.09
Qwen3.5-9B ( Qwen Team, 2026 )	Slide-T	51.25	46.33	64.18	45.71	35.43	51.52
Supervised Pathology Models

Appendix

Table 12: Performance on PatientVQA-Gastric using majority-voting aggregation for slide-level baselines. PlanPath is shown unchanged for reference; results using Qwen3.5-4B aggregation are reported in the main-text Table 1 . Best and second-best results are shown in bold and underlined , respectively.

Method	Input	Total	MCQ	Yes/No	Open	BLEU-1	ROUGE-L
Pretrained MLLM Models
GPT-5.5 ( OpenAI, 2026a )	Slide-T	46.29	35.95	62.69	24.43	22.12	21.69
GPT-5.6-Luna ( OpenAI, 2026b )	Slide-T	55.00	62.81	63.30	19.85	24.54	21.62
Qwen3.5-4B ( Qwen Team, 2026 )	Slide-T	55.57	48.76	70.64	30.53	24.77	21.87
Qwen3.5-9B ( Qwen Team, 2026 )	Slide-T	54.29	41.74	71.87	33.59	32.71	28.72
Supervised Pathology Models

Appendix

Table 13: Performance on PatientVQA-Renal using majority-voting aggregation for slide-level baselines. PlanPath is shown unchanged for reference; results using Qwen3.5-4B aggregation are reported in the main-text Table 1 . Best and second-best results are shown in bold and underlined , respectively.

Figure 7: Detailed compatibility results of PlanPath with different slide-level pathology agents on PatientVQA-Gastric and PatientVQA-Renal. For PathAgent, PathNavigate, and GIANT, results are reported separately for MCQ, Yes/No, and Open questions. BEACON is evaluated on MCQ and Yes/No questions. Each pair of bars compares the original agent (Base) with the same agent integrated into PlanPath (+PlanPath), and the red annotations indicate the absolute performance change in percentage points.

Configuration	PatientVQA-Gastric				PatientVQA-Renal
Configuration	Total	MCQ	Yes/No	Open	Total	MCQ	Yes/No	Open
PlanPath (Full)	60.24	57.92	70.15	55.05	72.71	78.51	74.92	56.49
w/o Slide Router	58.50	57.53	70.15	51.26	71.29	76.03	73.70	56.49
w/o Tissue Analysis	56.45	58.69	64.55	49.49	69.43	73.97	72.78	52.67
w/o Cell Quant.	59.59	58.30	70.15	53.28	72.57	78.51	74.92	55.73
w/o Tissue Analysis & Cell Quant.	57.31	56.37	64.18	53.28	69.57	73.55	72.78	54.20

Appendix

Table 14: Detailed ablation results of PlanPath on PatientVQA-Gastric and PatientVQA-Renal. Cell Quant. denotes the Cell-level Quantification Tool. All values are reported as percentages.

Figure 8: Detailed comparison of all baselines on patient-level quantitative questions. Performance is evaluated using Exact Match and tolerance-based accuracy with thresholds of $\pm 5$ , $\pm 10$ , $\pm 15$ , and $\pm 20$ . Numerical values denote accuracy (%). Color intensity is normalized independently within each evaluation criterion, with darker purple indicating relatively better performance within the same row. Red boxes indicate the best result for each criterion.

Figure 9: Patient-level estimation of the signet-ring component. The exploration plan targets poorly cohesive cells with intracellular mucin and displaced nuclei. Higher-magnification observations provide phenotype-specific support, which is integrated with tissue-level quantitative evidence to estimate a proportion of approximately 10%.

Figure 10: Patient-level estimation of the hepatoid component. PlanPath examines trabecular and solid tumor regions and acquires higher-magnification evidence of polygonal cells with eosinophilic cytoplasm. Quantitative integration across the selected slides yields an estimated proportion of approximately 30%.

Figure 11: Cross-slide assessment of primary tumor focality. Tissue-level localization identifies spatially separate candidate components, and higher-magnification observations confirm their cellular content. Integration yields two independently supported foci while avoiding repeated counting across serial sections.

Figure 12: Cross-slide assessment of the renal resection margin. PlanPath localizes relevant cut-edge relationships and continues exploration after an initial negative finding to satisfy the four-slide candidate coverage requirement. The integrated answer is no carcinoma identified at the margin.

Figure 13: Identification of an additional simple cyst. Targeted exploration reveals cystic architecture and a single bland cuboidal lining, providing sufficient positive evidence after two candidate slides. Cellular views provide morphological context.

Pathology VQA

See all 16 papers

PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA
May 22, 2026Chunze Yang, Qidong Liu, Wenjie Zhao +10Healthcare Medical VQA
School of Comp. Science & Technology, Xi’an Jiaotong University · Tencent Jarvis Lab · University of Cambridge
Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoning
Feb 28, 2026Wentao Huang, Weimin Lyu, Peiliang Lou +8Pathology VQA Medical VQA
Stony Brook University, NY, USA · Mayo Clinic, MN, USA · Harvard Medical School, MA, USA +1
PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image
Jul 21, 2026Dankai Liao, Tianyi Zhang, Yufeng Wu +6VLM Evaluation Medical VLMs
Department of Biomedical Engineering, National University of Singapore, Singapore 117417, Singapore

May 22, 2026J/K move · Enter open · S save

PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA

Chunze Yang, Qidong Liu, Wenjie Zhao, Yue Tang+9

School of Comp. Science & Technology, Xi’an Jiaotong University · Tencent Jarvis Lab · University of Cambridge

Healthcare Medical VQA Tool-Using Vision-Language Agents Visual Search Whole-Slide Image Analysis Pathology VQA

Open paper AskPDF arXiv

Whole-slide image visual question answering (WSI-VQA) frames pathology as an extreme-context search problem: to answer a free-form clinical query, a system must first navigate a gigapixel slide under a strict inspection budget to locate sparse, high-resolution evidence. Existing approaches largely fall into two paradigms: i) supervised pathology multimodal large language models (MLLMs) and agents can absorb localization and reasoning into learned modules, but they often couple navigation to task-specific supervision and retraining, limiting their practicality; ii) training-free pathology agents avoid this cost by keeping core models frozen, but often follow a question-first design, constructing the initial candidate set mainly from query-conditioned relevance. This can miss decisive morphology that is not named in the question, and force heavier inference-time scaffolding. To address this challenge, we introduce PathNavigate, a training-free pathology agent built around a scan-search-readout routine. Before question matching, PathNavigate scans the current slide at low magnification with a shared online memory module over frozen pathology features, producing a slide-specific surprise field that marks an abnormal-region pool. It then applies question-conditioned PLIP relevance only within this pool to select high-magnification search targets. Finally, it extracts local high-magnification evidence and answers with a frozen perceptor-adjudicator stack, using the same online memory as slide-level context. Experiments on WSI-VQA and SlideBench-BCNB show that the proposed scan-search-readout design improves answer accuracy and yields more interpretable evidence-selection trajectories with higher efficiency.The code is available online.

Abstract

Pathology VQA

Explore similar work

PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA

Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoning

PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image