CT-Flow: Orchestrating CT Interpretation Workflow with Model Context Protocol Servers

Yannian Gu; Xizhuo Zhang; Linjie Mu; Yongrui Yu; Zhongzhen Huang; Shaoting Zhang; Xiaofan Zhang

cs.CVFeb 23, 2026

CT-Flow: Orchestrating CT Interpretation Workflow with Model Context Protocol Servers

Authors: Yannian Gu, Xizhuo Zhang, Linjie Mu, Yongrui Yu, Zhongzhen Huang, Shaoting Zhang, Xiaofan Zhang

Organizations: Qing Yuan Research Institute, Shanghai Jiao Tong University, Shanghai, China · Sensetime Research, Shanghai, China · Shanghai Innovation Institute, Shanghai, China

Abstract

Recent advances in Large Vision-Language Models (LVLMs) have shown strong potential for multi-modal radiological reasoning, particularly in tasks like diagnostic visual question answering (VQA) and radiology report generation. However, most existing approaches for 3D CT analysis largely rely on static, single-pass inference. In practice, clinical interpretation is a dynamic, tool-mediated workflow where radiologists iteratively review slices and use measurement, radiomics, and segmentation tools to refine findings. To bridge this gap, we propose CT-Flow, an agentic framework designed for interoperable volumetric interpretation. By leveraging the Model Context Protocol (MCP), CT-Flow shifts from closed-box inference to an open, tool-aware paradigm. We curate CT-FlowBench, the first large-scale instruction-tuning benchmark tailored for 3D CT tool-use and multi-step reasoning. Built upon this, CT-Flow functions as a clinical orchestrator capable of decomposing complex natural language queries into automated tool-use sequences. Experimental evaluations on CT-FlowBench and standard 3D VQA datasets demonstrate that CT-Flow achieves state-of-the-art performance, surpassing baseline models by 41% in diagnostic accuracy and achieving a 95% success rate in autonomous tool invocation. This work provides a scalable foundation for integrating autonomous, agentic intelligence into real-world clinical radiology.

Figures & tables

Figure 1: Comparison of 3D CT analysis paradigms. Left: Traditional End-to-End LVLMs rely on passive visual ingestion of 3D data, resulting in static textual outputs. Right: The proposed CT-Flow framework leverages the Model Context Protocol to transform the LLM into an active agent. It dynamically orchestrates specialized tools to deliver precise, multi-modal diagnosis.

Figure 2: Overview of the CT-Flow framework. (i) Data Construction: The pipeline for raw data curation, trajectory synthesis, and the establishment of the CT-Flow benchmark. (ii) Architectures: The system decouples the LLM orchestrator from the imaging environment via FASTMCP, bridging high-level servers with medical imaging infrastructures to provide a suite of atomic tools in the Tool Space. (iii) Case Study: A demonstration of a Language-Action Trajectory $\mathcal{T}$ . The orchestrator performs Active Probing by iteratively generating reasoning states ( $s_{t}$ ), executing tool calls ( $a_{t}$ ), and interpreting high-fidelity observations ( $o_{t}$ ) to reach a grounded diagnostic answer.

Models	Tool-use	3D-RAD					CT-FlowBench (Acc. %)
Models	Tool-use	BLEU-4	ROUGE-L	B-Score	LLM-Judge	ACC(%)	QA	AM	DD	Avg.
Leading-edge Models
GPT-5.2	$\checkmark$	22.08	26.06	85.54	18.25	63.50	35.00	40.00	37.00	37.33
Gemini-3-Pro-Preview	$\checkmark$	29.59	35.59	89.46	26.38	62.59	45.00	43.00	44.00	44.00
Claude-Sonnet-4.5	$\checkmark$	20.14	26.81	85.71	8.85	54.83	44.00	44.00	43.00	43.67
Qwen3-VL-235B-A22B-Instruct	$\checkmark$	20.55	22.62	85.44	9.52	54.21	30.00	36.00	36.00	34.00

Table 1: Performance comparison on 3D-RAD and CT-FlowBench. For 3D-RAD, the reported metrics represent the average performance across all sub-tasks. For CT-FlowBench, QA, AM, and DD correspond to Quantitative Analysis, Spatial Mapping, and Diagnostic Inference, respectively.

Figure 3: Comparative performance of various models using the CT-Flow framework vs. the slice-based baseline.

Models	3D-RAD			CT-FlowBench
Models	Calls	Name Errors	Args Errors	Calls	Name Errors	Args Errors
GPT-5.2	4.133	0.006	0.056	7.193	0.003	0.108
Gemini-3-Pro-Preview	5.088	0.008	0.156	6.667	0.100	0.451
Claude-Sonnet-4.5	5.930	0.002	0.092	9.490	0.017	0.407
Qwen3-VL-235B-A22B-Instruct	6.072	0.085	0.190	6.719	0.224	0.453
Qwen3-VL-8B-Instruct	5.963	0.782	0.211	11.196	0.969	0.385

Table 2: Tool usage and error statistics on 3D-RAD and CT-FlowBench. Calls , Name Errors , and Args Errors denote the average number of tool calls, tool-name errors and tool-argument errors per case, respectively.

Figure 4: Performance impact of tool category ablation. Bars indicate accuracy (%) and the red line tracks format errors. Removing specific tool classes (cls. 2-4) leads to decreased diagnostic accuracy and increased errors across all tasks, validating the necessity of the full hierarchical toolset.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Included Tools	Clinical Objective & Description
I. Data Ingestion	load_data , load_mask , inspect_metadata , inspect_mask_labels , search_anatomy_names , list_window_presets	Data Integrity: Establishes the workspace and validates metadata (modality, spacing, labels) to ensure subsequent analysis is performed on accurate objects.
II. Global Observation	view_montage , view_mip , view_minip , view_avgip	Macro-Navigation: Provides a “bird’s-eye view” using projection (MIP/MinIP) or array layouts to help users rapidly screen for high-density (nodules) or low-density (airways) abnormalities.
III. Detailed Measurement	view_slice , view_ortho , measure_distance , measure_max_diameter , find_organ_center , extract_vessel_centerline , auto_crop_body , edit_geometry	Spatial Localization: The most high-frequency module. Combines slice-by-slice exploration with 3D multi-planar reconstruction (MPR) and precise physical quantification of lesion size and geometry.
IV. Advanced Analytics	segment_total_anatomy , analyze_hu_distribution , analyze_shape_properties , extract_radiomics_signature , visualize_radiomics_chart , analyze_lesion_texture	Quantitative Biomarkers: Post-processing stage utilizing AI and statistics to transform visual data into structured radiomic features for prognostic modeling and malignancy assessment.

Appendix

Table 3: Classification and Functional Description of Medical Imaging Analysis Tools

Model	Task 1			Task 2			Task 3			Task 4	Task 5	Task 6	Avg.
Model	BLEU	Rouge	Bert	BLEU	Rouge	Bert	BLEU	Rouge	Bert	acc	acc	acc	Avg.
Gemini-3-Pro	11.72	15.81	73.30	12.47	19.36	75.35	12.05	12.04	53.30	61.00	42.50	63.00	55.50
w CT-Flow	26.85	38.29	87.94	25.95	34.38	87.67	35.98	34.10	92.78	60.50	52.26	75.00	62.59
	( $\uparrow$ 15.13 )	( $\uparrow$ 22.48 )	( $\uparrow$ 14.64 )	( $\uparrow$ 13.48 )	( $\uparrow$ 15.02 )	( $\uparrow$ 12.32 )	( $\uparrow$ 23.93 )	( $\uparrow$ 22.06 )	( $\uparrow$ 39.48 )	( $\downarrow$ 0.50)	( $\uparrow$ 9.76 )	( $\uparrow$ 12.00 )	( $\uparrow$ 7.09 )
Gpt-5.2	9.51	16.67	83.77	10.72	19.52	84.34	1.08	5.03	81.37	59.50	36.00	70.00	55.17
w CT-Flow	19.68	25.38	84.60	20.42	26.61	85.08	26.14	26.19	86.94	70.00	43.00	77.50	63.50

Appendix

Table 4: Performance comparison on the 3D Rad 1.2k dataset. M3D natively supports 3D volumes, whereas other baseline models process individual 2D slices. Models denoted with w CT Lens leverage our proposed CTLens framework to integrate spatial-contextual information.

Model	Scenario 1	Scenario 2	Scenario 3	Ave.
Gemini-3-Pro	35.00	35.00	31.00	33.67
w CT-Flow	45.00	43.00	44.00	44.00
	( $\uparrow$ 10.00 )	( $\uparrow$ 8.00 )	( $\uparrow$ 13.00 )	( $\uparrow$ 10.33 )
Gpt-5.2	26.00	36.00	37.00	33.00
w CT-Flow	35.00	40.00	37.00	37.33
	( $\uparrow$ 9.00 )	( $\uparrow$ 4.00 )	(0.00)	( $\uparrow$ 4.33 )

Appendix

Table 5: Performance comparison across different scenarios. Models denoted with w CT-Flow show the integration of spatial-contextual information.

Figure 5: Structure of the System Prompt. The core principles, critical thinking rules, and standard operating procedures (SOPs) for the AI Medical Imaging Assistant are displayed. To facilitate a clear presentation, the specific technical definitions of available tools have been truncated.

Medical Image Analysis

Medical image analysis uses computational methods to interpret clinical images, measure anatomy and disease, and support diagnosis and treatment.

See all 292 papers

RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomography
Apr 16, 2026Mélanie Roschewitz, Kenneth Styppa, Yitian Tao +10Tool-Using Vision-Language Agents Healthcare
Department of Biosystems Science and Engineering, ETH Zurich, Basel, Switzerland · ETH AI Center, Zurich, Switzerland · Department of Computer Science, ETH Zurich, Zurich, Switzerland +5
MARCH: Multi-Agent Radiology Clinical Hierarchy for CT Report Generation
Apr 17, 2026Yi Lin, Yihao Ding, Yonghui Wu +1Radiology Report Generation Multi-Agent Collaboration
Weill Cornell Medicine, New York, USA · University of Western Australia, Crawley, Australia · University of Florida, Florida, USA
NV-Reason-CT: 3D Visual Language Model for CT Analysis
Sep 23, 2026Andriy Myronenko, Dong Yang, Yucheng Tang +15Radiology Report Generation Radiology VLMs
NVIDIA, Santa Clara, CA · NIH, Bethesda, MD · University of Oxford, Oxford, UK +4

Apr 16, 2026J/K move · Enter open · S save

RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomography

Mélanie Roschewitz, Kenneth Styppa, Yitian Tao, Jiwoong Sohn+9

Department of Biosystems Science and Engineering, ETH Zurich, Basel, Switzerland · ETH AI Center, Zurich, Switzerland · Department of Computer Science, ETH Zurich, Zurich, Switzerland+5

Tool-Using Vision-Language Agents Healthcare Explainable Medical Image Analysis Computed Tomography Radiology Report Generation CT Report Generation+1

Open paper AskPDF arXiv

Vision-language models (VLM) have markedly advanced AI-driven interpretation and reporting of complex medical imaging, such as computed tomography (CT). Yet, existing methods largely relegate clinicians to passive observers of final outputs, offering no interpretable reasoning trace for them to inspect, validate, or refine. To address this, we introduce RadAgent, a tool-using AI agent that generates CT reports through a stepwise and interpretable process. Each resulting report is accompanied by a fully inspectable trace of intermediate decisions and tool interactions, allowing clinicians to examine how the reported findings are derived. In our experiments, we observe that RadAgent improves chest CT report generation over its 3D VLM counterpart, CT-Chat, across three dimensions. Clinical accuracy improves by 5.8 points (35.4% relative) in macro-F1 and 5.1 points (18.6% relative) in micro-F1. Robustness under adversarial conditions improves by 24.7 points (41.9% relative). Furthermore, RadAgent achieves 37.0% in faithfulness, a new capability entirely absent in its 3D VLM counterpart. By structuring the interpretation of chest CT as an explicit, tool-augmented and iterative reasoning trace, RadAgent brings us closer toward transparent and reliable AI for radiology.

Abstract

Medical Image Analysis

Explore similar work

RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomography

MARCH: Multi-Agent Radiology Clinical Hierarchy for CT Report Generation

NV-Reason-CT: 3D Visual Language Model for CT Analysis