Agentic Automation of BT-RADS Scoring: End-to-End Multi-Agent System for Standardized Brain Tumor Follow-up Assessment
Authors: Mohamed Sobhi Jabal, Jikai Zhang, Dominic LaBella, Jessica L. Houk, Dylan Zhang, Jeffrey D. Rudie, Kirti Magudia, Maciej A. Mazurowski, Evan Calabrese
Organizations: Department of Radiology, Duke University Medical Center, Durham, NC · Department of Electrical and Computer Engineering, Duke University, Durham, NC · Duke Center for Artificial Intelligence in Radiology, Duke University Medical Center, Durham, NC · Department of Radiation Oncology, Duke University Medical Center, Durham, NC · Department of Radiology, Santa Clara Valley Medical Center, San Jose, CA · Department of Radiology, University of California San Diego, San Diego, CA · Department of Radiology, Scripps Clinic Medical Group, San Diego, CA · Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC
The Brain Tumor Reporting and Data System (BT-RADS) standardizes post-treatment MRI response assessment in patients with diffuse gliomas but requires complex integration of imaging trends, medication effects, and radiation timing. This study evaluates an end-to-end multi-agent large language model (LLM) and convolutional neural network (CNN) system for automated BT-RADS classification. A multi-agent LLM system combined with automated CNN-based tumor segmentation was retrospectively evaluated on 509 consecutive post-treatment glioma MRI examinations from a single high-volume center. An extractor agent identified clinical variables (steroid status, bevacizumab status, radiation date) from unstructured clinical notes, while a scorer agent applied BT-RADS decision logic integrating extracted variables with volumetric measurements. Expert reference standard classifications were established by an independent board-certified neuroradiologist. Of 509 examinations, 492 met inclusion criteria. The system achieved 374/492 (76.0%; 95% CI, 72.1%-79.6%) accuracy versus 283/492 (57.5%; 95% CI, 53.1%-61.8%) for initial clinical assessments (+18.5 percentage points; P<.001). Context-dependent categories showed high sensitivity (BT-1b 100%, BT-1a 92.7%, BT-3a 87.5%), while threshold-dependent categories showed moderate sensitivity (BT-3c 74.8%, BT-2 69.2%, BT-4 69.3%, BT-3b 57.1%). For BT-4, positive predictive value was 92.9%. The multi-agent LLM system achieved higher BT-RADS classification agreement with expert reference standard compared to initial clinical scoring, with high accuracy for context-dependent scores and high positive predictive value for BT-4 detection.
Figures & tables
Figure 1: Overview of the multi-agent system for automated BT-RADS classification. The pipeline integrates CNN-based tumor segmentation (nnU-Net) for volumetric quantification of FLAIR and enhancement volumes with a 20-billion parameter open-weight LLM (extractor agent) that identifies clinical variables (steroid status, bevacizumab use, radiation date) from unstructured clinical notes with evidence span linking, and a scorer agent that applies BT-RADS decision logic integrating extracted variables with quantitative volumetric measurements. An Orchestrator coordinates data flow between agents. Schema-constrained generation with Pydantic validation ensures outputs conform to predefined formats.
Figure 2: BT-RADS decision flowchart implemented by the scoring agent. Sequential rules classify cases based on volumetric thresholds ( ± 20% stability, > 40% major change), medication effects (bevacizumab, steroids), and radiation timing ( < 90 days post-completion). Terminal nodes correspond to BT-RADS categories BT-0 through BT-4.
Category
Description
N
Percentage
BT-1a
Imaging improvement
55
11.2%
BT-1b
Likely treatment effect
51
10.4%
BT-2
Stable imaging
156
31.7%
BT-3a
Favor treatment effect
16
3.3%
BT-3b
Indeterminate
21
4.3%
BT-3c
Favor tumor
115
23.4%
Table 1: Expert Reference Standard BT-RADS Distribution
Figure 3: Composite classification performance. (A) Overall accuracy: multi-agent system versus initial workflow (McNemar P<.001 ); error bars represent 95% confidence intervals. (B) Per-category sensitivity of the agentic system in BT-RADS category order. (C) Multi-agent system confusion matrix. (D) Initial clinical assessment confusion matrix. In C and D, color intensity represents row-normalized sensitivity; cell values are raw counts; diagonal cells (correct classifications) are outlined.
Category
Type
N (RS)
Correct
Sensitivity
95% CI
BT-1a
Context-dependent
55
51
92.7%
82.7%–97.1%
BT-1b
Context-dependent
51
51
100.0%
93.0%–100.0%
BT-2
Threshold-dependent
156
108
69.2%
61.6%–75.9%
BT-3a
Context-dependent
16
14
87.5%
64.0%–96.5%
BT-3b
Threshold-dependent
21
12
57.1%
36.5%–75.5%
BT-3c
Threshold-dependent
115
86
74.8%
66.1%–81.8%
Table 2: Per-Category Classification Sensitivity
Category
N (RS)
Sensitivity
Specificity
PPV
NPV
LR+
LR −
BT-1a
55
92.7%
94.2%
67.1%
99.0%
16.0
0.08
BT-1b
51
100.0%
96.3%
76.1%
100.0%
27.0
0.00
BT-2
156
69.2%
93.7%
83.7%
86.7%
11.0
0.33
BT-3a
16
87.5%
99.6%
87.5%
99.6%
218.8
0.13
BT-3b
21
57.1%
97.2%
48.0%
98.1%
20.4
0.44
BT-3c
115
74.8%
90.9%
71.7%
92.1%
8.2
0.28
Table 3: Diagnostic Performance by BT-RADS Category
Scenario
Accuracy
Improvement
Description
Current system
76.0%
Baseline
Observed performance with all error sources
Perfect extraction
78.5%
+2.5 pp
All 3 clinical variable extractions match expert
Perfect algorithm
82.8%
+6.8 pp
Optimal rule-based logic for edge cases
Perfect extraction + algorithm
88.1%
+12.1 pp
Correct extractions + optimal classification
Theoretical maximum
99.4%
+23.4 pp
Only inherent categorical ambiguity remains
Table 4: Theoretical Performance Ceiling
Figure 4: Clinical information extraction and BT-RADS classification for three representative de-identified cases. (A) BT-1b: bevacizumab use identified from clinical notes; volumetric improvement routed through the medication effect pathway. (B) BT-2: both FLAIR and enhancement volumes stable within ± 20%; no active medications. (C) BT-4: both components worsening (FLAIR + 231%, enhancement + 187%) with at least one exceeding 40%; radiation completed more than 90 days prior. Each panel shows the clinical note excerpt with highlighted evidence spans, extracted variables, volumetric data, and the decision pathway leading to the final classification.
Figure 5: Per-category classification concordance between the automated system (76.0%; 374/492), initial clinical assessment (57.5%; 283/492), and study population reference standard ( n=492 ). Each square represents one examination, colored by BT-RADS category. Solid squares indicate correct classification; faded squares indicate misclassification. Per-category accuracy is listed below (automated system / initial clinical assessment). McNemar P<.001 .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Characteristic
Excluded ( n=17 )
Evaluable ( n=492 )
Exclusion Reason
No suitable baseline
9 (52.9%)
—
Segmentation quality issue
8 (47.1%)
—
Volumetric Changes
FLAIR % (mean ± SD)
+ 24.1% ± 72.3
+ 29.5% ± 95.0
Enhancement % (mean ± SD)
+ 20.9% ± 88.5
+ 49.0% ± 234.7
Appendix
Table S1. Characteristics of Excluded Cases Compared to Evaluable Cohort
Table S3. Classification Accuracy Stratified by Extraction Success
Figure S1. Distribution of Volumetric Changes: FLAIR and Enhancement ( n=509 ). Overlaid histogram of FLAIR (blue) and enhancement (red) volumetric percentage changes across all 509 cases. Dashed lines indicate the ± 20% stability and 40% worsening thresholds used in BT-RADS classification.
Figure S2. Threshold Boundary Analysis: Classification Accuracy by Volumetric Change Magnitude ( n=492 ). Distribution of (A) FLAIR and (B) enhancement volumetric changes colored by classification outcome (green = correct, red = incorrect). Vertical dashed lines indicate BT-RADS decision thresholds.
Clinicians diagnose brain tumors by synthesizing patient symptoms, medical history, and quantitative imaging data from modalities such as MRI and CT scans into a unified clinical judgement. However, most deep learning models rely on MRI/CT images alone, failing to replicate the clinicians multimodal reasoning. We explore a two-branch multimodal network combining raw MRI scans with 91 extracted radiomic features (intensity, texture, shape, and boundary descriptors) to classify brain tumors into glioma, meningioma, pituitary, and no-tumor. A pre-trained CNN backbone encodes the image stream, whereas a dedicated MLP encodes the radiomic stream. Both streams are fused via concatenation, gated, or bidirectional cross-modal attention strategies. Across nine experimental runs on a balanced 7,200 image dataset, all multimodal configurations outperform unimodal baselines with gated fusion achieving the best accuracy of 96.13%.
Wajih ul Islam, Muhammad Yaqoob, Javed Ali Khan +1
School of Physics, Engineering and Computer Science, University of Hertfordshire, UK
Accurate brain tumor typing requires integrating heterogeneous clinical evidence, including magnetic resonance imaging (MRI), histopathology, and pathology reports, which are often incomplete at the time of diagnosis. We introduce CoRe-BT, a cross-modal radiology-pathology-text benchmark for brain tumor typing, designed to study robust multimodal learning under missing modality conditions. The dataset comprises 310 patients with multi-sequence brain MRI (T1, T1c, T2, FLAIR), including 95 cases with paired H&E-stained whole-slide pathology images and pathology reports. All cases are annotated with tumor type and grade, and MRI volumes include expert-annotated tumor masks, enabling both region-aware modeling and auxiliary learning tasks. Tumors are categorized into six clinically relevant classes capturing the heterogeneity of common and rare glioma subtypes. We evaluate tumor typing under variable modality availability by comparing MRI-only models with multimodal approaches that incorporate pathology information when present. Baseline experiments demonstrate the feasibility of multimodal fusion and highlight complementary modality contributions across clinically relevant typing tasks. CoRe-BT provides a grounded testbed for advancing multimodal glioma typing and representation learning in realistic scenarios with incomplete clinical data.
Juampablo E. Heras Rivera, Daniel K. Low, Xavier Xiong +5
University of Washington, Seattle, Washington 98105, USA · University of Washington School of Medicine, Seattle, Washington 98105, USA · Microsoft Health AI, Redmond, Washington 98052-6399, USA
State-of-the-art large language models (LLMs) show high performance in general visual question answering. However, a fundamental limitation remains: current architectures lack the native 3D spatial reasoning required to directly analyze volumetric medical imaging, such as CT or MRI. Emerging agentic AI offers a new solution, eliminating the need for intrinsic 3D processing by enabling LLMs to orchestrate and leverage specialized external tools. Yet, the feasibility of such agentic frameworks in complex, multi-step radiological workflows remains underexplored. In this work, we present a training-free agentic pipeline for automated brain MRI analysis. Validating our methodology on several LLMs (GPT-5.4, Gemini 3.1 Pro, Claude Sonnet 4.6) with off-the-shelf domain-specific tools, our system autonomously executes complex end-to-end workflows, including preprocessing (skull stripping, registration), pathology segmentation (glioma, meningioma, metastases), and volumetric reasoning. We evaluate our framework across increasingly complex radiological tasks, from single-scan tumor and anatomy analysis to longitudinal response assessment requiring multi-timepoint comparisons. We analyze the impact of architectural design by comparing single-agent models against multi-agent "domain-expert" collaborations. To support rigorous evaluation of future agentic systems, we release a benchmark dataset of image-prompt-answer tuples derived from public data. Our results demonstrate that agentic AI can solve highly neuro-radiological image analysis tasks through tool use without the need for training or fine-tuning.
Ayhan Can Erdur, Daniel Scholz, Jiazhen Pan +3
Department of Radiation Oncology, TUM University Hospital, Munich, Germany · Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Chair for AI for Image-Guided Diagnosis and Therapy, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany +3