Agentic Automation of BT-RADS Scoring: End-to-End Multi-Agent System for Standardized Brain Tumor Follow-up Assessment

Mohamed Sobhi Jabal; Jikai Zhang; Dominic LaBella; Jessica L. Houk; Dylan Zhang; Jeffrey D. Rudie; Kirti Magudia; Maciej A. Mazurowski

cs.CLMar 23, 2026

Agentic Automation of BT-RADS Scoring: End-to-End Multi-Agent System for Standardized Brain Tumor Follow-up Assessment

Authors: Mohamed Sobhi Jabal, Jikai Zhang, Dominic LaBella, Jessica L. Houk, Dylan Zhang, Jeffrey D. Rudie, Kirti Magudia, Maciej A. Mazurowski,

Organizations: Department of Radiology, Duke University Medical Center, Durham, NC · Department of Electrical and Computer Engineering, Duke University, Durham, NC · Duke Center for Artificial Intelligence in Radiology, Duke University Medical Center, Durham, NC · Department of Radiation Oncology, Duke University Medical Center, Durham, NC ·

Abstract

The Brain Tumor Reporting and Data System (BT-RADS) standardizes post-treatment MRI response assessment in patients with diffuse gliomas but requires complex integration of imaging trends, medication effects, and radiation timing. This study evaluates an end-to-end multi-agent large language model (LLM) and convolutional neural network (CNN) system for automated BT-RADS classification. A multi-agent LLM system combined with automated CNN-based tumor segmentation was retrospectively evaluated on 509 consecutive post-treatment glioma MRI examinations from a single high-volume center. An extractor agent identified clinical variables (steroid status, bevacizumab status, radiation date) from unstructured clinical notes, while a scorer agent applied BT-RADS decision logic integrating extracted variables with volumetric measurements. Expert reference standard classifications were established by an independent board-certified neuroradiologist. Of 509 examinations, 492 met inclusion criteria. The system achieved 374/492 (76.0%; 95% CI, 72.1%-79.6%) accuracy versus 283/492 (57.5%; 95% CI, 53.1%-61.8%) for initial clinical assessments (+18.5 percentage points; P<.001). Context-dependent categories showed high sensitivity (BT-1b 100%, BT-1a 92.7%, BT-3a 87.5%), while threshold-dependent categories showed moderate sensitivity (BT-3c 74.8%, BT-2 69.2%, BT-4 69.3%, BT-3b 57.1%). For BT-4, positive predictive value was 92.9%. The multi-agent LLM system achieved higher BT-RADS classification agreement with expert reference standard compared to initial clinical scoring, with high accuracy for context-dependent scores and high positive predictive value for BT-4 detection.

Figures & tables

Figure 1: Overview of the multi-agent system for automated BT-RADS classification. The pipeline integrates CNN-based tumor segmentation (nnU-Net) for volumetric quantification of FLAIR and enhancement volumes with a 20-billion parameter open-weight LLM (extractor agent) that identifies clinical variables (steroid status, bevacizumab use, radiation date) from unstructured clinical notes with evidence span linking, and a scorer agent that applies BT-RADS decision logic integrating extracted variables with quantitative volumetric measurements. An Orchestrator coordinates data flow between agents. Schema-constrained generation with Pydantic validation ensures outputs conform to predefined formats.

Figure 2: BT-RADS decision flowchart implemented by the scoring agent. Sequential rules classify cases based on volumetric thresholds ( $\pm$ 20% stability, $>$ 40% major change), medication effects (bevacizumab, steroids), and radiation timing ( $<$ 90 days post-completion). Terminal nodes correspond to BT-RADS categories BT-0 through BT-4.

Category	Description	N	Percentage
BT-1a	Imaging improvement	55	11.2%
BT-1b	Likely treatment effect	51	10.4%
BT-2	Stable imaging	156	31.7%
BT-3a	Favor treatment effect	16	3.3%
BT-3b	Indeterminate	21	4.3%
BT-3c	Favor tumor	115	23.4%

Table 1: Expert Reference Standard BT-RADS Distribution

Figure 3: Composite classification performance. (A) Overall accuracy: multi-agent system versus initial workflow (McNemar $P<.001$ ); error bars represent 95% confidence intervals. (B) Per-category sensitivity of the agentic system in BT-RADS category order. (C) Multi-agent system confusion matrix. (D) Initial clinical assessment confusion matrix. In C and D, color intensity represents row-normalized sensitivity; cell values are raw counts; diagonal cells (correct classifications) are outlined.

Category	Type	N (RS)	Correct	Sensitivity	95% CI
BT-1a	Context-dependent	55	51	92.7%	82.7%–97.1%
BT-1b	Context-dependent	51	51	100.0%	93.0%–100.0%
BT-2	Threshold-dependent	156	108	69.2%	61.6%–75.9%
BT-3a	Context-dependent	16	14	87.5%	64.0%–96.5%
BT-3b	Threshold-dependent	21	12	57.1%	36.5%–75.5%
BT-3c	Threshold-dependent	115	86	74.8%	66.1%–81.8%

Table 2: Per-Category Classification Sensitivity

Category	N (RS)	Sensitivity	Specificity	PPV	NPV	LR+	LR $-$
BT-1a	55	92.7%	94.2%	67.1%	99.0%	16.0	0.08
BT-1b	51	100.0%	96.3%	76.1%	100.0%	27.0	0.00
BT-2	156	69.2%	93.7%	83.7%	86.7%	11.0	0.33
BT-3a	16	87.5%	99.6%	87.5%	99.6%	218.8	0.13
BT-3b	21	57.1%	97.2%	48.0%	98.1%	20.4	0.44
BT-3c	115	74.8%	90.9%	71.7%	92.1%	8.2	0.28

Table 3: Diagnostic Performance by BT-RADS Category

Scenario	Accuracy	Improvement	Description
Current system	76.0%	Baseline	Observed performance with all error sources
Perfect extraction	78.5%	+2.5 pp	All 3 clinical variable extractions match expert
Perfect algorithm	82.8%	+6.8 pp	Optimal rule-based logic for edge cases
Perfect extraction + algorithm	88.1%	+12.1 pp	Correct extractions + optimal classification
Theoretical maximum	99.4%	+23.4 pp	Only inherent categorical ambiguity remains

Table 4: Theoretical Performance Ceiling

Figure 4: Clinical information extraction and BT-RADS classification for three representative de-identified cases. (A) BT-1b: bevacizumab use identified from clinical notes; volumetric improvement routed through the medication effect pathway. (B) BT-2: both FLAIR and enhancement volumes stable within $\pm$ 20%; no active medications. (C) BT-4: both components worsening (FLAIR $+$ 231%, enhancement $+$ 187%) with at least one exceeding 40%; radiation completed more than 90 days prior. Each panel shows the clinical note excerpt with highlighted evidence spans, extracted variables, volumetric data, and the decision pathway leading to the final classification.

Figure 5: Per-category classification concordance between the automated system (76.0%; 374/492), initial clinical assessment (57.5%; 283/492), and study population reference standard ( $n=492$ ). Each square represents one examination, colored by BT-RADS category. Solid squares indicate correct classification; faded squares indicate misclassification. Per-category accuracy is listed below (automated system / initial clinical assessment). McNemar $P<.001$ .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Characteristic	Excluded ( $n=17$ )	Evaluable ( $n=492$ )
Exclusion Reason
No suitable baseline	9 (52.9%)	—
Segmentation quality issue	8 (47.1%)	—
Volumetric Changes
FLAIR % (mean $\pm$ SD)	$+$ 24.1% $\pm$ 72.3	$+$ 29.5% $\pm$ 95.0
Enhancement % (mean $\pm$ SD)	$+$ 20.9% $\pm$ 88.5	$+$ 49.0% $\pm$ 234.7

Appendix

Table S1. Characteristics of Excluded Cases Compared to Evaluable Cohort

Subgroup	Stratum	N	Accuracy	95% CI
Temporal (Post-RT)	$<$ 90 days	40	72.5%	57.2%–83.9%
	90–180 days	85	87.1%	78.3%–92.6%
	$>$ 180 days	280	75.7%	70.4%–80.4%
	Unknown	84	70.2%	59.8%–79.0%
Medication	Bevacizumab active	167	74.9%	67.8%–80.8%
	Steroids only	74	85.1%	75.3%–91.5%

Appendix

Table S1. Comprehensive Subgroup Analysis

Error Category	N	%	Remediability
Threshold boundary	52	44.1%	Irreducible
Extraction error propagation	34	28.8%	Remediable
Algorithm limitation	18	15.3%	Remediable
Ground truth ambiguity	14	11.9%	Irreducible
Total	118	100%

Appendix

Table S2. Error Attribution Analysis (

n=118

misclassifications)

Extraction Status	N	Correct	Accuracy	95% CI
All 3 extractions correct	380	296	77.9%	73.5%–81.8%
Any extraction incorrect	109	78	71.6%	62.5%–79.2%
Radiation date incorrect	43	28	65.1%	50.2%–77.6%
Steroid status incorrect	59	44	74.6%	62.2%–83.9%
Bevacizumab incorrect	14	11	78.6%	52.4%–92.4%

Appendix

Table S3. Classification Accuracy Stratified by Extraction Success

Figure S1. Distribution of Volumetric Changes: FLAIR and Enhancement ( $n=509$ ). Overlaid histogram of FLAIR (blue) and enhancement (red) volumetric percentage changes across all 509 cases. Dashed lines indicate the $\pm$ 20% stability and 40% worsening thresholds used in BT-RADS classification.

Figure S2. Threshold Boundary Analysis: Classification Accuracy by Volumetric Change Magnitude ( $n=492$ ). Distribution of (A) FLAIR and (B) enhancement volumetric changes colored by classification outcome (green = correct, red = incorrect). Vertical dashed lines indicate BT-RADS decision thresholds.

Brain Tumor Classification

Brain tumor classification uses medical images and molecular data to identify tumor types, grades, and subtypes for diagnosis and treatment planning.

See all 51 papers

Multimodal Brain Tumour Classification Using Feature Fusion
Jun 9, 2026Wajih ul Islam, Muhammad Yaqoob, Javed Ali Khan +1Multimodal Classification Medical Image Classification
School of Physics, Engineering and Computer Science, University of Hertfordshire, UK
CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing
Mar 4, 2026Juampablo E. Heras Rivera, Daniel K. Low, Xavier Xiong +5Brain Tumor Classification Medical Image Benchmarks
University of Washington, Seattle, Washington 98105, USA · University of Washington School of Medicine, Seattle, Washington 98105, USA · Microsoft Health AI, Redmond, Washington 98052-6399, USA
Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis
Apr 17, 2026Ayhan Can Erdur, Daniel Scholz, Jiazhen Pan +3Medical Image Analysis Longitudinal Medical Image Analysis
Department of Radiation Oncology, TUM University Hospital, Munich, Germany · Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Chair for AI for Image-Guided Diagnosis and Therapy, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany +3

Jun 9, 2026J/K move · Enter open · S save

Multimodal Brain Tumour Classification Using Feature Fusion

Wajih ul Islam, Muhammad Yaqoob, Javed Ali Khan, Volker Steuber

School of Physics, Engineering and Computer Science, University of Hertfordshire, UK

Multimodal Classification Medical Image Classification Oncology Brain Tumor Classification Multimodal Fusion

Open paper AskPDF arXiv

Clinicians diagnose brain tumors by synthesizing patient symptoms, medical history, and quantitative imaging data from modalities such as MRI and CT scans into a unified clinical judgement. However, most deep learning models rely on MRI/CT images alone, failing to replicate the clinicians multimodal reasoning. We explore a two-branch multimodal network combining raw MRI scans with 91 extracted radiomic features (intensity, texture, shape, and boundary descriptors) to classify brain tumors into glioma, meningioma, pituitary, and no-tumor. A pre-trained CNN backbone encodes the image stream, whereas a dedicated MLP encodes the radiomic stream. Both streams are fused via concatenation, gated, or bidirectional cross-modal attention strategies. Across nine experimental runs on a balanced 7,200 image dataset, all multimodal configurations outperform unimodal baselines with gated fusion achieving the best accuracy of 96.13%.

Abstract

Brain Tumor Classification

Explore similar work

Multimodal Brain Tumour Classification Using Feature Fusion

CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing

Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis