XMorph: Explainable Brain Tumor Analysis Via LLM-Assisted Hybrid Deep Intelligence

Sepehr Salem Ghahfarokhi; M. Moein Esfahani; Raj Sunderraman; Vince Calhoun; Mohammed Alser

cs.CVFeb 24, 2026

XMorph: Explainable Brain Tumor Analysis Via LLM-Assisted Hybrid Deep Intelligence

Authors: Sepehr Salem Ghahfarokhi, M. Moein Esfahani, Raj Sunderraman, Vince Calhoun, Mohammed Alser

Organizations: Department of Computer Science, Georgia State University, Atlanta, GA, USA · TReNDS Center, Georgia State University, Atlanta, GA, USA

Abstract

Deep learning has significantly advanced automated brain tumor diagnosis, yet clinical adoption remains limited by interpretability and computational constraints. Conventional models often act as opaque ''black boxes'' and fail to quantify the complex, irregular tumor boundaries that characterize malignant growth. To address these challenges, we present XMorph, an explainable and computationally efficient framework for fine-grained classification of three prominent brain tumor types: glioma, meningioma, and pituitary tumors. We propose an Information-Weighted Boundary Normalization (IWBN) mechanism that emphasizes diagnostically relevant boundary regions alongside nonlinear chaotic and clinically validated features, enabling a richer morphological representation of tumor growth. A dual-channel explainable AI module combines GradCAM++ visual cues with LLM-generated textual rationales, translating model reasoning into clinically interpretable insights. The proposed framework achieves a classification accuracy of 96.0%, demonstrating that explainability and high performance can co-exist in AI-based medical imaging systems. The source code and materials for XMorph are all publicly available at: https://github.com/ALSER-Lab/XMorph.

Figures & tables

Fig. 1: The end-to-end pipeline of the proposed XMorph framework for explainable brain tumor classification, illustrating the six-stage workflow. Stage 1 (Automated Tumor Segmentation): An input MRI is processed by a DeepLabV3 model to generate a tumor mask and extract the ROI (tumor boundary). Stage 2 (Tumor Specific Features Extraction): A multi-faceted feature set is extracted. The tumor boundary is converted to a time series to compute nonlinear chaotic descriptors (e.g., Entropies, Fractal Dimension), features from our novel Information-Weighted Boundary Normalization (IWBN), and quantitative clinical biomarkers. Stage 3 (Deep Features Extraction): In a parallel stream, a pre-trained ResNet-50 extracts high-level textural features from the input MRI. Stage 4 (Features Fusion): The Tumor Specific and deep feature sets are concatenated into a single feature vector. Stage 5 (Classification): An XGBoost classifier uses the fused vector to predict the tumor class (Glioma, Meningioma, or Pituitary). Stage 6 (Dual-Channel Explainability): The framework provides two complementary explanations for transparency. A visual GradCAM++ heatmap, generated from the deep feature model, shows where the model focuses. A textual rationale from an LLM explains why the final XGBoost classifier made its decision by interpreting the most influential features from the fused set, including the novel chaotic and clinical descriptors.

Tumor Type	Dice Score	IoU Score	Precision	Recall	F1 Score
Glioma	$0.913\pm 0.130$	$0.860\pm 0.155$	$0.914\pm 0.131$	$0.924\pm 0.136$	$0.913\pm 0.130$
Meningioma	$0.928\pm 0.105$	$0.882\pm 0.128$	$0.929\pm 0.107$	$0.938\pm 0.109$	$0.928\pm 0.105$
Pituitary	$0.940\pm 0.080$	$0.890\pm 0.099$	$0.937\pm 0.085$	$0.948\pm 0.085$	$0.940\pm 0.080$
Non-Tumor	$0.958\pm 0.068$	$0.923\pm 0.085$	$0.959\pm 0.076$	$0.959\pm 0.068$	$0.958\pm 0.068$
Overall	$\mathbf{0.932\pm 0.104}$	$\mathbf{0.885\pm 0.127}$	$\mathbf{0.932\pm 0.106}$	$\mathbf{0.940\pm 0.108}$	$\mathbf{0.932\pm 0.104}$

TABLE I: Performance metrics for brain tumor segmentation.

Fig. 2: Example of brain tumor segmentation results showing original MRI, masks, and differences.

Metric	DeepLabV3	U-Net
Dice Score	93.21% $\pm$ 0.104%	89.72% $\pm$ 0.146%
IoU Score	88.50% $\pm$ 0.126%	83.42% $\pm$ 0.160%
Precision	93.22% $\pm$ 0.106%	90.91% $\pm$ 0.145%
Recall	94.00% $\pm$ 0.108%	89.98% $\pm$ 0.157%
F1 Score	93.20% $\pm$ 0.104%	89.71% $\pm$ 0.146%

TABLE II: Performance comparison between DeepLabV3 and U-Net.

Fig. 3: The boundary-to-signal pipeline reveals distinct morphological signatures across different tumor types. The pipeline converts 2D tumor boundaries (Column 2) into 1D normalized time-series signals (Column 5). Each row illustrates a representative clinical case, demonstrating the direct relationship between visual morphology and the resulting signal characteristics. (a) Pituitary Tumor: A visually regular, well-circumscribed boundary translates into a smooth, low-amplitude signal, quantitatively confirmed by a low Irregularity Index (STD = 0.142). (b) Meningioma: A characteristic lobulated boundary produces a signal with more pronounced variations, reflecting an intermediate structural complexity (Irregularity Index = 0.157). (c) Glioma: A highly infiltrative and poorly defined malignant boundary generates a chaotic, non-periodic signal with sharp, high-amplitude oscillations, corresponding to a significantly higher Irregularity Index (STD = 0.253). This clear monotonic progression from a smooth to a chaotic signal provides strong visual evidence that the pipeline robustly captures clinically relevant morphological differences.

Fig. 4: Visual Validation of IWBN across Tumor Types. This figure illustrates how our novel IWBN method identifies and emphasizes regions of high morphological complexity. Each column represents a different tumor class, with the top row showing the final Information Weights and the bottom row showing the underlying Local Entropy that generated them. Top Row (Information Weights): Visualizes the amplification factor applied to each point on the boundary. Hotter colors (red/yellow) indicate higher weights, signifying regions of greater diagnostic interest. Bottom Row (Local Entropy): Maps the intrinsic complexity of the boundary. Brighter colors (yellow) correspond to segments with higher irregularity and unpredictability. Pituitary: The benign, regular boundary exhibits uniformly low entropy, resulting in low and evenly distributed weights. The method correctly identifies this as a simple structure. Meningioma: The method accurately pinpoints the lobulated portion of the boundary as a localized region of high entropy, assigning it a correspondingly high weight. Glioma: The malignant, infiltrative boundary shows widespread, high-magnitude entropy. The IWBN method correctly assigns high weights to these numerous chaotic segments, effectively highlighting the features of malignancy.

Fig. 5: Visualization of three key radiological biomarkers across tumor types: (Left) Ring Enhancement Index (core vs. rim intensity), (Center) Midline Shift (centroid deviation from the anatomical midline), and (Right) Skull-to-Tumor Distance (minimum distance and contact ratio). The figure demonstrates increasing enhancement irregularity and deformation gradient from pituitary $\rightarrow$ meningioma $\rightarrow$ glioma, reflecting both vascular and spatial indicators of malignancy.

Class	Irregularity	Roughness	Area	Mean Radius	Mean Local Entropy	Weight Range	Enh. Factor
Glioma	0.1801 $\pm$ 0.0936	3.001 $\pm$ 1.0483	5778.9 $\pm$ 4044.0	41.4 $\pm$ 15.4	0.7563 $\pm$ 0.3399	2.99 $\pm$ 0.65	5.28 $\pm$ 2.87
Meningioma	0.1215 $\pm$ 0.0632	2.443 $\pm$ 0.6120	4650.0 $\pm$ 3128.7	36.7 $\pm$ 12.9	0.2899 $\pm$ 0.2939	3.39 $\pm$ 0.58	7.19 $\pm$ 3.25
Pituitary	0.1435 $\pm$ 0.0738	2.750 $\pm$ 0.6749	2163.9 $\pm$ 1633.0	25.1 $\pm$ 8.8	0.1901 $\pm$ 0.2301	3.14 $\pm$ 0.53	6.19 $\pm$ 3.25

TABLE III: Summary of morphological and information-weighted features across tumor classes. Higher roughness and irregularity in gliomas reflect diffuse infiltration, while higher enhancement and weight range in meningiomas indicate stronger local heterogeneity captured by the IWBN method.

Stage	Configuration / Parameters
Segmentation	DeepLabV3 (ResNet-50), Input $256\times 256$ , Batch 16 Optimizer: Adam ( $lr=10^{-4}$ ), Loss: $L_{CE}+L_{Dice}$
Specific Features	Boundary $N=256$ , IWBN $\alpha=0.5$ , REI $\epsilon=10^{-6}$ Entropy ( $m=2,r=0.2\sigma$ ), Fractal Dim. (Box-counting)
Deep Features	ResNet-50 (ImageNet), Global Avg Pooling, PCA (95% var)
Classification	XGBoost: 300 estimators, Max Depth 8, Learning Rate 0.05

TABLE IV: Detailed hyperparameter configuration for XMorph modules.

Feature Configuration	Accuracy	Specificity	Sensitivity
Tumor Specific Features (XGBoost)	$0.900\,\pm\,0.013$	$0.950\,\pm\,0.007$	$0.895\,\pm\,0.012$
Deep Features (ResNet-50 + XGBoost)	$0.930\,\pm\,0.008$	$0.962\,\pm\,0.005$	$0.918\,\pm\,0.010$
Fused Deep + Tumor Specific Features	$\mathbf{0.960\,\pm\,0.010}$	$\mathbf{0.983\,\pm\,0.005}$	$\mathbf{0.962\,\pm\,0.012}$

TABLE V: Comparison of classification performance (mean

\pm

standard deviation) across feature configurations using five-fold cross-validation.

Fig. 6: ROC curves comparing the performance of Tumor Specific, deep, and fused feature configurations for brain tumor classification. The fusion model demonstrates a consistently higher true positive rate across all tumor classes, indicating improved sensitivity–specificity balance. The top panels illustrate per-class ROC curves for glioma, meningioma, and pituitary tumors, while the bottom panels present macro-averaged ROC, AUC comparisons, and performance summaries.

Fig. 7: Dual-channel explainability results highlighting the role of chaotic and information-weighted features. Each row shows: (1) the original MRI, (2) the extracted tumor mask, (3) GradCAM++ visualization of salient regions, and (4) the LLM-generated explanation derived from fused SHAP-weighted features. The pituitary case (top) demonstrates low boundary entropy and minimal mass effect, while the glioma case (bottom) reflects high local entropy, large radius, and heterogeneous enhancement-patterns amplified by our chaotic feature extraction. Together, the spatial (GradCAM++) and semantic (LLM) channels reveal how nonlinear boundary irregularity and entropy-based cues contribute to diagnostic reasoning, translating complex quantitative patterns into human-interpretable clinical insight.

Feature / Capability	[ 4 ]	[ 19 ]	[ 7 ]	[ 9 ]	[ 13 ]	[ 14 ]	[ 8 ]	XMorph
Deep Feature Learning	✓	✓	✓	✓	✓	✓	✓	✓
Fractal Dimension (FD)	✗	✗	✗	✗	✓	✓	✗	✓
Chaotic Metrics (ApEn, LE)	✗	✗	✗	✗	✗	✗	✗	✓
IWBN (Boundary Enhancement)	✗	✗	✗	✗	✗	✗	✗	✓
Clinical Biomarkers (REI, MLS, Dskull)	✗	✗	✗	✗	✗	✗	✗	✓
Visual XAI (Heatmaps)	✗	✗	✓	✓	✓	✗	✗	✓

TABLE VI: Capability comparison of XMorph vs. state-of-the-art approaches.

Brain Tumor Classification

Brain tumor classification uses medical images and molecular data to identify tumor types, grades, and subtypes for diagnosis and treatment planning.

See all 51 papers

Bridging visual saliency and large language models for explainable deep learning in medical imaging
May 7, 2026Paul Valery Nguezet, Elie Tagne Fute, Yusuf Brima +2Radiology Report Generation Medical Image Classification
Department of Mathematics and Computer science, University of Dschang, Dschang, 67, Cameroon. · Computer Vision, Institute of Cognitive Science, Osnabrück University, Osnabrueck, D-49090, Lower Saxony, Germany. · Department of Mathematics, Rhodes University, Grahamstown, Eastern Cape, 6140, Eastern Cape, South Africa. +1
TumorXAI: Self-Supervised Deep Learning Framework for Explainable Brain MRI Tumor Classification
May 3, 2026Abrar Hossain Zahin, Amit Kumar Saha, Tanvir Mridha +5Self-Supervised Learning Medical Image Classification
Department of Computer Science and Engineering, East West University, Dhaka, 1212, Bangladesh.
an interpretable vision transformer framework for automated brain tumor classification
Apr 23, 2026Chinedu Emmanuel Mbonu, Tochukwu Sunday Belonwu, Okwuchukwu Ejike Chukwuogo +1Vision Transformer Medical Image Classification
Department of Computer Science, Nazarbayev University, Astana · Department of Computer Science, Nnamdi Azikiwe University, Nigeria

May 7, 2026J/K move · Enter open · S save

Bridging visual saliency and large language models for explainable deep learning in medical imaging

Paul Valery Nguezet, Elie Tagne Fute, Yusuf Brima, Benoit Martin Azanguezet+1

Department of Mathematics and Computer science, University of Dschang, Dschang, 67, Cameroon. · Computer Vision, Institute of Cognitive Science, Osnabrück University, Osnabrueck, D-49090, Lower Saxony, Germany. · Department of Mathematics, Rhodes University, Grahamstown, Eastern Cape, 6140, Eastern Cape, South Africa.+1

Radiology Report Generation Medical Image Classification Explainable Medical Image Analysis Brain Tumor Classification Visual Saliency Medical Image Localization+1

Open paper AskPDF arXiv

The opaque nature of deep learning models remains a significant barrier to their clinical adoption in medical imaging. This paper presents a multimodal explainability framework that bridges the gap between convolutional neural network (CNN) predictions and clinically actionable insights for brain tumor classification, leveraging large language models (LLMs) to deliver human-interpretable diagnostic narratives. The proposed framework operates through three coupled stages. First, nine CNN architectures are extended with a dual-output hybrid formulation that simultaneously optimises a classification head and a segmentation head, enabling spatially richer feature learning. Second, visual saliency attribution methods, namely Grad-CAM, Grad-CAM++, and ScoreCAM, are applied to generate class-discriminative heatmaps, which are subsequently refined into binary tumor masks via an adaptive percentile thresholding pipeline. Third, the resulting masks are mapped onto the Harvard-Oxford cortical atlas to translate pixel-level evidence into named neuroanatomical structures, and the extracted findings are encoded into a structured JSON file that conditions three LLMs (Grok3, Mistral, and LLaMA) to generate coherent, radiological-style diagnostic reports. Evaluated on a dataset of 4,834 contrast-enhanced T1-weighted brain MRI images spanning three tumor classes, InceptionResNetV2 achieved the highest classification performance and Grad-CAM++ yielded the best segmentation overlap. Among the language models, Grok3 led in lexical diversity and coherence, while LLaMA achieved the highest readability score. By integrating visual, anatomical, and linguistic modalities into a unified pipeline, the framework produces explanations that are technically grounded and meaningfully interpretable, advancing the transparency and clinical accountability of artificial intelligence assisted brain tumor diagnosis.

Abstract

Brain Tumor Classification

Explore similar work

Bridging visual saliency and large language models for explainable deep learning in medical imaging

TumorXAI: Self-Supervised Deep Learning Framework for Explainable Brain MRI Tumor Classification

an interpretable vision transformer framework for automated brain tumor classification