3D MRI-Based Alzheimer's Disease Classification Using Multi-Modal 3D CNN with Leakage-Aware Subject-Level Evaluation

Md Sifat; Sania Akter; Akif Islam; Md. Ekramul Hamid; Abu Saleh Musa Miah; Najmul Hassan; Md Abdur Rahim; Jungpil Shin

cs.CVMar 18, 2026

3D MRI-Based Alzheimer's Disease Classification Using Multi-Modal 3D CNN with Leakage-Aware Subject-Level Evaluation

Authors: Md Sifat, Sania Akter, Akif Islam, Md. Ekramul Hamid, Abu Saleh Musa Miah, Najmul Hassan, Md Abdur Rahim, Jungpil Shin

Organizations: Department of Computer Science and Engineering, University of Rajshahi, Rajshahi, Bangladesh · School of Computer Science and Engineering, University of Aizu, Aizuwakamatsu, Japan · Department of Computer Science and Engineering, Pabna University of Science and Technology, Pabna, Bangladesh

Abstract

Deep learning has become an important tool for Alzheimer's disease (AD) classification from structural MRI. Many existing studies analyze individual 2D slices extracted from MRI volumes, while clinical neuroimaging practice typically relies on the full three dimensional structure of the brain. From this perspective, volumetric analysis may better capture spatial relationships among brain regions that are relevant to disease progression. Motivated by this idea, this work proposes a multimodal 3D convolutional neural network for AD classification using raw OASIS 1 MRI volumes. The model combines structural T1 information with gray matter, white matter, and cerebrospinal fluid probability maps obtained through FSL FAST segmentation in order to capture complementary neuroanatomical information. The proposed approach is evaluated on the clinically labelled OASIS 1 cohort using 5 fold subject level cross validation, achieving a mean accuracy of 72.34% plus or minus 4.66% and a ROC AUC of 0.7781 plus or minus 0.0365. GradCAM visualizations further indicate that the model focuses on anatomically meaningful regions, including the medial temporal lobe and ventricular areas that are known to be associated with Alzheimer's related structural changes. To better understand how data representation and evaluation strategies may influence reported performance, additional diagnostic experiments were conducted on a slice based version of the dataset under both slice level and subject level protocols. These observations help provide context for the volumetric results. Overall, the proposed multimodal 3D framework establishes a reproducible subject level benchmark and highlights the potential benefits of volumetric MRI analysis for Alzheimer's disease classification.

Figures & tables

CDR	Label	N	Mean Age	Mean MMSE	Male	Female
0.0	Non-demented	135	69.07	29.10	38	97
0.5	Very mild AD	70	76.21	25.64	31	39
1.0	Mild AD	28	77.75	21.68	9	19
2.0	Moderate AD	2	82.00	15.00	1	1

TABLE I: Distribution of the Clinically Labelled OASIS-1 Cohort

Fig. 1: Overview of the proposed multi-modal 3D CNN architecture, where modality-specific encoders process T1, GM, WM, and CSF inputs independently and their learned representations are fused through a shared late-fusion classification head for binary dementia classification.

Class	Unique Subjects	Total Slices
Non-demented	285	67,222
Very mild dementia	58	13,725
Mild dementia	21	5,002
Moderate dementia	2	488
Total	366	86,437

TABLE II: Distribution of the Kaggle-Derived OASIS-1 Slice Dataset

Fig. 2: Preprocessed multi-channel MRI input. Channel 0: bias-corrected T1 image; Channels 1–3: GM, WM, and CSF probability maps from FSL FAST segmentation.

Fold	Accuracy (%)	ROC-AUC
Fold 1	76.6	0.7759
Fold 2	72.3	0.7926
Fold 3	63.8	0.7093
Fold 4	76.6	0.8130
Fold 5	72.3	0.8000
Mean	72.34 $\pm$ 4.66	0.7781 $\pm$ 0.0365

TABLE III: Per-Fold and Mean Performance of the Proposed Multi-Modal 3D CNN

Panel of Figure 6 — Fig. 3: Qualitative comparison between an input MRI volume and its corresponding GradCAM visualization in the proposed multi-modal 3D framework.

Fig. 4: Training and validation loss curves across the five subject-level cross-validation folds of the proposed multi-modal 3D CNN. The dashed vertical line in each panel marks the epoch of the selected best checkpoint for that fold.

Panel of Figure 8 — Fig. 5: Slice-level evaluation: input MRI slice and corresponding GradCAM map.

Panel of Figure 9 — Fig. 6: Subject-level evaluation: input MRI slice and corresponding GradCAM map.

Study	Dataset	Model	Input / Split	Acc. (%)
Tufail et al. [ 11 ]	OASIS-1 (416)	InceptionV3, Xception	2D slices, subject-level 5-fold CV	63–65
Yagis et al. [ 12 ]	OASIS-1 (200)	VGG16, VGG19, ResNet-18	2D slices, subject-level split	66
Ours	OASIS-1 (235)	Multi-modal 3D CNN	3D volumes, 5-fold subject CV	72.34

TABLE IV: Comparison with subject-level studies on OASIS-1. Results are not directly comparable across rows due to differences in cohort size, subject demographics, and binary grouping criteria.

Alzheimer's Disease Classification

Alzheimer’s disease classification uses clinical, behavioral, and brain-imaging data to distinguish disease stages and predict progression.

See all 92 papers

A Multimodal Explainable Deep Learning Framework for Alzheimer's Disease Diagnosis using 3D Magnetic Resonance Imaging and Clinical Data
Sep 11, 2026Yusuf Brima, Marcellin Atemkeng, Lakshmana Rao Namamula +13D Medical Imaging Multimodal Classification
Computer Vision, Institute of Cognitive Science, Osnabrück University, Germany. · Department of Mathematics, Rhodes University, South Africa. · National Institute for Theoretical and Computational Sciences (NITheCS), Stellenbosch, 7600, South Africa. +2
Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI
Sep 14, 2026Paul-Gabriel Nicolae, Irina Georgiana MocanuData Leakage Alzheimer's Disease Classification
Computer Science and Engineering Department, Faculty of Automatic Control and Computers, National University of Science and Technology POLITEHNICA Bucharest, Romania
Multimodal Deep Learning for Dynamic and Static Neuroimaging: Integrating MRI and fMRI for Alzheimer Disease Analysis
Mar 9, 2026Anima Kujur, Zahra MonfaredAlzheimer's Disease Classification Neuroimaging
Interdisciplinary Centre for Scientific Computing (IWR), Department of Mathematics and Computer Science, Heidelberg University, Germany

Sep 11, 2026J/K move · Enter open · S save

A Multimodal Explainable Deep Learning Framework for Alzheimer's Disease Diagnosis using 3D Magnetic Resonance Imaging and Clinical Data

Yusuf Brima, Marcellin Atemkeng, Lakshmana Rao Namamula, Antoine Vacavant

Computer Vision, Institute of Cognitive Science, Osnabrück University, Germany. · Department of Mathematics, Rhodes University, South Africa. · National Institute for Theoretical and Computational Sciences (NITheCS), Stellenbosch, 7600, South Africa.+2

3D Medical Imaging Multimodal Classification Alzheimer's Disease Classification Explainable Medical Image Analysis Multimodal Model Interpretability

Open paper AskPDF arXiv

Dementia is a major and growing global health burden, with Alzheimer's disease (AD) accounting for most cases. Timely and accurate diagnosis is central to managing this burden and increasingly depends on integrating complementary clinical and imaging information. Multimodal deep learning can combine these modalities for AD diagnosis, but how its explanations behave across modalities, fusion strategies, and cohorts remains unclear. We developed an explainable multimodal framework pairing a 3D CNN encoder for T1-weighted MRI with a feedforward network for harmonized clinical and demographic data, comparing varied model setups on three-way and pairwise diagnostic tasks using 6,479 internal records from the ADNI and 1,703 independent records from the OASIS-3. On ADNI, the tabular-only model achieved the highest three-class AUC-ROC of 0.879 and best discriminated cognitively normal (CN) versus mild cognitive impairment (MCI; 0.903), while cross-attention performed best for MCI versus AD (0.861); CN versus AD was highly discriminative overall. On OASIS-3, the vision-only model performed best (three-class AUC-ROC 0.910); CN versus MCI remained difficult, and no fusion strategy consistently outperformed single modalities across tasks and cohorts. SHAP and Integrated Gradients identified the MMSE as the dominant tabular feature in both cohorts, with global feature rankings agreeing strongly in ADNI (

ρ=0.94

) and OASIS-3 (

ρ=0.96

); CAM-based explanations, however, changed with model configuration and cohort. These findings show that multimodal performance and explanations are task, modality, fusion, and cohort-dependent: a dominant cognitive signal persisted across cohorts, but feature contributions and CAM explanations did not, underscoring the need to evaluate explainability under cohort shift rather than as a stable, intrinsic property.

Abstract

Alzheimer's Disease Classification

Explore similar work

A Multimodal Explainable Deep Learning Framework for Alzheimer's Disease Diagnosis using 3D Magnetic Resonance Imaging and Clinical Data

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

Multimodal Deep Learning for Dynamic and Static Neuroimaging: Integrating MRI and fMRI for Alzheimer Disease Analysis