HybridDualPathNet: Cross-Fusion Attention of EfficientNetV2 and Swin Transformer for Brain MRI Classification of Alzheimer's and Parkinson's Disease

Jarpula Nirjala; Rahul Patel

cs.CVOct 12, 2026

HybridDualPathNet: Cross-Fusion Attention of EfficientNetV2 and Swin Transformer for Brain MRI Classification of Alzheimer's and Parkinson's Disease

Authors: Jarpula Nirjala, Rahul Patel

Organizations: Department of Electronics and Communication Engineering, Indian Institute of Information Technology, Surat, India

Abstract

Separating Alzheimer's Disease (AD), Parkinson's Disease (PD), and Healthy Controls (HC) using structural MRI remains challenging because disease signatures overlap and model performance can be strongly affected by data partitioning. We present HybridDualPathNet, a hybrid architecture combining EfficientNetV2-S for local features and Swin-Tiny for global context through a Cross-Fusion Attention (CFA) module. We evaluate the model in two experiments. First, on the Kaggle APD-3-Class dataset using an image-level split, the model achieves 98.25% accuracy. Second, on a combined Kaggle and Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset containing 17,859 slices, we use a per-source split with patient-wise separation for ADNI subjects. In this setting, the model achieves 73.37% standard accuracy and a macro AUC of 0.880. Source-stratified evaluation reveals a substantial domain gap, with 91.2% accuracy on Kaggle test images and 59.2% on ADNI test images. Temperature scaling reduces expected calibration error from 0.130 to 0.015, while split-conformal prediction achieves 89.5% coverage. Saliency analyses using Grad-CAM++, Integrated Gradients, and occlusion indicate that model activations concentrate on brain tissue. These results highlight the importance of patient-level separation, cross-dataset evaluation, and uncertainty estimation when assessing MRI classification systems. The findings also show that strong performance on a single dataset does not necessarily translate to reliable generalization across imaging cohorts.

Figures & tables

Fig. 1: HybridDualPathNet. Two pretrained encoders feed the Cross-Fusion Attention module, whose per-sample sigmoid gate $\gamma$ blends local and global features before a shallow classifier. The auxiliary heads add deep supervision only while training.

Fig. 2: Kaggle-only class distribution. PD is roughly 14% of the 6,477 images, motivating the balanced sampler and class-weighted focal loss.

Fig. 3: Representative Kaggle axial T1 slices: AD (top) with wider ventricles and temporal atrophy, CN (middle) with intact folding, and PD (bottom).

Split	AD	CN	PD	Total
Train (70%)	1,792	2,107	634	4,533
Validation (15%)	384	452	136	972
Test (15%)	385	451	136	972
Full	2,561	3,010	906	6,477

TABLE I: Kaggle-only dataset and stratified image-level split.

Fig. 4: Kaggle-only training and validation curves. Best checkpoint at epoch 82 (val acc 97.84%); early stopping at epoch 97.

Epoch	Tr Loss	Tr Acc	Val Acc	Val F1
6	0.279	83.25%	85.58%	0.865
19	0.196	93.44%	93.20%	0.932
40	0.150	95.97%	97.22%	0.967
82*	0.129	92.90%	97.84%	0.976

TABLE II: Kaggle-only training milestones.

Metric	Value
Test accuracy (7-pass TTA)	98.25%
Standard accuracy (no TTA)	98.46%
Macro F1	0.984
Cohen’s $\kappa$	0.974
Mean AUC-ROC	0.9991
AUC (AD / CN / PD)	0.999 / 0.998 / 1.000

TABLE III: Kaggle-only final test metrics (

N=972

, 7-pass TTA).

Fig. 5: Kaggle-only per-class precision, recall, and F1 ( $N=972$ , TTA). Every class exceeds F1 = 0.981; PD posts the highest recall.

Class	Precision	Recall	F1	Support
AD	0.989	0.974	0.982	385
CONTROL	0.976	0.987	0.981	451
PD	0.985	0.993	0.989	136
Macro	0.983	0.985	0.984	972

TABLE IV: Kaggle-only per-class results (

N=972

, TTA).

Fig. 6: Kaggle-only confusion matrices (counts and normalized), $N=972$ . Per-class recall: AD 97.4%, CN 98.7%, PD 99.3%.

True $\backslash$ Pred	AD	CN	PD
AD (385)	375	10	0
CN (451)	4	445	2
PD (136)	0	1	135

TABLE V: Kaggle-only confusion matrix (

N=972

, TTA).

Fig. 7: Kaggle-only one-vs-rest ROC. All AUC $\geq$ 0.998; macro 0.9991.

Fig. 8: PCA (left, 87.4% variance) and t-SNE (right) of the Kaggle-only fused embeddings. The three classes form separable clusters.

Configuration	Acc (%)	F1
EfficientNetV2-S only	95.37	0.951
Swin-Tiny only	94.03	0.938
Concat fusion (no CFA)	96.91	0.967
CFA (no aux)	97.12	0.969
CFA + aux (no TTA)	97.84	0.976
Full + TTA	98.25	0.984

TABLE VI: Kaggle-only ablation (accuracy / F1).

Fig. 9: Kaggle-only comparison with published methods on accuracy, F1, and AUC.

Method	Acc (%)	F1	AUC
VGG-16	89.2	0.881	0.943
ResNet-50 [ 4 ]	91.5	0.908	0.957
DenseNet-121	92.8	0.921	0.964
EfficientNet-B0 [ 5 ]	94.1	0.935	0.973
ViT-B/16 [ 7 ]	94.4	0.937	0.975
Swin-T [ 8 ]	92.2	0.915	0.967

TABLE VII: Kaggle-only comparison with published methods.

Fig. 10: Kaggle-only reliability diagram and confidence distribution. Correct predictions sit at higher confidence than errors.

Fig. 11: Kaggle-only Grad-CAM++ maps. AD activates the hippocampus and medial temporal lobe; PD the putamen and substantia nigra; CN stays diffuse.

Source	AD	CN	PD	Total
Kaggle	3,200	3,672	967	7,839
ADNI	5,040	4,980	0	10,020
Combined	8,240	8,652	967	17,859
Patient-wise split: Train 12,495 / Val 2,611 / Test 2,753, 0 overlap.
Held-out test: AD 1,025 / CN 1,593 / PD 135 (Kaggle 1,173, ADNI 1,580).

TABLE VIII: Combined dataset composition and patient-wise split.

Fig. 12: Representative combined-cohort slices per class. ADNI scans differ visibly from Kaggle ones in intensity and field of view—the source of the domain gap we quantify.

Class	Precision	Recall	F1	AUC
AD	0.596	0.925	0.725	0.824
CONTROL	0.923	0.579	0.712	0.820
PD	0.822	0.993	0.899	0.997
Macro	0.780	0.832	0.779	0.880

TABLE IX: Combined held-out test results (

N=2{,}753

, TTA).

Fig. 13: Combined patient-wise confusion matrix. The dominant error is Control predicted as AD; AD and PD are almost never mistaken for each other.

Fig. 14: Combined one-vs-rest ROC. PD is near-perfectly separable (AUC 0.997); AD and CN are the hard pair under the leak-free protocol.

Source ( $n$ )	Accuracy	AD recall	CN recall
Kaggle (1,173)	91.2%	0.873	0.929
ADNI (1,580)	59.2%	0.975	0.404

TABLE X: Source-stratified test performance (one combined model).

MC-Dropout: 73.3% overall, 76.3% on 90% most-confident.
Setting	ECE	MCE	Brier
Before temperature scaling	0.130	0.146	0.261
After ( $T=0.60$ )	0.015	0.085	0.235
Conformal ( $\alpha{=}0.10$ ): coverage 89.5% $\pm$ 0.8%, set size 1.49.

TABLE XI: Calibration and uncertainty on the combined test set.

Fig. 15: Combined-study reliability diagrams before and after temperature scaling; after scaling the curve tracks the diagonal closely.

Fig. 16: Combined-study Grad-CAM++ overlays for AD, CN, and PD. Activation concentrates on brain tissue rather than skull or background, consistent with the quantitative bias check.

Explore similar work

Evidence Before Accuracy: A MRI-PET Fusion Network for Alzheimer Disease Classification with Causal Regional Validation
Sep 28, 2026Saeid Firouzi Daghigh, Saeed AyatAlzheimer's Disease Classification Explainable Medical Image Analysis
Department of Computer Engineering and Information Technology, Payame Noor University, Tehran, Iran
Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI
Sep 14, 2026Paul-Gabriel Nicolae, Irina Georgiana MocanuData Leakage Alzheimer's Disease Classification
Computer Science and Engineering Department, Faculty of Automatic Control and Computers, National University of Science and Technology POLITEHNICA Bucharest, Romania
A comparison of CNN architectures for Alzheimer's disease detection in single-view MRI scans
Aug 12, 2026Hiram Zuniga, Ulises Orozco-Rosas, Kenia PicosAlzheimer's Disease Classification Convolutional Neural Networks
CETYS Universidad, Ave. CETYS Universidad No. 4, Fracc. El Lago, C.P. 22210, Tijuana, Baja California, Mexico

Sep 28, 2026J/K move · Enter open · S save

Evidence Before Accuracy: A MRI-PET Fusion Network for Alzheimer Disease Classification with Causal Regional Validation

Saeid Firouzi Daghigh, Saeed Ayat

Department of Computer Engineering and Information Technology, Payame Noor University, Tehran, Iran

Alzheimer's Disease Classification Explainable Medical Image Analysis Multimodal Fusion Brain MRI

Open paper AskPDF arXiv

Deep learning models for Alzheimer disease (AD) classification routinely report near-perfect discrimination, yet few are shown to rest on AD-relevant neurobiology rather than on dataset artifacts, subject-level leakage, or non-brain image content. We present a fusion network combining T1 MRI and FDG PET across axial, coronal, and sagittal planes, trained on ADNI consists of 554 paired subjects. The fusion model reaches AUC 0.962, accuracy 0.909, and F1 0.891, competitive with recent 3D CNN and multimodal transformer systems at substantially lower cost. We first quantify how much modality, plane and slice geometry matter. A validation-only search over slice centres and neighbour spacings moves AUC by 0.180 for MRI and 0.078 for PET, selecting narrow spacing for MRI and wide spacing for PET, with the chosen coronal centres falling on the hippocampal body and on the posterior cingulate respectively. The contribution, however, is the evidence layer built around that number. Shortcut controls collapse the model to AUC 0.622 (silhouette), 0.608 (exterior), and 0.500 (blank), and a label-permutation null yields 0.456. Forward region-of-interest (ROI) ablation shows that masking medial temporal cortex in MRI and the posterior default-mode network (DMN) in PET produces the largest shift in the AD logit, while area-matched controls remain indistinguishable from that null. Reverse ROI ablation shows that the medial temporal lobe alone retains 89.2% of above-chance discrimination in MRI and the posterior DMN alone retains 79.0% in PET. A quantitative comparison of attribution methods shows occlusion sensitivity reaching 3.5-5.0* enrichment inside a priori AD regions against 0.10-0.43* in controls. Ablation and attribution independently establish a biologically correct double dissociation: hippocampal evidence is carried by MRI, posterior cingulate evidence by PET.

Abstract

Explore similar work

Evidence Before Accuracy: A MRI-PET Fusion Network for Alzheimer Disease Classification with Causal Regional Validation

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

A comparison of CNN architectures for Alzheimer's disease detection in single-view MRI scans