A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection

Nojod M. Alotaibi; Areej M. Alhothali

cs.CVApr 11, 2026

A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection

Authors: Nojod M. Alotaibi, Areej M. Alhothali

Organizations: Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia

Abstract

Major depressive disorder (MDD) is a prevalent mental disorder associated with complex neurobiological changes that cannot be fully captured using a single imaging modality. The use of multimodal magnetic resonance imaging (MRI) provides a more comprehensive understanding of brain changes by combining structural and functional data. Despite this, the effective integration of these modalities remains challenging. In this study, we propose a dual cross-attention-based multimodal fusion framework that explicitly models bidirectional interactions between structural MRI (sMRI) and resting-state functional MRI (rs-fMRI) representations. The proposed approach is tested on the large-scale REST-meta-MDD dataset using both structural and functional brain atlas configurations. Numerous experiments conducted under a 10-fold stratified cross-validation demonstrated that the proposed fusion algorithm achieves robust and competitive performance across all atlas types. The proposed method consistently outperforms conventional feature-level concatenation for functional atlases, while maintaining comparable performance for structural atlases. The most effective dual cross-attention multimodal model obtained 84.71% accuracy, 86.42% sensitivity, 82.89% specificity, 84.34% precision, and 85.37% F1-score. These findings emphasize the importance of explicitly modeling cross-modal interactions for multimodal neuroimaging-based MDD classification.

Figures & tables

Group	Number of subjects	Male	Female	Average $\pm$ Standard deviation (%)
Group	Number of subjects	Male	Female	Age	Education
MDD	810	293	517	34.39 $\pm$ 11.55	11.96 $\pm$ 3.37
HC	753	307	446	34.61 $\pm$ 13.17	13.57 $\pm$ 3.42

Table 1: Demographic Information of the 1563 Study Subjects.

Figure 1: Overall architecture of the proposed dual cross-attention multimodal framework. sMRI, structural MRI; rs-fMRI, functional MRI; ROI, brain regions of interest; MLP, multilayer perceptron; GAT, graph attention network; LeakyReLU, leaky rectified linear unit; GAP, global average pooling; MDD, major depressive disorder; HC, healthy controls.

\mathbf{Z}_{ij}=\frac{1}{2}\ln\left(\frac{1+FCN_{ij}}{1-FCN_{ij}}\right)

Algorithm 2 Functional Graph Construction Process

Parameter	Value
Number of layers	2
Hidden dimension	64
Output dimension	2
Learning rate	$1\times 10^{-2}$
Weight decay	$1\times 10^{-4}$
Dropout	0.5

Table 2: Configuration of the Feature-Level Concatenation Model

Parameter	Value
Embedding dimension	64
Number of attention heads	8
Classification head	multilayer perceptron (MLP)
Learning rate	$5\times 10^{-4}$
Weight decay	$3\times 10^{-5}$
Dropout	0.3

Table 3: Configuration of the Dual Cross-Attention Fusion Model

Method		Average $\pm$ Standard deviation (%)
Method		Accuracy	Sensitivity	Specificity	Precision	F1-score
Concatenation	AAL-based model	75.74 $\pm$ 0.19	80.25 $\pm$ 0.00	70.92 $\pm$ 0.39	74.62 $\pm$ 0.25	77.33 $\pm$ 0.14
	HO-based model	78.53 $\pm$ 0.29	75.31 $\pm$ 0.00	81.97 $\pm$ 0.60	81.66 $\pm$ 0.50	78.36 $\pm$ 0.23
	Dose-based model	81.98 $\pm$ 0.58	80.99 $\pm$ 0.82	83.02 $\pm$ 0.71	83.57 $\pm$ 0.61	82.26 $\pm$ 0.59
	CK-based model	76.82 $\pm$ 1.07	76.42 $\pm$ 1.61	77.24 $\pm$ 2.36	78.20 $\pm$ 1.61	77.28 $\pm$ 0.98
Dual cross-attention	AAL-based model	75.54 $\pm$ 0.31	80.25 $\pm$ 0.87	70.52 $\pm$ 1.21	74.38 $\pm$ 0.62	77.20 $\pm$ 0.23

Table 4: Average Classification Results and Standard Deviation of 10-fold Cross-validation for Multimodal Fusion Models. Bolded values indicate the best performing results.

Method		Accuracy%	Sensitivity%	Specificity%	Precision%	F1-score%
Concatenation	AAL-based model	75.80	80.25	71.05	74.71	77.38
	HO-based model	78.98	75.31	82.89	82.43	78.71
	Dose-based model	82.17	81.48	82.89	83.54	82.50
	CK-based model	77.71	76.54	78.95	79.49	77.99
Dual cross-attention	AAL-based model	75.80	80.25	71.05	74.71	77.38
	HO-based model	78.98	75.31	82.89	82.43	78.71

Table 5: Classification Results for the Fold with the Highest Accuracy from 10-Fold Stratified Cross-Validation. Bolded values indicate the best performing results

Method		Average $\pm$ Standard deviation (%)
Method		Accuracy	Sensitivity	Specificity	Precision	F1-score
sMRI models	AAL-based model	74.52 $\pm$ 0.90	79.75 $\pm$ 1.85	68.95 $\pm$ 3.12	73.31 $\pm$ 1.66	76.36 $\pm$ 0.64
	HO-based model	77.77 $\pm$ 0.66	74.32 $\pm$ 2.05	81.45 $\pm$ 1.71	81.06 $\pm$ 1.12	77.52 $\pm$ 0.91
	Dose-based model	75.99 $\pm$ 1.18	81.24 $\pm$ 3.70	70.40 $\pm$ 5.80	74.77 $\pm$ 3.08	77.73 $\pm$ 0.69
	CK-based model	72.80 $\pm$ 1.28	75.06 $\pm$ 3.26	70.40 $\pm$ 3.07	73.05 $\pm$ 1.64	73.98 $\pm$ 1.53
rs-fMRI models	AAL-based model	65.29 $\pm$ 2.32	76.67 $\pm$ 7.46	53.16 $\pm$ 10.96	64.02 $\pm$ 3.50	69.41 $\pm$ 1.94

Table 6: Average Classification Results and Standard Deviation of 10-fold Cross-validation for Existing Multimodal Fusion Models. Bolded values indicate the best performing results.

Method		Accuracy	Sensitivity	Specificity	Precision	F1-score
Multimodal and sMRI models	AAL-based model	0.0048	0.4695	0.1751	0.0865	0.0017
	HO-based model	0.0294	0.0890	0.6959	0.9300	0.0295
	Dose-based model	0.0000	0.0965	0.0000	0.0000	0.0000
	CK-based model	0.0000	0.0547	0.0000	0.0000	0.0000
Multimodal and rs-fMRI models	AAL-based model	0.0000	0.1691	0.0002	0.0000	0.0000
	HO-based model	0.0000	0.5674	0.0000	0.0000	0.0000

Table 7: Statistical Analysis of Significant Differences (p-values) Between Single-Modal and Multimodal Approaches. Bolded values indicate statistically significant differences between models at

p<0.05

Explore similar work

M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification
Aug 31, 2026Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin +1Depression Detection Cross-Domain Generalization
School of Information and Communication Technology, Griffith University, Gold Coast, QLD, 4215, Australia
Multi-Source Multi-View Graph Domain Adaptation with Hyperbolic Residual Encoding for Cross-Site MDD Identification from rs-fMRI
Jul 31, 2026Zhanpeng Zheng, Xiran Chen, Haiteng Jiang +6Multi-Source Domain Adaptation Domain Adaptation
School of Computer and Artificial Intelligence, Shandong Jianzhu University, Shandong, China. · School of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing, China. · Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, Sichuan, China. +4
Multimodal Connectome Fusion via Cross-Attention for Autism Spectrum Disorder Classification Using Graph Learning
Mar 16, 2026Ansar Rahman, Hassan Shojaee-Mend, Sepideh HatamikiaMultimodal Graph Learning Multimodal Fusion
Department of Medicine, Danube Private University (DPU), Krems, Austria · Department of Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria · Department of Medical Informatics, Faculty of Medicine, Gonabad University of Medical Sciences, Gonabad, Iran +1

Aug 31, 2026J/K move · Enter open · S save

M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification

Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin, Alan Wee-Chung Liew

School of Information and Communication Technology, Griffith University, Gold Coast, QLD, 4215, Australia

Depression Detection Cross-Domain Generalization Neuroimaging Multimodal Disentangled Representation Learning Multimodal Fusion

Open paper AskPDF arXiv

Classification models based on resting-state functional magnetic resonance imaging (rs-fMRI) often show lower performance at imaging sites not included during model development, which can limit their use in clinical settings. Domain generalization (DG) addresses this issue by learning representations from source sites that remain effective for unseen target sites. However, existing DG approaches for psychiatric disorder classification commonly rely on a single imaging modality and may not fully account for site-specific acquisition effects on the learned representation space. Subjects scanned at the same site share scanner hardware, acquisition settings, and preprocessing characteristics, which can cause representations to reflect acquisition conditions rather than diagnostic information. In this work, we present M2LG-DG, a source-only multimodal local-global framework for cross-site major depressive disorder (MDD) classification. The framework employs a dual-stream rs-fMRI encoder, where the global pathway models inter-regional dependencies through self-attention and the local pathway performs graph-constrained aggregation over functional connectivity-derived brain graphs. Imaging and non-imaging representations are decomposed into shared and private components and integrated through bidirectional cross-attention with a learned modality gate. A cross-site supervised contrastive objective forms positive pairs from same-class subjects acquired at different source sites, encouraging the fused representation to preserve diagnostic information across acquisition domains. On four held-out REST-meta-MDD sites, M2LG-DG achieves an AUC of 69.48% and exceeds the closest comparison method by 2.18 percentage points. Experiments on the Autism Brain Imaging Data Exchange (ABIDE) dataset further support its applicability to other psychiatric neuroimaging classification tasks.

Abstract

Explore similar work

M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification

Multi-Source Multi-View Graph Domain Adaptation with Hyperbolic Residual Encoding for Cross-Site MDD Identification from rs-fMRI

Multimodal Connectome Fusion via Cross-Attention for Autism Spectrum Disorder Classification Using Graph Learning