M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification

Muhammad Asif Hasan; Yanming Zhu; Xuefei Yin; Alan Wee-Chung Liew

cs.CVAug 31, 2026

M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification

Authors: Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin, Alan Wee-Chung Liew

Organizations: School of Information and Communication Technology, Griffith University, Gold Coast, QLD, 4215, Australia

Abstract

Classification models based on resting-state functional magnetic resonance imaging (rs-fMRI) often show lower performance at imaging sites not included during model development, which can limit their use in clinical settings. Domain generalization (DG) addresses this issue by learning representations from source sites that remain effective for unseen target sites. However, existing DG approaches for psychiatric disorder classification commonly rely on a single imaging modality and may not fully account for site-specific acquisition effects on the learned representation space. Subjects scanned at the same site share scanner hardware, acquisition settings, and preprocessing characteristics, which can cause representations to reflect acquisition conditions rather than diagnostic information. In this work, we present M2LG-DG, a source-only multimodal local-global framework for cross-site major depressive disorder (MDD) classification. The framework employs a dual-stream rs-fMRI encoder, where the global pathway models inter-regional dependencies through self-attention and the local pathway performs graph-constrained aggregation over functional connectivity-derived brain graphs. Imaging and non-imaging representations are decomposed into shared and private components and integrated through bidirectional cross-attention with a learned modality gate. A cross-site supervised contrastive objective forms positive pairs from same-class subjects acquired at different source sites, encouraging the fused representation to preserve diagnostic information across acquisition domains. On four held-out REST-meta-MDD sites, M2LG-DG achieves an AUC of 69.48% and exceeds the closest comparison method by 2.18 percentage points. Experiments on the Autism Brain Imaging Data Exchange (ABIDE) dataset further support its applicability to other psychiatric neuroimaging classification tasks.

Figures & tables

Figure 1: Overview of the proposed M 2 LG-DG framework. In Stage 1, multi-site rs-fMRI time series and non-imaging variables are preprocessed and provided to the dual-stream fMRI encoder in Stage 2, which includes a global encoder (temporal $\rightarrow$ spatial using 1D CNN and Transformer layers) and a local encoder (spatial $\rightarrow$ temporal using Chebyshev graph convolutions), together with a non-imaging MLP encoder. In Stage 3, the learned representations are separated into shared and private components and guided by similarity, orthogonality, and difference losses. In Stage 4, the decomposed tokens are integrated using bidirectional cross-attention and a learned modality gate that assigns sample-specific weights to each modality. Finally, in Stage 5, the fused representation $h_{\mathrm{fused}}$ is employed for MDD/HC classification through $\mathcal{L}_{\mathrm{CE}}$ and cross-site supervised contrastive learning $\mathcal{L}_{\mathrm{con}}$ , where same-site same-class pairs are excluded from the positive set to support site-invariant disease representation learning.

Figure 2: Illustration of cross-site supervised contrastive learning. (a) Pair construction for an MDD anchor from Site A. Same-class subjects from different source sites form the positive set, whereas HC subjects are treated as negatives regardless of site. Same-site same-class subjects are excluded from the positive set, although the denominator of the loss includes all non-anchor samples in the batch. (b) Conceptual effect of the proposed formulation on the representation space. Before contrastive learning, subjects may form site-dependent clusters. After contrastive learning, same-class subjects from different sites are encouraged to form diagnosis-related groups with greater mixing of site identities.

Dual-Stream fMRI Encoder
Global encoder Conv1D channels	[32, 64, 128]
Transformer hidden dimension	128
Transformer attention heads	8
Transformer layers	2
Global encoder output dimension	256
Local encoder ST-GCN channels	[64, 128, 256]

Table 1: Hyperparameters for the proposed M 2 LG-DG framework.

REST-meta-MDD dataset
	Site 9					Site 15
Method	ACC	SPE	SEN	AUC	F1	ACC	SPE	SEN	AUC	F1
BrainNetCNN	53.40 $\pm$ 3.05	62.90 $\pm$ 10.84	44.60 $\pm$ 9.12	60.30 $\pm$ 6.41	41.78 $\pm$ 7.20	54.90 $\pm$ 3.48	58.10 $\pm$ 8.22	51.80 $\pm$ 10.10	58.70 $\pm$ 5.55	53.48 $\pm$ 6.90
CDANN	54.60 $\pm$ 2.74	64.80 $\pm$ 13.10	44.90 $\pm$ 12.90	63.20 $\pm$ 9.55	42.71 $\pm$ 5.12	56.40 $\pm$ 2.58	59.70 $\pm$ 6.02	53.20 $\pm$ 7.05	60.80 $\pm$ 8.44	54.99 $\pm$ 4.60
Mixup	55.10 $\pm$ 4.02	66.40 $\pm$ 6.88	43.70 $\pm$ 7.95	62.10 $\pm$ 6.02	42.41 $\pm$ 11.40	53.20 $\pm$ 4.51	58.90 $\pm$ 4.05	47.80 $\pm$ 7.70	58.20 $\pm$ 3.60	50.61 $\pm$ 11.80
MLDG	54.90 $\pm$ 5.55	65.10 $\pm$ 15.10	44.20 $\pm$ 15.40	60.70 $\pm$ 8.62	42.29 $\pm$ 12.80	55.40 $\pm$ 5.44	58.40 $\pm$ 7.44	52.30 $\pm$ 15.80	59.50 $\pm$ 6.44	53.95 $\pm$ 10.85

Table 2: Classification performance (%) on the REST-meta-MDD dataset, reported as the mean

\pm

standard deviation over five random seeds. Site 9, Site 15, Site 20, and Site 21 were each treated as a held-out target domain in a separate experiment, while the remaining 21 of the 22 included sites were used as source domains for model development. The best result in each column is shown in bold and the second-best result is underlined.

Panel A: Overall performance across the four held-out target sites
Method	ACC	SPE	SEN	AUC	F1
BrainNetCNN	$52.08\pm 6.08$	$54.05\pm 13.65$	$50.05\pm 3.72$	$59.38\pm 0.96$	$50.89\pm 6.29$
CDANN	$53.33\pm 5.81$	$55.38\pm 13.81$	$51.10\pm 4.15$	$61.88\pm 2.02$	$52.07\pm 6.37$
Mixup	$51.65\pm 6.13$	$55.33\pm 13.79$	$47.62\pm 3.03$	$59.95\pm 2.10$	$49.43\pm 5.09$
MLDG	$52.90\pm 5.95$	$54.88\pm 13.92$	$50.42\pm 4.17$	$60.00\pm 0.76$	$51.45\pm 6.25$
BrainGNN	$51.67\pm 6.89$	$57.15\pm 15.09$	$46.30\pm 2.49$	$60.70\pm 3.76$	$49.00\pm 4.03$

Table 3: Classification performance (%) on the REST-meta-MDD dataset across the four held-out target sites (Site 9, Site 15, Site 20, and Site 21). Panel A reports the unweighted macro-average performance across the four target sites, while Panel B reports paired statistical comparisons between M 2 LG-DG and four competitive baseline methods.

Config	Variant	Dual	Non-img	Decomp	CrossAttn	Contrastive	ACC	AUC	F1
Input modality analysis
M1	Non-imaging-only	$\times$	✓	$\times$	$\times$	$\times$	53.38	56.70	52.83
M2	fMRI-only	✓	$\times$	$\times$	$\times$	$\times$	57.85	61.92	55.74
Component analysis
C1	w/o Decomp	✓	✓	$\times$	$\times$ †	✓	59.96	64.21	57.87
C2	w/o CrossAttn	✓	✓	✓	$\times$ *	✓	60.85	64.92	59.74

Table 4: Ablation study (%) of the input modalities and the main components of M 2 LG-DG on the REST-meta-MDD dataset under the site-wise source-only evaluation protocol. Results are averaged across Site 9, Site 15, Site 20, and Site 21 over five random seeds. Bold and underlined values denote the best and second-best results across all configurations.

Figure 3: Sensitivity analysis of hyperparameters. The curves show the mean ACC, F1, and AUC averaged across Site 9, Site 15, Site 20, and Site 21. The yellow band indicates the selected default value.

Figure 4: Top 20 ROIs identified by M 2 LG-DG based on integrated gradients attribution scores. The top 10 ROIs are highlighted in red, while ranks 11 to 20 are shown in gray. Higher scores indicate a larger contribution to the model’s classification decision.

ABIDE dataset
Site	Method	ACC	SPE	SEN	AUC	F1
NYU	BrainNetCNN	56.49 $\pm$ 3.22	62.24 $\pm$ 3.15	48.77 $\pm$ 3.95	63.15 $\pm$ 1.09	48.90 $\pm$ 3.83
	CDANN	57.31 $\pm$ 4.24	62.24 $\pm$ 5.73	50.68 $\pm$ 3.36	65.35 $\pm$ 1.33	50.39 $\pm$ 3.89
	Mixup	54.50 $\pm$ 2.35	58.78 $\pm$ 2.66	48.77 $\pm$ 5.00	65.59 $\pm$ 2.91	47.71 $\pm$ 3.71
	MLDG	56.14 $\pm$ 3.33	58.37 $\pm$ 3.01	53.15 $\pm$ 4.27	65.49 $\pm$ 1.38	50.84 $\pm$ 3.85
	BrainGNN	55.56 $\pm$ 4.22	61.43 $\pm$ 6.18	47.67 $\pm$ 3.80	67.89 $\pm$ 1.96	47.84 $\pm$ 3.79

Table 5: Classification results (%) on the ABIDE dataset, reported as mean

\pm

standard deviation over five random seeds. NYU and USM were each held out separately as unseen target domains, while the remaining 19 of the 20 included sites were used as source domains for model development. The best result in each column is shown in bold and the second-best is underlined.

Explore similar work

Jul 31, 2026

Multi-Source Multi-View Graph Domain Adaptation with Hyperbolic Residual Encoding for Cross-Site MDD Identification from rs-fMRI

Cross-site identification of major depressive disorder (MDD) from resting-state functional magnetic resonance imaging (rs-fMRI) is hindered by inter-site distribution shifts and heterogeneous functional connectivity (FC) views. These views capture complementary neural relationships but exhibit distinct site biases and graph topologies, complicating alignment without sacrificing disease-relevant information or cross-view consistency. Existing studies largely treat multi-view connectome learning and cross-site adaptation separately. To the best of our knowledge, few studies have jointly modeled multiple FC views under multi-source unsupervised domain adaptation for cross-site rs-fMRI-based MDD classification. We construct Pearson correlation, sparse representation, and Granger causality graphs, each encoded by a view-specific graph attention network. Dual-stream adaptive fusion explicitly integrates pairwise cross-view interactions, followed by lightweight hyperbolic residual encoding for curvature-aware representation refinement. Class-wise Cauchy--Schwarz alignment reduces inter-source and source-target discrepancies, complemented by adversarial learning, information maximization, and confidence-aware pseudo-labeling. Across seven unlabeled target domains, our framework achieves 73.60% mean accuracy and 71.90% AUC, demonstrating effective generalization under heterogeneous acquisition conditions. These results highlight the effectiveness of unified heterogeneous-view modeling, curvature-aware refinement, and multi-source domain adaptation for cross-site MDD identification.The source code is at https://github.com/OPUS-Lightphenexx/MM-HyperGDA

Zhanpeng Zheng, Xiran Chen, Haiteng Jiang +6

School of Computer and Artificial Intelligence, Shandong Jianzhu University, Shandong, China. · School of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing, China. · Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, Sichuan, China. +4

Multi-Source Domain Adaptation Domain Adaptation Depression Detection Multi-View Learning

Ask PDF

Apr 11, 2026

A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection

Major depressive disorder (MDD) is a prevalent mental disorder associated with complex neurobiological changes that cannot be fully captured using a single imaging modality. The use of multimodal magnetic resonance imaging (MRI) provides a more comprehensive understanding of brain changes by combining structural and functional data. Despite this, the effective integration of these modalities remains challenging. In this study, we propose a dual cross-attention-based multimodal fusion framework that explicitly models bidirectional interactions between structural MRI (sMRI) and resting-state functional MRI (rs-fMRI) representations. The proposed approach is tested on the large-scale REST-meta-MDD dataset using both structural and functional brain atlas configurations. Numerous experiments conducted under a 10-fold stratified cross-validation demonstrated that the proposed fusion algorithm achieves robust and competitive performance across all atlas types. The proposed method consistently outperforms conventional feature-level concatenation for functional atlases, while maintaining comparable performance for structural atlases. The most effective dual cross-attention multimodal model obtained 84.71% accuracy, 86.42% sensitivity, 82.89% specificity, 84.34% precision, and 85.37% F1-score. These findings emphasize the importance of explicitly modeling cross-modal interactions for multimodal neuroimaging-based MDD classification.

Nojod M. Alotaibi, Areej M. Alhothali

Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia

Depression Detection Multimodal Fusion Neuroimaging Multimodal Graph Learning Cross-Modal Attention

Ask PDF

May 22, 2026

fMRI-Diffusion: Generating fMRI Time Series Via a Temporal Transformer Diffusion Model for Major Depressive Disorder Diagnosis

Diagnosing Major Depressive Disorder (MDD) from functional magnetic resonance imaging (fMRI) using functional connectivity (FC) analysis requires large amounts of labeled data that are scarce in clinical settings. Existing augmentation methods synthesize FC matrices, which compress fMRI recordings into static pairwise summaries and discard temporal information. We propose fMRI-Diffusion, a framework that synthesizes region-of-interest (ROI)-level fMRI time series rather than FC matrices. A Temporal Transformer serves as the denoising network within a denoising diffusion probabilistic model, treating each time point as a token to capture temporal dependencies through self-attention. A supervised pretraining strategy initializes the Transformer with task-relevant representations before diffusion training, and FC matrices are derived from the synthesized time series for classification. Experiments on the REST-meta-MDD dataset show that augmenting training data with synthetic time series consistently improves diagnostic accuracy across ten classifiers, six parcellation atlases, and three acquisition sites. The method outperforms five recent FC-based synthesis approaches, with accuracy gains of up to 3.7 percentage points over the strongest baseline. Ablation studies confirm the contributions of both the Transformer-based denoiser and the pretraining strategy. Distributional fidelity metrics remain below 0.06 across all conditions, indicating close agreement between real and synthetic distributions. These findings suggest that synthesizing fMRI time series before FC computation preserves temporal information lost in matrix-level augmentation and provides a practical strategy for MDD diagnosis under limited data.

Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin +1

aSchool of Information and Communication Technology, Griffith University, Gold Coast, QLD, 4215, Australia

Depression Detection Time Series Generation Time Series Data Augmentation Diffusion Models

Ask PDF