M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification

Muhammad Asif Hasan; Yanming Zhu; Xuefei Yin; Alan Wee-Chung Liew

cs.CVAug 31, 2026

M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification

Authors: Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin, Alan Wee-Chung Liew

Organizations: School of Information and Communication Technology, Griffith University, Gold Coast, QLD, 4215, Australia

Abstract

Classification models based on resting-state functional magnetic resonance imaging (rs-fMRI) often show lower performance at imaging sites not included during model development, which can limit their use in clinical settings. Domain generalization (DG) addresses this issue by learning representations from source sites that remain effective for unseen target sites. However, existing DG approaches for psychiatric disorder classification commonly rely on a single imaging modality and may not fully account for site-specific acquisition effects on the learned representation space. Subjects scanned at the same site share scanner hardware, acquisition settings, and preprocessing characteristics, which can cause representations to reflect acquisition conditions rather than diagnostic information. In this work, we present M2LG-DG, a source-only multimodal local-global framework for cross-site major depressive disorder (MDD) classification. The framework employs a dual-stream rs-fMRI encoder, where the global pathway models inter-regional dependencies through self-attention and the local pathway performs graph-constrained aggregation over functional connectivity-derived brain graphs. Imaging and non-imaging representations are decomposed into shared and private components and integrated through bidirectional cross-attention with a learned modality gate. A cross-site supervised contrastive objective forms positive pairs from same-class subjects acquired at different source sites, encouraging the fused representation to preserve diagnostic information across acquisition domains. On four held-out REST-meta-MDD sites, M2LG-DG achieves an AUC of 69.48% and exceeds the closest comparison method by 2.18 percentage points. Experiments on the Autism Brain Imaging Data Exchange (ABIDE) dataset further support its applicability to other psychiatric neuroimaging classification tasks.

Figures & tables

Figure 1: Overview of the proposed M 2 LG-DG framework. In Stage 1, multi-site rs-fMRI time series and non-imaging variables are preprocessed and provided to the dual-stream fMRI encoder in Stage 2, which includes a global encoder (temporal $\rightarrow$ spatial using 1D CNN and Transformer layers) and a local encoder (spatial $\rightarrow$ temporal using Chebyshev graph convolutions), together with a non-imaging MLP encoder. In Stage 3, the learned representations are separated into shared and private components and guided by similarity, orthogonality, and difference losses. In Stage 4, the decomposed tokens are integrated using bidirectional cross-attention and a learned modality gate that assigns sample-specific weights to each modality. Finally, in Stage 5, the fused representation $h_{\mathrm{fused}}$ is employed for MDD/HC classification through $\mathcal{L}_{\mathrm{CE}}$ and cross-site supervised contrastive learning $\mathcal{L}_{\mathrm{con}}$ , where same-site same-class pairs are excluded from the positive set to support site-invariant disease representation learning.

Figure 2: Illustration of cross-site supervised contrastive learning. (a) Pair construction for an MDD anchor from Site A. Same-class subjects from different source sites form the positive set, whereas HC subjects are treated as negatives regardless of site. Same-site same-class subjects are excluded from the positive set, although the denominator of the loss includes all non-anchor samples in the batch. (b) Conceptual effect of the proposed formulation on the representation space. Before contrastive learning, subjects may form site-dependent clusters. After contrastive learning, same-class subjects from different sites are encouraged to form diagnosis-related groups with greater mixing of site identities.

Dual-Stream fMRI Encoder
Global encoder Conv1D channels	[32, 64, 128]
Transformer hidden dimension	128
Transformer attention heads	8
Transformer layers	2
Global encoder output dimension	256
Local encoder ST-GCN channels	[64, 128, 256]

Table 1: Hyperparameters for the proposed M 2 LG-DG framework.

REST-meta-MDD dataset
	Site 9					Site 15
Method	ACC	SPE	SEN	AUC	F1	ACC	SPE	SEN	AUC	F1
BrainNetCNN	53.40 $\pm$ 3.05	62.90 $\pm$ 10.84	44.60 $\pm$ 9.12	60.30 $\pm$ 6.41	41.78 $\pm$ 7.20	54.90 $\pm$ 3.48	58.10 $\pm$ 8.22	51.80 $\pm$ 10.10	58.70 $\pm$ 5.55	53.48 $\pm$ 6.90
CDANN	54.60 $\pm$ 2.74	64.80 $\pm$ 13.10	44.90 $\pm$ 12.90	63.20 $\pm$ 9.55	42.71 $\pm$ 5.12	56.40 $\pm$ 2.58	59.70 $\pm$ 6.02	53.20 $\pm$ 7.05	60.80 $\pm$ 8.44	54.99 $\pm$ 4.60
Mixup	55.10 $\pm$ 4.02	66.40 $\pm$ 6.88	43.70 $\pm$ 7.95	62.10 $\pm$ 6.02	42.41 $\pm$ 11.40	53.20 $\pm$ 4.51	58.90 $\pm$ 4.05	47.80 $\pm$ 7.70	58.20 $\pm$ 3.60	50.61 $\pm$ 11.80
MLDG	54.90 $\pm$ 5.55	65.10 $\pm$ 15.10	44.20 $\pm$ 15.40	60.70 $\pm$ 8.62	42.29 $\pm$ 12.80	55.40 $\pm$ 5.44	58.40 $\pm$ 7.44	52.30 $\pm$ 15.80	59.50 $\pm$ 6.44	53.95 $\pm$ 10.85

Table 2: Classification performance (%) on the REST-meta-MDD dataset, reported as the mean

\pm

standard deviation over five random seeds. Site 9, Site 15, Site 20, and Site 21 were each treated as a held-out target domain in a separate experiment, while the remaining 21 of the 22 included sites were used as source domains for model development. The best result in each column is shown in bold and the second-best result is underlined.

Panel A: Overall performance across the four held-out target sites
Method	ACC	SPE	SEN	AUC	F1
BrainNetCNN	$52.08\pm 6.08$	$54.05\pm 13.65$	$50.05\pm 3.72$	$59.38\pm 0.96$	$50.89\pm 6.29$
CDANN	$53.33\pm 5.81$	$55.38\pm 13.81$	$51.10\pm 4.15$	$61.88\pm 2.02$	$52.07\pm 6.37$
Mixup	$51.65\pm 6.13$	$55.33\pm 13.79$	$47.62\pm 3.03$	$59.95\pm 2.10$	$49.43\pm 5.09$
MLDG	$52.90\pm 5.95$	$54.88\pm 13.92$	$50.42\pm 4.17$	$60.00\pm 0.76$	$51.45\pm 6.25$
BrainGNN	$51.67\pm 6.89$	$57.15\pm 15.09$	$46.30\pm 2.49$	$60.70\pm 3.76$	$49.00\pm 4.03$

Table 3: Classification performance (%) on the REST-meta-MDD dataset across the four held-out target sites (Site 9, Site 15, Site 20, and Site 21). Panel A reports the unweighted macro-average performance across the four target sites, while Panel B reports paired statistical comparisons between M 2 LG-DG and four competitive baseline methods.

Config	Variant	Dual	Non-img	Decomp	CrossAttn	Contrastive	ACC	AUC	F1
Input modality analysis
M1	Non-imaging-only	$\times$	✓	$\times$	$\times$	$\times$	53.38	56.70	52.83
M2	fMRI-only	✓	$\times$	$\times$	$\times$	$\times$	57.85	61.92	55.74
Component analysis
C1	w/o Decomp	✓	✓	$\times$	$\times$ †	✓	59.96	64.21	57.87
C2	w/o CrossAttn	✓	✓	✓	$\times$ *	✓	60.85	64.92	59.74

Table 4: Ablation study (%) of the input modalities and the main components of M 2 LG-DG on the REST-meta-MDD dataset under the site-wise source-only evaluation protocol. Results are averaged across Site 9, Site 15, Site 20, and Site 21 over five random seeds. Bold and underlined values denote the best and second-best results across all configurations.

Figure 3: Sensitivity analysis of hyperparameters. The curves show the mean ACC, F1, and AUC averaged across Site 9, Site 15, Site 20, and Site 21. The yellow band indicates the selected default value.

Figure 4: Top 20 ROIs identified by M 2 LG-DG based on integrated gradients attribution scores. The top 10 ROIs are highlighted in red, while ranks 11 to 20 are shown in gray. Higher scores indicate a larger contribution to the model’s classification decision.

ABIDE dataset
Site	Method	ACC	SPE	SEN	AUC	F1
NYU	BrainNetCNN	56.49 $\pm$ 3.22	62.24 $\pm$ 3.15	48.77 $\pm$ 3.95	63.15 $\pm$ 1.09	48.90 $\pm$ 3.83
	CDANN	57.31 $\pm$ 4.24	62.24 $\pm$ 5.73	50.68 $\pm$ 3.36	65.35 $\pm$ 1.33	50.39 $\pm$ 3.89
	Mixup	54.50 $\pm$ 2.35	58.78 $\pm$ 2.66	48.77 $\pm$ 5.00	65.59 $\pm$ 2.91	47.71 $\pm$ 3.71
	MLDG	56.14 $\pm$ 3.33	58.37 $\pm$ 3.01	53.15 $\pm$ 4.27	65.49 $\pm$ 1.38	50.84 $\pm$ 3.85
	BrainGNN	55.56 $\pm$ 4.22	61.43 $\pm$ 6.18	47.67 $\pm$ 3.80	67.89 $\pm$ 1.96	47.84 $\pm$ 3.79

Table 5: Classification results (%) on the ABIDE dataset, reported as mean

\pm

standard deviation over five random seeds. NYU and USM were each held out separately as unseen target domains, while the remaining 19 of the 20 included sites were used as source domains for model development. The best result in each column is shown in bold and the second-best is underlined.

Explore similar work

Multi-Source Multi-View Graph Domain Adaptation with Hyperbolic Residual Encoding for Cross-Site MDD Identification from rs-fMRI
Jul 31, 2026Zhanpeng Zheng, Xiran Chen, Haiteng Jiang +6Multi-Source Domain Adaptation Domain Adaptation
School of Computer and Artificial Intelligence, Shandong Jianzhu University, Shandong, China. · School of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing, China. · Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, Sichuan, China. +4
A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection
Apr 11, 2026Nojod M. Alotaibi, Areej M. AlhothaliDepression Detection Multimodal Fusion
Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia
fMRI-Diffusion: Generating fMRI Time Series Via a Temporal Transformer Diffusion Model for Major Depressive Disorder Diagnosis
May 22, 2026Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin +1Depression Detection Time Series Generation
aSchool of Information and Communication Technology, Griffith University, Gold Coast, QLD, 4215, Australia

Jul 31, 2026J/K move · Enter open · S save

Multi-Source Multi-View Graph Domain Adaptation with Hyperbolic Residual Encoding for Cross-Site MDD Identification from rs-fMRI

Zhanpeng Zheng, Xiran Chen, Haiteng Jiang, Renjie Tian+5

School of Computer and Artificial Intelligence, Shandong Jianzhu University, Shandong, China. · School of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing, China. · Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.+4

Multi-Source Domain Adaptation Domain Adaptation Depression Detection Multi-View Learning

Open paper AskPDF arXiv

Cross-site identification of major depressive disorder (MDD) from resting-state functional magnetic resonance imaging (rs-fMRI) is hindered by inter-site distribution shifts and heterogeneous functional connectivity (FC) views. These views capture complementary neural relationships but exhibit distinct site biases and graph topologies, complicating alignment without sacrificing disease-relevant information or cross-view consistency. Existing studies largely treat multi-view connectome learning and cross-site adaptation separately. To the best of our knowledge, few studies have jointly modeled multiple FC views under multi-source unsupervised domain adaptation for cross-site rs-fMRI-based MDD classification. We construct Pearson correlation, sparse representation, and Granger causality graphs, each encoded by a view-specific graph attention network. Dual-stream adaptive fusion explicitly integrates pairwise cross-view interactions, followed by lightweight hyperbolic residual encoding for curvature-aware representation refinement. Class-wise Cauchy--Schwarz alignment reduces inter-source and source-target discrepancies, complemented by adversarial learning, information maximization, and confidence-aware pseudo-labeling. Across seven unlabeled target domains, our framework achieves 73.60% mean accuracy and 71.90% AUC, demonstrating effective generalization under heterogeneous acquisition conditions. These results highlight the effectiveness of unified heterogeneous-view modeling, curvature-aware refinement, and multi-source domain adaptation for cross-site MDD identification.The source code is at https://github.com/OPUS-Lightphenexx/MM-HyperGDA

Abstract

Explore similar work

Multi-Source Multi-View Graph Domain Adaptation with Hyperbolic Residual Encoding for Cross-Site MDD Identification from rs-fMRI

A Dual Cross-Attention Graph Learning Framework For Multimodal MRI-Based Major Depressive Disorder Detection

fMRI-Diffusion: Generating fMRI Time Series Via a Temporal Transformer Diffusion Model for Major Depressive Disorder Diagnosis