Multimodal Connectome Fusion via Cross-Attention for Autism Spectrum Disorder Classification Using Graph Learning

Ansar Rahman; Hassan Shojaee-Mend; Sepideh Hatamikia

cs.CVMar 16, 2026

Multimodal Connectome Fusion via Cross-Attention for Autism Spectrum Disorder Classification Using Graph Learning

Authors: Ansar Rahman, Hassan Shojaee-Mend, Sepideh Hatamikia

Organizations: Department of Medicine, Danube Private University (DPU), Krems, Austria · Department of Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria · Department of Medical Informatics, Faculty of Medicine, Gonabad University of Medical Sciences, Gonabad, Iran · Austrian Center for Medical Innovation and Technology (ACMIT), Wiener Neustadt, Austria

Abstract

Autism spectrum disorder (ASD) is a complex neurodevelopmental condition characterized by atypical functional brain connectivity and subtle structural alterations. rs-fMRI has been widely used to identify disruptions in large-scale brain networks, while structural MRI provides complementary information about morphological organization. Despite their complementary nature, effectively integrating these heterogeneous imaging modalities within a unified framework remains challenging. This study proposes a multimodal graph learning framework that preserves the dominant role of functional connectivity while integrating structural imaging and phenotypic information for ASD classification. The proposed framework is evaluated on ABIDE-I dataset. Each subject is represented as a node within a population graph. Functional and structural features are extracted as modality-specific node attributes, while inter-subject relationships are modeled using a pairwise association encoder (PAE) based on phenotypic information. Two Edge Variational GCNs are trained to learn subject-level embeddings. To enable effective multimodal integration, we introduce a novel asymmetric transformer-based cross-attention mechanism that allows functional embeddings to selectively incorporate complementary structural information while preserving functional dominance. The fused embeddings are then passed to a MLP for ASD classification. Using stratified 10-fold cross-validation, the framework achieved an AUC of 87.3% and an accuracy of 84.4%. Under leave-one-site-out cross-validation (LOSO-CV), the model achieved an average cross-site accuracy of 82.0%, outperforming existing methods by approximately 3% under 10-fold cross-validation and 7% under LOSO-CV. The proposed framework effectively integrates heterogeneous multimodal data from the multi-site ABIDE-I dataset, improving automated ASD classification across imaging sites.

Figures & tables

Figure 1: Overview of multimodal ASD diagnosis using heterogeneous ABIDE multi-site data, highlighting the transition from clinician-based assessment to AI-driven multimodal analysis

Figure 2: End-to-End Workflow Diagram(a) rs-fMRI, sMRI, and phenotypic data are processed to extract functional and structural connectivity features; (b) modality-specific population graphs are constructed and processed using EV-GCN to learn functional and structural node embeddings;(c) the resulting embeddings are integrated through asymmetric cross-attention fusion and passed to a Feed Forward MLP classifier to predict ASD vs. TD

Figure 3: Transformer-based asymmetric cross-attention fusion integrating modality-specific EV-GCN embeddings from functional and structural brain connectivity.

Note: (*) indicates the inclusion of non-imaging data alongside the specified
Model	Modality	AUC	Acc	Sensitivity	Specificity	F1-Score
	rs-fMRI	79.40	77.40	76.70	84.41	80.00
Proposed	rs-fMRI*	84.80	82.60	83.90	85.30	84.10
Method	sMRI	73.90	73.30	73.00	81.00	76.40
	sMRI*	82.30	81.00	86.73	76.50	80.90
	Multimodal	87.30	84.40	85.20	87.00	85.70

Table 1: Effect of Multimodal Learning, evaluated using stratified 10-fold cross-validation

Reference	Model	AUC	Acc
[ 11 ]	FCN	69.60	66.20
Proposed Method		82.30	81.00

Table 2: Results comparison for sMRI + non-imaging Data , evaluated using stratified 10-fold cross-validation

Figure 4: ROC–AUC curve for the proposed model under 10-fold cross-validation

Multimodal Fusion Strategy	AUC	Acc	Sens	Spec	F1-Score
Concatenation	84.90	83.00	84.90	84.20	84.30
Symmetric Cross-Attention	82.30	81.30	84.40	82.30	82.90
Asymmetric Cross-Attention	87.30	84.40	85.20	87.00	85.70

Table 3: Effect of the proposed asymmetric cross-attention–based fusion strategy on multimodal ASD classification performance, evaluated using stratified 10-fold cross-validation

Reference	Model	Modality	AUC	ACC
[ 19 ]	Brain Network Transformer	rs-fMRI*	80.2	71.0
[ 4 ]	Com-BrainTF	rs-fMRI*	79.6	72.5
[ 17 ]	ASDFormer	rs-fMRI*	81.17	74.60
[ 33 ]	DeepGCN	rs-fMRI*	82.59	77.27
[ 32 ]	MMGCN	rs-fMRI*	84.00	78.31
[ 16 ]	EV-GCN	rs-fMRI*	84.72	81.06

Table 4: Stratified 10-fold cross-validation based results comparison with SOTA ASD classification methods

Site	Size	ASD-	ASD-	DeepGCN	Our
		DiagNet	SAENet		Proposed
CALTECH	15	52.8	56.7	71.7	86.7
CMU	11	68.5	70.6	80.5	81.8
KKI	33	69.5	72.6	76.2	63.6
LEUVEN	56	61.3	64.6	74.3	71.4
MAX_MUN	46	48.6	47.5	69.7	87.0

Table 5: Site-wise classification accuracy (%) under LOSO-CV evaluation

Figure 5: Bar plot comparing the performance of the proposed model with SOTA methods under LOSO cross-validation

Explore similar work

Mar 3, 2026

Beyond Anatomy: Explainable ASD Classification from rs-fMRI via Functional Parcellation and Graph Attention Networks

Anatomical brain parcellations dominate rs-fMRI-based Autism Spectrum Disorder (ASD) classification, yet their rigid boundaries may fail to capture the idiosyncratic connectivity patterns that characterise ASD. We present a graph-based deep learning framework comparing anatomical (AAL, 116 ROIs) and functionally-derived (MSDL, 39 ROIs) parcellation strategies on the ABIDE I dataset. Our FSL preprocessing pipeline handles multi-site heterogeneity across 400 balanced subjects, with site-stratified 70/15/15 splits to prevent data leakage. Gaussian noise augmentation within training folds expands samples from 280 to 1,680. A three phase pipeline progresses from a baseline GCN with AAL (73.3% accuracy, AUC=0.74), to an optimised GCN with MSDL (84.0%, AUC=0.84), to a Graph Attention Network ensemble achieving 95.0% accuracy (AUC=0.98), outperforming all recent GNN-based benchmarks on ABIDE I. The 10.7-point gain from atlas substitution alone demonstrates that functional parcellation is the most impactful modelling decision. Gradient-based saliency and GNNExplainer analyses converge on the Posterior Cingulate Cortex and Precuneus as core Default Mode Network hubs, validating that model decisions reflect ASD neuropathology rather than acquisition artefacts. All code and datasets will be publicly released upon acceptance.

Syeda Hareem Madani, Noureen Bibi, Adam Rafiq Jeraj +3

Salim Habib University · VentureDive · National University of Computer and Emerging Sciences

Autism Spectrum Disorder Neuroimaging Explainable Medical Image Analysis Graph Attention Networks Functional Connectivity

Ask PDF

Mar 27, 2026

Graph Attention Network-Based Detection of Autism Spectrum Disorder

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition characterized by atypical brain connectivity. One of the crucial steps in addressing ASD is its early detection. This study introduces a novel computational framework that employs an Attention-Based Graph Convolutional Network, referred to as the GATGraphClassifier, for detecting ASD. We utilize Functional Magnetic Resonance Imaging (fMRI) data from the Autism Brain Imaging Data Exchange (ABIDE) repository to construct functional connectivity matrices using Pearson correlation, which captures interactions between various brain regions. These matrices are then transformed into graph representations, where the nodes and edges represent the brain regions and functional connections, respectively. The GATGraphClassifier employs attention mechanisms to identify critical connectivity patterns, thereby enhancing the model's interpretability and diagnostic accuracy. Our proposed framework demonstrates superior performance across all standard classification metrics compared to existing state-of-the-art methods. Notably, we achieved an average accuracy of 88.79% on the test data over 30 independent runs, surpassing the benchmark model's performance by 12.27%. In addition, we identified the crucial brain regions associated with ASD, consistent with the previous studies, and a few novel regions. This study not only contributes to the advancement of ASD detection but also shows the potential for broader adaptability of GATGraphClassifier in analyzing complex relational data in various fields, where understanding intricate connectivity and interaction patterns is essential.

Abigail Kelly, Ramchandra Rimal, Arpan Sainju

Computational and Data Science, Middle Tennessee State University, 1301 E. Main St., Murfreesboro, 37132, TN, USA. · Department of Mathematical Sciences, Middle Tennessee State University, 1301 E. Main St., Murfreesboro, 37132, TN, USA.

Autism Spectrum Disorder Brain Network Analysis Graph Attention Networks

Ask PDF

Sep 17, 2026

HyperAMS-Net: Adaptive Multi-Scale Spatial Hypergraph Network for Brain Disorder Classification

Accurate classification of brain disorders from neuroimaging data remains challenging because of substantial inter-subject heterogeneity and the complex multi-scale patterns present in functional connectivity and morphological representations. To address these challenges, we propose HyperAMS-Net, a deep learning framework for brain disorder classification using neuroimaging representations derived from resting-state functional MRI or structural MRI. HyperAMS-Net integrates adaptive multi-scale convolution, hypergraph attention, spatial-channel attention, and adaptive feature fusion. Specifically, adaptive multi-scale convolution learns data-driven weights over multiple receptive fields to capture complementary patterns at different scales. Hypergraph attention models higher-order dependencies among learned feature representations through node--hyperedge--node message passing, while spatial-channel attention enhances discriminative feature learning. Adaptive feature fusion further aggregates complementary information across parallel network branches. HyperAMS-Net is evaluated on three benchmark datasets spanning distinct brain disorders: ABIDE for autism spectrum disorder, REST-meta-MDD for major depressive disorder, and ADNI for Alzheimer's disease, using 5-fold stratified cross-validation. HyperAMS-Net achieves state-of-the-art performance across all evaluated datasets, attaining the highest accuracy and AUC among the compared methods. Ablation studies further demonstrate the contribution of each proposed component, with the largest performance degradation observed when hypergraph attention is removed.

Proloy Kumar Mondal, Md Kamran Hussin Chowdhury, Hoi Leong Lee

Department of Computer Science and Engineering, University of North Texas, Denton, Texas, USA · Inje University, Gimhae-si, Republic of Korea · Faculty of Electronic Engineering and Technology, Universiti Malaysia Perlis, Arau, Malaysia

Medical Diagnosis Multi-Scale Feature Fusion Hypergraph Neural Networks Alzheimer's Disease Classification Neuroimaging

Ask PDF