Learning Structural-Functional Brain Representations through Multi-Scale Adaptive Graph Attention for Cognitive Insight
Authors: Badhan Mazumder, Sir-Lord Wiafe, Aline Kotoski, Vince D. Calhoun, Dong Hye Ye
Organizations: Department of Computer Science, Georgia State University · Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, and Emory University · Neuroscience Institute, Georgia State University
Understanding how brain structure and function interact is key to explaining intelligence yet modeling them jointly is challenging as the structural and functional connectome capture complementary aspects of organization. We introduced Multi-scale Adaptive Graph Network (MAGNet), a Transformer-style graph neural network framework that adaptively learns structure-function interactions. MAGNet leverages source-based morphometry from structural MRI to extract inter-regional morphological features and fuses them with functional network connectivity from resting-state fMRI. A hybrid graph integrates direct and indirect pathways, while local-global attention refines connectivity importance and a joint loss simultaneously enforces cross-modal coherence and optimizes the prediction objective end-to-end. On the ABCD dataset, MAGNet outperformed relevant baselines, demonstrating effective multimodal integration for advancing our understanding of cognitive function.
Figures & tables
Figure 1: (A) After generating FNC from rs-fMRI and SBM from sMRI, a hybrid brain graph was constructed with unimodal, cross-modal (CMC) and multi-scale detour connections (MDC) measuring structural detour (SD). (B) MAGNet ’s flowchart: scaled dot-product attention enabled local message passing with node features and edge attributes, followed by multi-head self-attention for global refinement for l layers. Refined node embeddings then underwent global average pooling (GAP) and a fully connected layer (FCL) for intelligence score prediction optimized with a joint loss.
Method
Fluid
Crystallized
Composite
MSE
MAE
Correlation
MSE
MAE
Correlation
MSE
MAE
Correlation
GAT [ 15 ]
118.40 ± 2.58
10.38 ± 0.10
0.15 ± 0.0145
62.57 ± 3.31
7.63 ± 0.17
0.11 ± 0.0238
88.57 ± 3.17
9.24 ± 0.15
0.16 ± 0.0238
GT [ 5 ]
116.02 ± 5.56
9.64 ± 0.26
0.24 ± 0.0150
57.15 ± 5.36
6.90 ± 0.36
0.26 ± 0.0229
85.74 ± 6.06
8.40 ± 0.29
0.26 ± 0.0187
SFDN [ 16 ]
112.43 ± 0.80
9.15 ± 0.41
0.29 ± 0.0162
53.69 ± 0.81
6.04 ± 0.42
0.34 ± 0.0135
83.46 ± 0.81
8.02 ± 0.41
0.35 ± 0.0148
SFIN [ 18 ]
113.23 ± 0.51
9.66 ± 0.15
0.30 ± 0.0273
54.91 ± 0.32
6.16 ± 0.27
0.32 ± 0.0226
84.21 ± 0.45
8.11 ± 0.65
0.33 ± 0.0221
Joint GCN [ 8 ]
115.29 ± 0.31
9.91 ± 0.58
0.27 ± 0.0186
55.81 ± 0.53
6.32 ± 0.50
0.28 ± 0.0389
84.73 ± 0.79
8.57 ± 0.43
0.30 ± 0.0263
Table 1: Comparison with SOTA baselines (mean ± standard deviation).
Figure 2: Outcomes of the performed ablation experiments (mean ± standard deviation).
Figure 3: Top 3% significant brain network connections identified for (a) fluid, (b) crystallized, and (c) total intelligence.
School of Medicine, University of Dundee, UK · College of Medicine and Biological Information Engineering, Northeastern University, China · School of Airspace Science and Engineering, Shandong University, China +5
School of Information Technology, Monash University Malaysia · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Department of Radiology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA