Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

Chyong Yi Poh; Hwa Hui Tew; Junn Yong Loo; Raphaël C. -W. Phan; Fuad Noman; Pew-Thian Yap; Chee-Ming Ting

cs.LGOct 2, 2026

Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

Authors: Chyong Yi Poh, Hwa Hui Tew, Junn Yong Loo, Raphaël C. -W. Phan, Fuad Noman, Pew-Thian Yap, Chee-Ming Ting

Organizations: School of Information Technology, Monash University Malaysia · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Department of Radiology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

Abstract

Structural connectivity (SC) and functional connectivity (FC) provide complementary information on interactions between brain regions and are widely used in neuroimaging studies of neuropsychiatric disorders. Generative modelling can alleviate the scarcity of large-scale paired SC-FC data, but existing approaches typically use pairwise graphs that capture only dyadic interactions and often generate SC and FC independently, limiting preservation of higher-order structure-function relationships. We propose a Multimodal Hypergraph Flow Matching (MHG-FM) framework for joint SC-FC connectivity generation and cross-modal translation. MHG-FM constructs modality-specific hypergraphs, learns higher-order representations with Hypergraph Neural Network (HGNN) encoders, and performs bidirectional cross-modal fusion using Dual Cross-Attention (DCA). A variational autoencoder maps the fused representations to a compact latent space, where conditional flow matching enables connectivity synthesis and multimodal translation via latent transport. Experiments on the Human Connectome Project Young Adult (HCP-YA) dataset show that MHG-FM outperforms several state-of-the-art baselines in reconstruction quality, topology preservation, distributional similarity, and SC-FC coupling, while achieving approximately 8x faster sampling than a matched diffusion backbone.

Figures & tables

Fig. 1 : Overview of the proposed MHG-FM framework for joint structural-functional brain connectivity generation and translation. Panels: (1) Multimodal input and hypergraph construction; (2) Multimodal encoding and DCA fusion; (3) Flow-matching latent trajectory learning; (4) Decoding; (5) Sampling and inference; (6) Applications and outputs.

Connectivity	Method	$r\uparrow$	MAE $\downarrow$	$R^{2}\uparrow$	Frob. $\downarrow$	$W_{1}\downarrow$
SC	WGAN-SC	$0.9358{\pm}0.0187$	$0.0480{\pm}0.0078$	$0.8356{\pm}0.0607$	$7.0753{\pm}1.1537$	$0.0258{\pm}0.0135$
	VAE-SC	$0.9536{\pm}0.0200$	$0.0894{\pm}0.0286$	$0.5004{\pm}0.2892$	$11.3880{\pm}3.2387$	$0.0867{\pm}0.0315$
	topoGAN-SC	$0.8504{\pm}0.0141$	$0.1144{\pm}0.0258$	$0.0938{\pm}0.2397$	$16.0885{\pm}2.6417$	$0.1141{\pm}0.0261$
	MGCN-GAN-SC	$0.8088{\pm}0.0222$	$0.1227{\pm}0.0175$	$0.0049{\pm}0.3048$	$17.4149{\pm}1.9215$	$0.1116{\pm}0.0225$
	HYGENE-SC	$0.2800{\pm}0.0286$	$0.4410{\pm}0.0249$	$-10.6193{\pm}0.9216$	$56.2986{\pm}2.7309$	$0.4240{\pm}0.0319$
	MHG-DiT-SC (Ours)	$\bm{0.9594{\pm}0.0164}$	$\bm{0.0361{\pm}0.0068}$	$\bm{0.8984{\pm}0.0394}$	$\bm{5.3415{\pm}1.0790}$	$\bm{0.0193{\pm}0.0104}$

TABLE I : Connectivity-level reconstruction quality on HCP-YA 3T (paired encoder input; see Table II for de novo generation quality and Table IV for sampling efficiency). Best result in each column (per block) is bolded, second-best is underlined.

\uparrow

higher is better;

\downarrow

lower is better.

Connectivity	Method	Deg- $W_{1}\downarrow$	Edge- $W_{1}\downarrow$	NMI $\uparrow$	ARI $\uparrow$	$\|\Delta\mathrm{MI}\|\downarrow$
SC	WGAN-SC	$0.5411$	$\bm{0.0117}$	$0.4029$	$0.3921$	–
	VAE-SC	$0.6815$	$0.0915$	$\bm{0.4109}$	$\bm{0.3942}$	–
	topoGAN-SC	$1.2719$	$0.1191$	$0.4085$	$0.3540$	–
	MGCN-GAN-SC	$2.2099$	$0.1099$	$0.3554$	$0.3525$	–
	HYGENE-SC	$5.5510$	$0.4233$	$0.2791$	$0.1679$	–
	MHG-DiT-SC (Ours)	$10.0485$	$0.5218$	$0.2351$	$0.0536$	–

TABLE II : NMI and ARI are computed by applying data-driven community detection independently to each generated sample and each real test-subject sample. Because generation is unconditional (

\mathbf{z}\sim\mathcal{N}(0,\mathbf{I})

), generated samples have no subject-wise correspondence with the real test subjects. To obtain a population-level estimate independent of arbitrary sample ordering, NMI and ARI are therefore averaged over repeated random permutations of the generated samples relative to the real samples, using the same procedure for all compared methods.

|\Delta\mathrm{MI}|

is defined only for jointly generated SC–FC pairs and is reported once per method, spanning both its SC and FC rows. Best result in each column (per block) is bolded; second-best is underlined.

Fig. 2 : Ground truth versus generated SC and FC matrices, averaged across all $n{=}70$ test subjects, for MHG-FM (top) and MHG-DiT (bottom). Columns: SC ground truth/generated/residual, FC ground truth/generated/residual.

	SC			FC			Coupling
Model	$r\,\uparrow$	MAE $\,\downarrow$	$R^{2}\,\uparrow$	$r\,\uparrow$	MAE $\,\downarrow$	$R^{2}\,\uparrow$	$\|\Delta\mathrm{MI}\|\,\downarrow$
Unimodal variants
MHG-FM-SC	$0.9526{\pm}0.0207$	$0.0409{\pm}0.0115$	$0.8673{\pm}0.0713$	n/a	n/a	n/a	–
MHG-FM-FC	n/a	n/a	n/a	$0.8359{\pm}0.0450$	$0.0834{\pm}0.0185$	$0.6625{\pm}0.0911$	–
Generative-backbone variants
MHG-A1	$0.9590{\pm}0.0203$	$0.0371{\pm}0.0073$	$0.8928{\pm}0.0480$	$0.8118{\pm}0.0470$	$0.0867{\pm}0.0207$	$0.6286{\pm}0.0946$	$0.0258$

TABLE III : Ablation study on HCP-YA 3T (

n_{\mathrm{test}}=70

), paired-encoder reconstruction mode. Higher is better for

r

and

R^{2}

; lower is better for MAE and

|\Delta\mathrm{MI}|

[ 9 , 7 ] . Bold indicates the best result among the five joint (SC+FC) variants (generative-backbone and encoder groups); the unimodal rows are shown for reference and excluded from this ranking, since removing a modality changes what is being measured. See notes below the table for a brief description of each variant.

Fig. 3 : Effect sizes (matched-pairs rank-biserial correlation $r$ ) for MHG-FM versus MHG-DiT in reconstruction mode ( $n=70$ ). Positive values favour MHG-FM and negative values favour MHG-DiT. Filled markers indicate significance after Holm–Bonferroni correction ( $p_{\mathrm{corr}}<0.05$ ); open markers indicate non-significant differences. Shaded bands denote conventional small, medium, and large effect-size thresholds ( $|r|\approx 0.1/0.3/0.5$ ).

Method	Params (M) $\downarrow$	Time/sample (ms) $\downarrow$	Peak Mem. (GB) $\downarrow$	NFE $\downarrow$
WGAN-Joint	$7.1490$	$0.3170$	$0.0357$	$1$
VAE-Joint	$56.5630$	$0.2630$	$0.8555$	$1$
topoGAN-Joint	$3.4940$	$0.3640$	$0.0210$	$1$
MGCN-GAN-Joint	$0.4590$	$0.7680$	$0.0184$	$1$
HYGENE-Joint	$0.8460$	$83.0000$	$0.0231$	$50$
MHG-DiT (Ours)	$14.0450$	$848.2320$	$0.2407$	$250$

TABLE IV : Computational cost and sampling efficiency on HCP-YA 3T (

n_{test}=70

). NFE: number of function evaluations of the denoising/velocity network per generated sample (mean

\pm

SD where the solver is adaptive; single forward pass reported as N/A/1 for non-iterative methods). Time/sample: wall-clock generation time per paired SC–FC sample. Peak Mem.: peak GPU memory during inference, at matched batch size. Lower is better for every column. All measured on NVIDIA L40S.

	Method	$r\uparrow$	MAE $\downarrow$	$R^{2}\uparrow$	Frob. $\downarrow$	$W_{1}\downarrow$
FC $\to$ SC	topoGAN	$0.8720{\pm}0.0110$	$0.0780{\pm}0.0210$	$0.4980{\pm}0.1580$	$11.9500{\pm}2.2300$	$0.0705{\pm}0.0263$
	MGCN-GAN	$0.7812{\pm}0.0452$	$0.1227{\pm}0.0059$	$-0.0311{\pm}0.0418$	$17.9772{\pm}0.8867$	$0.1227{\pm}0.0059$
	MHG-DiT (Ours)	$0.9344{\pm}0.0280$	$0.0466{\pm}0.0090$	$0.8308{\pm}0.0760$	$6.8246{\pm}1.3387$	$0.0280{\pm}0.0122$
	MHG-FM (Ours)	$\bm{0.9414{\pm}0.0236}$	$\bm{0.0444{\pm}0.0081}$	$\bm{0.8474{\pm}0.0613}$	$\bm{6.5208{\pm}1.2258}$	$\bm{0.0271{\pm}0.0112}$
SC $\to$ FC	topoGAN	$0.5375{\pm}0.0637$	$0.1397{\pm}0.0308$	$0.0913{\pm}0.1975$	$23.4433{\pm}3.9359$	$0.0737{\pm}0.0411$
	MGCN-GAN	$0.5630{\pm}0.0626$	$0.1543{\pm}0.0304$	$-0.0547{\pm}0.0690$	$24.6287{\pm}4.2944$	$0.1543{\pm}0.0304$

TABLE V : Cross-modal translation quality on HCP-YA 3T. Top: FC

\to

SC. Bottom: SC

\to

FC. Best result in each column (across all four methods) is bolded, second-best underlined.

\uparrow

higher is better;

\downarrow

lower is better. GAN-Joint and VAE-Joint are omitted as their joint encoders require both SC and FC as input and cannot perform unimodal cross-modal inference.

Explore similar work

Mar 21, 2026

Hierarchical Multiscale Structure-Function Coupling for Brain Connectome Integration

Integrating structural and functional connectomes remains challenging because their relationship is non-linear and organized over nested modular hierarchies. We propose a hierarchical multiscale structure-function coupling framework for connectome integration that jointly learns individualized modular organization and hierarchical coupling across structural connectivity (SC) and functional connectivity (FC). The framework includes: (i) Prototype-based Modular Pooling (PMPool), which learns modality-specific multiscale communities by selecting prototypical ROIs and optimizing a differentiable modularity-inspired objective; (ii) an Attention-based Hierarchical Coupling Module (AHCM) that models both within-hierarchy and cross-hierarchy SC-FC interactions to produce enriched hierarchical coupling representations; and (iii) a Coupling-guided Clustering loss (CgC-Loss) that regularizes SC and FC community assignments with coupling signals, allowing cross-modal interactions to shape community alignment across hierarchies. We evaluate the model's performance across four cohorts for predicting brain age, cognitive score, and disease classification. Our model consistently outperforms baselines and other state-of-the-art approaches across three tasks. Ablation and sensitivity analyses verify the contributions of key components. Finally, the visualizations of learned coupling reveal interpretable differences, suggesting that the framework captures biologically meaningful structure-function relationships.

Jianwei Chen, Zhengyang Miao, Wenjie Cai +10

School of Medicine, University of Dundee, UK · College of Medicine and Biological Information Engineering, Northeastern University, China · School of Airspace Science and Engineering, Shandong University, China +5

Brain Network Analysis Neuroimaging Functional Connectivity Multimodal Graph Learning Hierarchical Representation Learning

Ask PDF

May 31, 2026

NeuroAlign: Hierarchical Multimodal Fusion of Dynamic and Structural Neuroimaging for MCI Analysis

Multimodal neuroimaging fusion of functional MRI (fMRI) and diffusion tensor imaging (DTI) provides complementary information for cognitive impairment analysis, but remains challenged by heterogeneous feature spaces and misaligned representations. We propose NeuroAlign, a hierarchical framework for structured multimodal fusion. It introduces (1) Dual-Modal Hierarchical Alignment (DMHA), which models multi-scale dynamic connectivity and aligns dynamic-static and functional-structural embeddings; and (2) Dual-Domain Hierarchical Interaction (DDHI), which enables fine-grained modulation and global interaction between connectivity- and region-level features. To support feature-level inspection, we design Synergistic Activation Mapping (SAM), a gradient-free, marker-oriented attribution method for DFC, SFC, ALFF, and FA. Evaluated on GUTCM, ADNI, and OASIS under five-fold validation, NeuroAlign achieves competitive MCI/SCD detection and preliminary cross-dataset transferability. Attribution analyses reveal modality-specific and partially consistent brain patterns, providing model-derived evidence for multimodal representation analysis.

Xiongri Shen, Zhenxi Song, Jiaqi wang +13

Department of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, 518055, China · School of Intelligence Science and Engineering, College of Artificial Intelligence, Harbin Institute of Technology, Shenzhen, 518055, China · School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China +5

Feature Attribution Functional Connectivity Cross-Modal Alignment Neuroimaging Multimodal Fusion

Ask PDF

Mar 31, 2026

Learning Structural-Functional Brain Representations through Multi-Scale Adaptive Graph Attention for Cognitive Insight

Understanding how brain structure and function interact is key to explaining intelligence yet modeling them jointly is challenging as the structural and functional connectome capture complementary aspects of organization. We introduced Multi-scale Adaptive Graph Network (MAGNet), a Transformer-style graph neural network framework that adaptively learns structure-function interactions. MAGNet leverages source-based morphometry from structural MRI to extract inter-regional morphological features and fuses them with functional network connectivity from resting-state fMRI. A hybrid graph integrates direct and indirect pathways, while local-global attention refines connectivity importance and a joint loss simultaneously enforces cross-modal coherence and optimizes the prediction objective end-to-end. On the ABCD dataset, MAGNet outperformed relevant baselines, demonstrating effective multimodal integration for advancing our understanding of cognitive function.

Badhan Mazumder, Sir-Lord Wiafe, Aline Kotoski +2

Department of Computer Science, Georgia State University · Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, and Emory University · Neuroscience Institute, Georgia State University

Brain Network Analysis Multimodal Graph Learning Neuroimaging Graph Attention Networks Functional Connectivity

Ask PDF