BrainCast: A Spatio-Temporal Forecasting Model for Whole-Brain fMRI Time Series Prediction
Authors: Yunlong Gao, Jinbo Yang, Li Xiao, Haiye Huo, Yang Ji, Hao Wang, Aiying Zhang, Yu-Ping Wang
Organizations: Institute of Advanced Technology, University of Science and Technology of China, Hefei 230052, China. · MoE Key Laboratory of Brain-Inspired Intelligence Perception and Cognition, University of Science and Technology of China, Hefei 230052, China. · School of Mathematics and Computer Sciences, Nanchang University, Nanchang 330031, China. · Anhui Province Key Laboratory of Biomedical Imaging and Intelligent Processing, HFCNS Institute of Artificial Intelligence, Hefei 230088, China. · School of Data Science, University of Virginia, Charlottesville, VA 22903, USA. · Department of Biomedical Engineering, Tulane University, New Orleans, LA 70118, USA.
Functional magnetic resonance imaging (fMRI) enables noninvasive investigation of brain function, while short clinical scan durations, arising from human and non-human factors, usually lead to reduced data quality and limited statistical power for neuroimaging research. In this paper, we propose BrainCast, a novel spatio-temporal forecasting framework specifically tailored for whole-brain fMRI time series forecasting, to extend informative fMRI time series without additional data acquisition. It formulates fMRI time series forecasting as a multivariate time series prediction task and jointly models temporal dynamics within regions of interest (ROIs) and spatial interactions across ROIs. Specifically, BrainCast integrates a Spatial Interaction Awareness module to characterize inter-ROI dependencies via embedding every ROI time series as a token, a Temporal Feature Refinement module to capture intrinsic neural dynamics within each ROI by enhancing both low- and high-energy temporal components of fMRI time series at the ROI level, and a Spatio-temporal Pattern Alignment module to combine spatial and temporal representations for producing informative whole-brain features. Experimental results on resting-state and task fMRI datasets from the Human Connectome Project demonstrate the superiority of BrainCast over state-of-the-art time series forecasting baselines. Moreover, fMRI time series extended by BrainCast improve downstream cognitive ability prediction, highlighting the clinical and neuroscientific impact brought by whole-brain fMRI time series forecasting in scenarios with restricted scan durations.
Figures & tables
Fig. 1 : The overall framework of BrainCast. (a) fMRI data preprocessing; (b) The Spatial Interaction Awareness (SIA) module for learning between-ROI spatial interactions; (c) The Temporal Feature Refinement (TFR) module for extracting temporal dy- namics within every ROI; and (d) The Spatio-temporal Pattern Alignment (SPA) module for aligning spatio-temporal represent- ations across ROIs and time points.
Fig. 2 : A sliding window partition technique on the fMRI data to generate sequential samples.
Fig. 3 : (a) Illustration of SIAformer, in which self-attention (b) is applied to learning spatial interactions between ROIs.
fMRI paradigm
Frames per run
Run duration (min:sec)
Resting-state
1200
14:33
Working memory
405
5:01
Emotional processing
176
2:16
Language processing
316
3:57
Relational reasoning
232
2:56
Social cognition
274
3:27
TABLE I : Run duration and the number of frames (i.e., time points) per run for each fMRI paradigm.
Dataset
Metric
Model
Ours
iTransformer
DLinear
Amplifier
PatchTST
CrossFormer
FourierGNN
LSTM
RNN
HCP-rs-fMRI
MSE ↓
0.380 (0.004)
0.398 (0.002)
0.529 (0.012)
0.407 (0.006)
0.394 (0.003)
0.417 (0.001)
0.562 (0.005)
0.602 (0.013)
0.592 (0.006)
MAE ↓
0.461 (0.004)
0.478 (0.002)
0.562 (0.009)
0.481 (0.005)
0.473 (0.004)
0.488 (0.003)
0.580 (0.004)
0.601 (0.011)
0.597 (0.008)
R ↑
0.665 (0.002)
0.631 (0.001)
0.446 (0.013)
0.627 (0.006)
0.635 (0.003)
0.628 (0.004)
0.432 (0.002)
0.355 (0.008)
0.379 (0.005)
R 2↑
0.438 (0.003)
0.397 (0.003)
0.199 (0.011)
0.390 (0.003)
0.402 (0.005)
0.396 (0.006)
0.187 (0.003)
0.120 (0.007)
0.143 (0.007)
HCP-t-fMRI
MSE ↓
0.435 (0.004)
0.488 (0.004)
0.633 (0.005)
0.460 (0.009)
0.485 (0.006)
0.480 (0.010)
0.705 (0.007)
0.681 (0.005)
0.663 (0.004)
TABLE II : Performance comparison of different models on HCP-rs-fMRI and HCP-t-fMRI datasets. The mean is reported with the standard deviation shown in parentheses below. The best results are highlighted in bold.
Fig. 4 : Visualization of forecasting results on a randomly selected ROI of one test sample for HCP-rs-fMRI (a) and HCP-t-fMRI (b), respectively. The black dots denote the previous and ground-truth future data, and the blue dots denote the predicted data.
Fig. 5 : Ablation results for the SIA, TFR, and SPA modules in our BrainCast on both HCP-rs-fMRI (a) and HCP-t-fMRI (b), respectively. The dark blue bars correspond to the left y-axis, whereas the dark red bars correspond to the right y-axis.
Fig. 6 : Visualization of the cortical attention maps in the coronal, sagittal, and axial planes, where red indicates higher attention scores (i.e., greater contribution to the model’s prediction) and blue indicates lower contributions.
Fig. 7 : The R results of BrainCast with varying look-back len- gths (i.e., the length of the previous fMRI time series L ) and the fixed future length (i.e., the length of the future fMRI time series T=20 ) on both HCP-rs-fMRI and HCP-t-fMRI.
Method
HCP-rs-fMRI (previous)
HCP-rs-fMRI (prolonged)
BrainNetCNN
5.0155 ± 0.0017
4.8925 ± 0.0303
MLP
4.8778 ± 0.0382
4.7443 ± 0.0221
GCN
4.8473 ± 0.0150
4.7383 ± 0.0470
SVR
4.7698 ± 0.0121
4.6361 ± 0.0598
TABLE III : PMAT prediction performance on HCP-rs-fMRI, measured in terms of RMSE ( mean±standard deviation ).
Modeling resting-state functional magnetic resonance imaging (rs-fMRI) data is crucial for understanding brain-wide neural activity. However, traditional methods struggle to capture complex temporal dynamics over long horizons, to account for the brain's anatomical spatial structure, and to model high-dimensional ambient signals that lie on a low-dimensional intrinsic subspace. We propose FAST-Brain, a unified flow-aligned spatio-temporal surrogate brain model that addresses all three challenges. At its core is a flow-aligned generative framework that directly predicts the clean blood-oxygen-level-dependent (BOLD) signal, paired with a graph convolutional network that captures spatial structural constraints and a Transformer that models long-range temporal dependencies. Theoretically, we show that under a low-dimensional subspace assumption, the approximation error of our model scales with the intrinsic dimension rather than the ambient dimension, which justifies our direct modeling of the BOLD signal. Extensive experiments on synthetic and Human Connectome Project datasets demonstrate that FAST-Brain achieves state-of-the-art performance in recovering functional connectivity, effective connectivity, and the implicit low-dimensional signal subspace.
Shucheng Liu, Changchun Shi, Kai Zhang +1
University of North Carolina at Chapel Hill · London School of Economics and Political Science
Functional magnetic resonance imaging (fMRI) commonly characterizes large-scale brain organization through functional connectivity (FC). However, an observed FC matrix is not a fixed brain graph but a statistical estimate determined by the finite BOLD frames used to compute it. Conventional FC-based approaches typically represent each scan with a single Pearson-correlation matrix, leaving frame-resolved variations in regional interactions unexploited. To address this limitation, we introduce CoFlu-Brain, which organizes framewise cofluctuation observations into multiple retained and dropped views, each summarized as an FC matrix. The model learns connectivity structure reproducible across retained views and predicts the representations of the corresponding dropped views, while adopting SIGReg from LeJEPA to prevent representation collapse. We pretrained CoFlu-Brain on 13,444 subjects from eight dataset groups and evaluated its representations via frozen-encoder linear probing on seven classification and two age-prediction tasks across seven datasets. CoFlu-Brain showed strong transfer across demographic, developmental, and clinical prediction tasks, outperforming the evaluated released models on most tasks. Random frame partitioning improved aggregate transfer, while the complete objective achieved the strongest overall performance. Code is available.
Ziteng Sui, Junfeng Xia, Wenhao Ye +2
Department of Biomedical Engineering, Southern University of Science and Technology, China · School of Biomedical Engineering, Shenzhen University, China · Department of Computer Science, University of Warwick, UK
Functional Magnetic Resonance Imaging (fMRI) is an imaging technique widely used to study human brain activity. fMRI signals in areas across the brain transiently synchronise and desynchronise their activity in a highly structured manner, even when an individual is at rest. These functional connectivity dynamics may be related to behaviour and neuropsychiatric disease. To model these dynamics, temporal brain connectivity representations are essential, as they reflect evolving interactions between brain regions and provide insight into transient neural states and network reconfigurations. However, conventional graph neural networks (GNNs) often struggle to capture long-range temporal dependencies in dynamic fMRI data. To address this challenge, we propose BrainATCL, an unsupervised, nonparametric framework for adaptive temporal brain connectivity learning, enabling functional link prediction and age estimation. Our method dynamically adjusts the lookback window for each snapshot based on the rate of newly added edges. Graph sequences are subsequently encoded using a GINE-Mamba2 backbone to learn spatial-temporal representations of dynamic functional connectivity in resting-state fMRI data of 1,000 participants from the Human Connectome Project. To further improve spatial modeling, we incorporate brain structure and function-informed edge attributes, i.e., the left/right hemispheric identity and subnetwork membership of brain regions, enabling the model to capture biologically meaningful topological patterns. We evaluate our BrainATCL on two tasks: functional link prediction and age estimation. The experimental results demonstrate superior performance and strong generalization, including in cross-session prediction scenarios.