MnemoDyn: Learning Resting State Dynamics from 40K FMRI sequences

Sourav Pal; Viet Luong; Hoseok Lee; Tingting Dan; Guorong Wu; Richard Davidson; Won Hwa Kim; Vikas Singh

cs.LGAug 25, 2026

MnemoDyn: Learning Resting State Dynamics from 40K FMRI sequences

Authors: Sourav Pal, Viet Luong, Hoseok Lee, Tingting Dan, Guorong Wu, Richard Davidson, Won Hwa Kim, Vikas Singh

Organizations: University of Wisconsin–Madison, Madison, United States · Pohang University of Science and Technology (POSTECH), Pohang, South Korea · University of North Carolina at Chapel Hill, Chapel Hill, United States

Abstract

We present a dynamical-systems based model for resting-state functional magnetic resonance imaging (rs-fMRI), trained on a dataset of roughly 40K rs-fMRI sequences covering a wide variety of public and available-by-permission datasets. While most existing proposals use transformer backbones, we utilize multi-resolution temporal modeling of the dynamics across parcellated brain regions. We show that MnemoDyn is compute efficient and generalizes very well across diverse populations and scanning protocols. When benchmarked against current state-of-the-art transformer-based approaches, MnemoDyn consistently delivers superior reconstruction quality. Overall, we find that with such large-scale pre-training on (non-proprietary) rs-fMRI datasets, we get a highly performant model for various downstream tasks. Our results also provide evidence of the efficacy of the model on small sample size studies which has implications for neuroimaging studies at large where resting state fMRI is a commonly acquired imaging modality.

Figures & tables

Figure 1: Overview of the MnemoDyn framework. Our foundation model for rs-fMRI treats temporal signals as trajectories in a latent dynamical system, parameterized by learnable operators. The pipeline begins with data curation and pre-processing of large-scale rs-fMRI cohorts into standardized gray-ordinate representations. The pre-training stage learns a multi-resolution non-linear dynamical operator via $\frac{dz(t)}{dt}=f_{\theta}((z(t))$ , enabling the model to capture cross-scale temporal dependencies while preserving local dynamics. During fine-tuning, lightweight adapter layers adapt the pretrained model to diverse downstream cohorts. The resulting representations support prediction of clinical variables across heterogeneous populations (age, sex, cognitive traits, neurodegeneration markers, etc.), highlighting MnemoDyn’s ability to generalize from large-scale dynamical modeling to scientifically meaningful tasks.

Figure 2: Wavelet-based multi-resolution decomposition of rs-fMRI signals and operator along with CP tensor factorization of model parameters enable efficient computation in MnemoDyn .

Panel of Figure 3 — Figure 3: Reconstruction of rs-fMRI for UK-Biobank. Three parcels of unseen data are shown. We show the output of vanilla auto-encoding on the left and masked auto-encoding on the right.

Foundation Model	UK-Biobank	HCP
Foundation Model	(MSE, R 2 )	(MSE, R 2 )
MnemoDyn-UKB	$2.36\text{e-}5,\;0.985$	$4.52\text{e-}08,\;0.934$
MnemoDyn-HCP	$1.86\text{e-}09,\;0.969$	$3.94\text{e-}06,\;0.987$

Table 1: Validation reconstruction MSE and

R^{2}

score. We see good generalization across foundation models trained with different data.

Figure 4: Masked pre-training and fine-tuning of MnemoDyn . We note that the dynamics are evolved in the latent space followed by the encoder, which is eventually used to fine-tune adapter layers for downstream tasks. In pre-training phase, loss is computed only on the unseen spatio-temporal data.

Methods	NC/MCI (ADNI)		Amyloid $+$ ve/ $-$ ve (ADNI)		Age (UKB)	Sex (UKB)
Methods	ACC(%) $\uparrow$	F1(%) $\uparrow$	ACC(%) $\uparrow$	F1(%) $\uparrow$	MSE $\downarrow$	ACC(%) $\uparrow$	F1(%) $\uparrow$
BrainNetCNN	60.00 (3.51)	64.72 (3.18)	59.00 (2.00)	59.43 (1.14)	0.99 (0.03)	77.86 (0.98)	78.17 (0.86)
BrainGNN	67.40 (2.93)	71.42 (2.87)	57.00 (4.00)	62.61 (3.48)	0.93 (0.04)	77.31 (0.33)	79.23 (0.31)
BNT	78.90 (4.12)	83.14 (3.58)	62.00 (2.45)	59.53 (0.58)	0.86 (0.03)	80.78 (0.40)	82.42 (0.36)
BrainLM	75.79 (1.05)	85.66 (1.27)	67.00 (7.48)	68.82 (8.48)	0.61 (0.04)	86.47 (0.74)	86.84 (0.43)
Brain-JEPA	76.84 (1.05)	86.32 (0.54)	71.00 (4.90)	75.97 (3.93)	0.50 (0.03)	88.17 (0.06)	88.58 (0.11)

Table 2: Test set performance in terms of mean (standard deviation) of MnemoDyn when fine-tuned on separate predictive tasks. Results include disease diagnosis (NC vs. MCI) and biomarker status prediction (Amyloid

+

ve/

-

ve) from ADNI (left), as well as age regression and sex classification from UK Biobank (right). Different fine-tuned variants of MnemoDyn consistently achieve lower error and higher/comparable accuracy/F1 compared to prior baselines. We report F1 score in addition to accuracy to take into account class imbalance.

Methods	Age	Sex		Neuroticism	Flanker
Methods	MSE $\downarrow$	ACC (%) $\uparrow$	F1 (%) $\uparrow$	MSE $\downarrow$	MSE $\downarrow$
BrainLM	1.14 (0.22)	75.27 (1.24)	73.19 (1.12)	1.05 (0.12)	0.77 (0.11)
Brain-JEPA	1.02 (0.01)	79.17 (1.29)	76.29 (1.17)	0.99 (0.01)	1.28 (0.01)
MnemoDyn-Denoise	0.91 (0.02)	80.2 (0.12)	80.11 (0.11)	0.91 (0.02)	0.61 (0.02)
MnemoDyn-Mask	0.90 (0.01)	83.10 (0.57)	82.77 (0.54)	0.90 (0.03)	0.60 (0.01)
MnemoDyn-Mask-JEPA	0.90 (0.01)	82.57 (0.35)	82.23(0.41)	0.90 (0.02)	0.60 (0.01)

Table 3: Performance of MnemoDyn when fine-tuned on External tasks of demographics and trait prediction on HCP-Aging. We report mean(standard deviation) for each metric. Our model achieves strong improvements in predicting age, sex, and cognitive/behavioral traits (Neuroticism, Flanker) on the test set. We report F1 score in addition to accuracy account for class imbalance.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Methods	Sex		Age
Methods	ACC(%) $\uparrow$	F1(%) $\uparrow$	MSE $\downarrow$
Brain-JEPA	58.52 (3.72)	29.12 (1.64)	1.02 (0.01)
MnemoDyn	82.37 (2.1)	82.19 (1.9)	0.84 (0.01)
MnemoDyn-Mask	81.55 (1.6)	81.35 (1.6)	0.80 (0.01)

Appendix

Table 4: Performance of fine-tuned MnemoDyn on Healthy Brain Network (HBN). We report mean(standard deviation) on the test set. For both regression and classification, we get better performance than baseline.

Figure 5: Operator output $A_{j}(u_{j})$ across wavelet levels $j=0$ to $4$ .

Figure 6: $L_{2}$ norm of operator output $A_{j}(u_{j})$ across wavelet levels $j=0$ to $4$ .

# Blocks	Age (MSE $\downarrow$ )	Sex (Acc; F1 $\uparrow$ )
$1$	$0.914(0.012)$	$0.768(0.013)$ $0.763(0.011)$
$2$	$0.903(0.014)$	$0.8010(0.016)$ ; $0.807(0.014)$

Appendix

Table 5: Model-size scaling: downstream task performance for different numbers of operator blocks for HCP-Aging .

# Subjects	Age (MSE $\downarrow$ )	Sex (Acc; F1 $\uparrow$ )
$1000$	$1.023(0.019)$	$0.706(0.017)$ ; $0.707(0.019)$
$10000$	$1.021(0.014)$	$0.753(0.013)$ ; $0.743(0.011)$
$20000$	$1.012(0.013)$	$0.793(0.013)$ ; $0.790(0.011)$

Appendix

Table 6: Data-size scaling: downstream performance when pre-trained on different fraction of UK-Biobank and finetuned for HCP-Aging.

Task	Metric	No-Wavelet	MnemoDyn-Mask
NC/MCI (ADNI)	ACC / F1	75.12 $\pm$ 1.02 / 74.39 $\pm$ 1.01	96.12 $\pm$ 0.31 / 95.98 $\pm$ 0.29
Amyloid (ADNI)	ACC / F1	72.19 $\pm$ 2.01 / 71.23 $\pm$ 1.78	95.27 $\pm$ 0.39 / 95.61 $\pm$ 0.37
Age (UKB)	MSE	0.67 $\pm$ 0.07	0.44 $\pm$ 0.05
Sex (UKB)	ACC / F1	76.16 $\pm$ 0.13 / 75.14 $\pm$ 0.23	88.40 $\pm$ 0.32 / 88.27 $\pm$ 0.41

Appendix

Table 7: Comparison of MnemoDyn with and without multi-resolution wavelet decomposition.

Methods	ADHD/TDC
Methods	ACC(%) $\uparrow$	F1(%) $\uparrow$
Brain-JEPA	45.7 (0.89)	46.5 (0.78)
MnemoDyn	54.24 (0.3)	54.07 (0.42)
MnemoDyn-Mask	54.36 (1.24)	54.26 (1.27)
MnemoDyn-Mask-JEPA	54.70 (0.75)	54.55 (0.80)

Appendix

Table 8: Performance comparison on the ADHD200 dataset for ADHD vs. TDC classification. Mean accuracy (ACC) and F1-score (F1) with standard deviations are reported. Fine-tuned variants of MnemoDyn consistently outperform the Brain-JEPA baseline.

Methods	Sex
Methods	ACC(%) $\uparrow$	F1(%) $\uparrow$
Brain-JEPA	66.52 (0.27)	63.97 (0.40)
MnemoDyn	87.52 (0.49)	87.52 (0.48)
MnemoDyn-Mask	88.37 (0.45)	88.36 (0.45)
MnemoDyn-Mask-JEPA	87.70 (0.29)	87.70 (0.27)

Appendix

Table 9: Performance comparison on the NKIR dataset for sex classification. We report mean accuracy (ACC) and F1-score (F1) with standard deviations in parentheses. Finetuned MnemoDyn significantly outperform the Brain-JEPA baseline.

Methods	Autism / TSC
Methods	ACC(%) $\uparrow$	F1(%) $\uparrow$
MnemoDyn	60.32 (1.44)	59.26 (2.04)
MnemoDyn-Mask	58.93 (1.25)	58.57 (1.24)

Appendix

Table 10: Classification performance on the ABIDE dataset for Autism vs. TSC for fine-tuned MnemoDyn. The table reports mean accuracy (ACC) and F1-score (F1) with standard deviations in parentheses, averaged across multiple runs.

Methods	NC/MCI		Amyloid $+$ ve/ $-$ ve
Methods	ACC(%) $\uparrow$	F1(%) $\uparrow$	ACC(%) $\uparrow$	F1(%) $\uparrow$
BrainNetCNN	60.00 (3.51)	64.72 (3.18)	59.00 (2.00)	59.43 (1.14)
BrainGNN	67.40 (2.93)	71.42 (2.87)	57.00 (4.00)	62.61 (3.48)
BNT	78.90 (4.12)	83.14 (3.58)	62.00 (2.45)	59.53 (0.58)
BrainLM	75.79 (1.05)	85.66 (1.27)	67.00 (7.48)	68.82 (8.48)
Brain-JEPA	76.84 (1.05)	86.32 (0.54)	71.00 (4.90)	75.97 (3.93)

Appendix

Table 11: Performance of MnemoDyn pre-trained on HCP and fine-tuned on ADNI tasks of NC/MCI diagnosis and Amyloid

+

ve/

-

ve classification. Results are reported as mean (standard deviation) for accuracy (ACC) and F1-score (F1). F1 is included to account for class imbalance across cohorts. MnemoDyn variants substantially outperform prior baselines, highlighting the benefits of pretraining on HCP before transfer to clinical tasks.

Methods	Age	Sex		Neuroticism	Flanker
Methods	MSE $\downarrow$	ACC (%) $\uparrow$	F1 (%) $\uparrow$	MSE $\downarrow$	MSE $\downarrow$
BrainLM	1.14 (0.22)	75.27 (1.24)	73.19 (1.12)	1.05 (0.12)	0.77 (0.11)
Brain-JEPA	1.02 (0.01)	79.17 (1.29)	76.29 (1.17)	0.99 (0.01)	1.28 (0.01)
MnemoDyn-HCP-Mask	0.90 (0.03)	80.90 (0.13)	80.70 (0.12)	0.90 (0.09)	0.58 (0.09)
MnemoDyn-HCP-Mask-JEPA	0.90 (0.04)	80.98 (2.13)	80.63 (2.12)	0.90 (0.57)	0.60(0.06)

Appendix

Table 12: Performance of MnemoDyn pre-trained on HCP and fine-tuned on external tasks from HCP-Aging, including age regression, sex classification, and trait prediction (Neuroticism, Flanker). Results are reported as mean (standard deviation) for MSE, accuracy (ACC), and F1-score (F1). F1 is included for classification tasks to account for class imbalance. MnemoDyn variants achieve consistent improvements over prior baselines across both demographic and cognitive trait prediction.

fMRI Foundation Models

Functional magnetic resonance imaging (fMRI) foundation models learn reusable brain representations for prediction, transfer, and neuroscience discovery.

See all 15 papers

FAST-Brain: A Flow-Aligned Spatio-Temporal Surrogate Brain Model
Sep 28, 2026Shucheng Liu, Changchun Shi, Kai Zhang +1Neural Surrogate Modeling Functional Connectivity
University of North Carolina at Chapel Hill · London School of Economics and Political Science
Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining
Apr 14, 2026Junfeng Xia, Wenhao Ye, Xuanye Pan +3Medical Imaging Foundation Models Diffusion Transformer
Department of Biomedical Engineering, Southern University of Science and Technology, China · School of Biomedical Engineering, Shenzhen University, China
Can Natural Image Autoencoders Compactly Tokenize fMRI Volumes for Long-Range Dynamics Modeling?
Apr 4, 2026Peter Yongho Kim, Juhyeon Park, Jungwoo Park +4fMRI Representation Learning Long-Context Modeling
ECE, Seoul National University · IPAI, Seoul National University · BCS, Seoul National University +2

Sep 28, 2026J/K move · Enter open · S save

FAST-Brain: A Flow-Aligned Spatio-Temporal Surrogate Brain Model

Shucheng Liu, Changchun Shi, Kai Zhang, Hongtu Zhu

University of North Carolina at Chapel Hill · London School of Economics and Political Science

Neural Surrogate Modeling Functional Connectivity Generative Modeling Flow-Based Generative Modeling Neuroimaging Spatiotemporal GNNs+1

Open paper AskPDF arXiv

Modeling resting-state functional magnetic resonance imaging (rs-fMRI) data is crucial for understanding brain-wide neural activity. However, traditional methods struggle to capture complex temporal dynamics over long horizons, to account for the brain's anatomical spatial structure, and to model high-dimensional ambient signals that lie on a low-dimensional intrinsic subspace. We propose FAST-Brain, a unified flow-aligned spatio-temporal surrogate brain model that addresses all three challenges. At its core is a flow-aligned generative framework that directly predicts the clean blood-oxygen-level-dependent (BOLD) signal, paired with a graph convolutional network that captures spatial structural constraints and a Transformer that models long-range temporal dependencies. Theoretically, we show that under a low-dimensional subspace assumption, the approximation error of our model scales with the intrinsic dimension rather than the ambient dimension, which justifies our direct modeling of the BOLD signal. Extensive experiments on synthetic and Human Connectome Project datasets demonstrate that FAST-Brain achieves state-of-the-art performance in recovering functional connectivity, effective connectivity, and the implicit low-dimensional signal subspace.

Abstract

fMRI Foundation Models

Explore similar work

FAST-Brain: A Flow-Aligned Spatio-Temporal Surrogate Brain Model

Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining

Can Natural Image Autoencoders Compactly Tokenize fMRI Volumes for Long-Range Dynamics Modeling?