CalM: A Self-Supervised Foundation Model for Population Dynamics in Calcium Imaging Data

Xinhong Xu; Yimeng Zhang; Qichen Qian; Yuanlong Zhang

q-bio.QMApr 3, 2026

CalM: A Self-Supervised Foundation Model for Population Dynamics in Calcium Imaging Data

Authors: Xinhong Xu, Yimeng Zhang, Qichen Qian, Yuanlong Zhang

Organizations: Department of Automation, Tsinghua University, Beijing, China · School of Life Sciences, Tsinghua University, Beijing, China

Abstract

Recent work suggests that large-scale, multi-animal modeling can significantly improve neural recording analysis. However, for functional calcium traces, existing approaches remain task-specific, limiting transfer across common neuroscience objectives. To address this challenge, we propose CalM, a self-supervised neural foundation model trained solely on neuronal calcium traces and adaptable to multiple downstream tasks, including forecasting and decoding. Our key contribution is a pretraining framework, composed of a high-performance tokenizer mapping single-neuron traces into a shared discrete vocabulary, and a dual-axis autoregressive transformer modeling dependencies along both the neural and the temporal axis. We evaluate CalM on a large-scale, multi-animal, multi-session dataset. On the neural population dynamics forecasting task, CalM achieves competitive performance against strong specialized baselines after pretraining. With a task-specific head, CalM further adapts to the behavior decoding task and achieves superior results compared with supervised decoding models. Moreover, linear analyses of CalM representations reveal interpretable functional structures beyond predictive accuracy. Taken together, we propose a novel and effective self-supervised pretraining paradigm for foundation models based on calcium traces, paving the way for scalable pretraining and broad applications in functional neural analysis. Code is released at https://github.com/TSuXinH/CalM.

Figures & tables

Metric: corr	Held in	Held out
POCO	0.2781 $\pm$ 0.0343	0.2568 $\pm$ 0.0289
CalM	0.2783 $\pm$ 0.0562	0.2592 $\pm$ 0.0364

Table 1: Forecasting results for extra mouse dataset

Metric: corr	Held in	Held out
POCO	0.1935 $\pm$ 0.0670	0.1969 $\pm$ 0.0468
CalM	0.1841 $\pm$ 0.0898	0.1601 $\pm$ 0.0757

Table 2: Forecasting results for C. elegans dataset

Metric: $R^{2}$	Held in	Held out
POYO+	0.8810 $\pm$ 0.0745	0.8870 $\pm$ 0.0649
CalM	0.9706 $\pm$ 0.0241	0.9581 $\pm$ 0.0376

Table 3: Decoding position results for extra mouse dataset

Metric: $R^{2}$	Held in	Held out
POYO+	0.6922 $\pm$ 0.2321	0.7394 $\pm$ 0.0950
CalM	0.7065 $\pm$ 0.2658	0.7443 $\pm$ 0.1010

Table 4: Decoding velocity results for C. elegans dataset

Metric: $R^{2}$	Held in	Held out
POYO+	0.4597 $\pm$ 0.1469	0.4953 $\pm$ 0.1145
CalM	0.5149 $\pm$ 0.1378	0.5606 $\pm$ 0.0807

Table 5: Decoding head angle results for C. elegans dataset

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Subject id	3	4	5	6	7	8	9	10
Number of sessions	17	23	31	46	35	42	41	51
Number of neurons	17861	20592	34293	40262	36621	42550	45787	35804
Mean trial frames	87	105	90	83	90	95	77	104

Appendix

Table 6: Dataset distribution on mouse subject

Region	AM	MM	PM	RSC	V1	visA
Number of sessions	42	41	35	55	43	70
Number of neurons	42088	44520	32291	51262	39619	63990
Mean trial length	90	93	91	90	92	92

Appendix

Table 7: Dataset distribution on brain region

Hyperparameter	Value
Hidden size	128
Layers	1
Number of heads	16
Latent tokens	8
FFN hidden size	1024
Compression Factor	8

Appendix

Table 8: Hyperparameters used for training POCO model

Hyperparameter	Multi Session	Single session
Embedding Dimension	128	64
Head Dimension	64	64
Number of Latents	32	32
Latent step	0.125	0.1
Depth	4	4
Number of heads	8	8

Appendix

Table 9: Hyperparameters used for training POYO+ model

Hyperparameter	Value
Embedding dimension	512
Codebook size	128
Encoder / Decoder layers	4
Attention heads	4
Discretization window / overlap	4
EMA decay	0.99

Appendix

Table 10: Hyperparameters used for training CalM (NQ) model

Hyperparameter	Value
Model dimension	512
Layers	6
Attention heads	8
FFN dimension	2048
Scheduled sampling probability	0.6
Scheduled sampling block length	6

Appendix

Table 11: Hyperparameters used for training CalM (DAT) model

Alpha	NQ Recon	DAT Forecast	POCO Forecast	Mean decoding $R^{2}$
0.15 (default)	0.9561 $\pm$ 0.0114	0.4600 $\pm$ 0.0767	0.3328 $\pm$ 0.0706	0.7499 $\pm$ 0.0138
0.60	0.8984 $\pm$ 0.0100	0.3603 $\pm$ 0.0727	0.2470 $\pm$ 0.0501	0.7746 $\pm$ 0.0192
1.00 (w/o smoothing)	0.8784 $\pm$ 0.0213	0.3003 $\pm$ 0.0769	0.2020 $\pm$ 0.0450	0.7796 $\pm$ 0.0258

Appendix

Table 12: Ablation results for different alpha values

Metric	baseline	w/o AR loss	w/o gumbel	w/o orthogonal loss
AR corr	0.3424 $\pm$ 0.0692	0.2968 $\pm$ 0.1098	0.3148 $\pm$ 0.0826	0.3124 $\pm$ 0.1194

Appendix

Table 13: Ablation results for NQ

Metric	baseline	w/o scheduled sampling	w/o neighborhood replacement
AR corr	0.3058 $\pm$ 0.1098	0.2079 $\pm$ 0.1027	0.2870 $\pm$ 0.1160

Appendix

Table 14: Ablation results for DAT

Metric (Corr)	Held-in	Held-out
Train	0.9479 $\pm$ 0.0142	0.9486 $\pm$ 0.0147
Eval	0.9473 $\pm$ 0.0143	0.9479 $\pm$ 0.0149
Test	0.9473 $\pm$ 0.0142	0.9482 $\pm$ 0.0147

Appendix

Table 15: Reconstruction result with

\sigma

Metric (Corr)	Real	Simulation
iTransformer	0.0822 $\pm$ 0.0383	0.1941 $\pm$ 0.0996
PatchTST	0.1918 $\pm$ 0.0526	0.2563 $\pm$ 0.1501
TCN	0.1854 $\pm$ 0.0613	0.4077 $\pm$ 0.1564
POCO	0.2901 $\pm$ 0.0783	0.4688 $\pm$ 0.1365
Ours	0.4025 $\pm$ 0.0858	0.5207 $\pm$ 0.0692

Appendix

Table 16: Single-session forecasting results with

\sigma

Metric ( $R^{2}$ )	Pitch	Roll	Yaw
GLM	0.6248 $\pm$ 0.0864	0.5779 $\pm$ 0.1190	0.5764 $\pm$ 0.0650
RRR	0.6220 $\pm$ 0.0912	0.5787 $\pm$ 0.1195	0.5764 $\pm$ 0.0649
TCN	0.6217 $\pm$ 0.0948	0.6007 $\pm$ 0.1145	0.6076 $\pm$ 0.0762
POYO+	0.6077 $\pm$ 0.1664	0.6066 $\pm$ 0.0719	0.6711 $\pm$ 0.0725
Ours-Linear	0.7017 $\pm$ 0.1017	0.6595 $\pm$ 0.0875	0.5745 $\pm$ 0.1391
Ours-Nonlinear	0.7715 $\pm$ 0.0687	0.7314 $\pm$ 0.1003	0.6493 $\pm$ 0.0755

Appendix

Table 17: Single-session decoding results with

\sigma

Metric (Corr)	Held-in	Held-out
POCO	0.3668 $\pm$ 0.0605	0.3322 $\pm$ 0.0511
Ours	0.3799 $\pm$ 0.0978	0.3383 $\pm$ 0.0860
diff	+0.0131	+0.0061
p-value	$7.82\times 10^{-4}$	0.3684
significance		n.s.

Appendix

Table 18: Statistical test for multi-session forecasting

Metric ( $R^{2}$ )	Held-in pitch	Held-in roll	Held-in yaw	Held-out pitch	Held-out roll	Held-out yaw
POYO+	0.7227 $\pm$ 0.0840	0.6751 $\pm$ 0.1132	0.5853 $\pm$ 0.2321	0.7073 $\pm$ 0.0611	0.6446 $\pm$ 0.1299	0.5430 $\pm$ 0.2240
Ours	0.7873 $\pm$ 0.0737	0.7540 $\pm$ 0.0883	0.7076 $\pm$ 0.1691	0.7309 $\pm$ 0.0785	0.6980 $\pm$ 0.1200	0.6022 $\pm$ 0.2172
diff	+0.0647	+0.0790	+0.1223	+0.0236	+0.0534	+0.0592
p-value	$2.09\times 10^{-32}$	$1.47\times 10^{-32}$	$7.37\times 10^{-30}$	$2.59\times 10^{-6}$	$1.84\times 10^{-10}$	$2.14\times 10^{-7}$
significance

Appendix

Table 19: Statistical test for multi-session decoding

Figure 1: Confusion matrices for classification using CalM session embedding. (A) Mouse subject classification. (B) Brain region classification.

Figure 2: Shuffle analysis of the LDA structure shown in Figure . (A) Shuffle for LDA structure of cue-encoding for all held-in neural embeddings. (B) Shuffle for LDA structure of choice-encoding for all held-in neural embeddings.

Embeddings	Subject	Brain Region	Layer
Neural	0.224 $\pm$ 0.001	0.184 $\pm$ 0.001	0.621 $\pm$ 0.002
Session	0.930 $\pm$ 0.025	0.755 $\pm$ 0.025	1.000 $\pm$ 0.000

Appendix

Table 20: Classification accuracies for neural and session embedding.

Figure 3: Linear analysis of held-out dataset for CalM framework. (A) PCA visualization for strong cue- and choice- tuning neurons. (B) LDA analysis shows similar orthogonal gradient structure. (C) Shuffle analysis for LDA.

Self-Supervised Pre-Training

Self-supervised pre-training learns transferable representations from unlabeled data to reduce the labels needed for downstream tasks.

See all 250 papers

CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics
Jul 25, 2026Xinhong Xu, Yimeng Zhang, Yuanlong ZhangTransformer Neural Decoding
Department of Automation, Tsinghua University, Beijing, China · School of Life Sciences, Tsinghua University, Beijing, China
Cross-Subject Modeling for Widefield Calcium Imaging via Atlas-Aligned Spatiotemporal Tokenization
Jul 5, 2026Mohammad Hosseini, Eray Erturk, Saba Hashemi +1Neural Decoding
Department of Electrical and Computer Engineering, USC Viterbi School of Engineering, University of Southern California, Los Angeles, CA, USA · Department of Computer Science, University of Southern California, Los Angeles, CA, USA · Alfred E. Mann Department of Biomedical Engineering, University of Southern California, Los Angeles, CA, USA
SpikeSSL: A Universal Spike Inference Framework with Dynamics-Informed State-Space Layers
Oct 8, 2026Chenghao Yue, Siming Xing, Shuran Liu +2Neural Decoding State Space Models
School of Life Sciences, Tsinghua University, Beijing, China · School of Future Information Innovation, Fudan University, Shanghai, China

Jul 25, 2026J/K move · Enter open · S save

CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

Xinhong Xu, Yimeng Zhang, Yuanlong Zhang

Department of Automation, Tsinghua University, Beijing, China · School of Life Sciences, Tsinghua University, Beijing, China

Transformer Neural Decoding Autoregressive Transformer Transfer Learning Time-Series Foundation Models

Open paper AskPDF arXiv

Calcium imaging has enabled large-scale recordings of neural population activity across diverse animals, laboratories, and experimental paradigms. However, these datasets remain highly fragmented: Existing approaches are often designed for individual datasets or tasks, making it unclear whether their learned representations are reusable for new calcium imaging trace datasets. To tackle this gap, we present CAPT, a Continuous Autoregressive Population Transformer for calcium population dynamics. CAPT models continuous calcium traces directly through a continuous patch tokenization strategy and is trained autoregressively with a mean squared error objective, enabling end-to-end training and adaptation to diverse downstream tasks. We first pretrain CAPT on a large-scale mouse calcium imaging dataset and evaluate its transferability across independent mouse, larval zebrafish, and C. elegans datasets collected by different laboratories. In these transfer settings, the pretrained backbone is frozen and only adaptation modules, i.e., neuron and session embeddings or task-specific heads, are updated. Across neural population forecasting and behavior decoding tasks, CAPT consistently outperforms specialized and general-purpose baselines, demonstrating that a single continuous autoregressive backbone can learn representations that transfer beyond the original pretraining distribution. Beyond predictive performance, multimodal analyses using NeuroPAL annotations in C. elegans datasets show that CAPT embeddings form a shared functional space across datasets and capture cell-identity-related structure. Together, these results suggest that continuous autoregressive modeling provides an effective framework for learning reusable calcium population representations that transfer across datasets, experimental paradigms, and species.

Abstract

Self-Supervised Pre-Training

Explore similar work

CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

Cross-Subject Modeling for Widefield Calcium Imaging via Atlas-Aligned Spatiotemporal Tokenization

SpikeSSL: A Universal Spike Inference Framework with Dynamics-Informed State-Space Layers