Behavior-dLDS: A decomposed linear dynamical systems model for neural activity partially constrained by behavior

Eva Yezerets; En Yang; Misha B. Ahrens; Adam S. Charles

q-bio.NCMar 5, 2026

Behavior-dLDS: A decomposed linear dynamical systems model for neural activity partially constrained by behavior

Authors: Eva Yezerets, En Yang, Misha B. Ahrens, Adam S. Charles

Organizations: Department of Biomedical Engineering Center for Imaging Science Johns Hopkins University · Department of Biology University of North Carolina · Janelia Research Campus Howard Hughes Medical Institute

Abstract

Brain-wide recordings of large-scale networks of neurons now provide an unprecedented view into how the brain drives behavior. However, brain activity contains both information directly related to behavior as well as the potential for many internal computations. Moreover, observable behavior is executed not only by the brain, but also by the spinal cord and peripheral nervous system. Behavior is a coarse-grained product of neural activity, and we thus take the view that it can be best represented by lower-dimensional latent neural dynamics. Capturing this indirect relationship while disambiguating behavior-generating networks from internal computations running in parallel requires new modeling approaches that can embody the parallel and distributed nature of large-scale neural populations. We thus present behavior-decomposed linear dynamical systems (b-dLDS) to disentangle simultaneously recorded subsystems and identify how the latent neural subsystems relate to behavior. We demonstrate the ability of b-dLDS to decouple behavioral vs. internal computations on controlled, simulated data, showing improvements over a state-of-the-art model that uses behavior to supervise all dynamics based on behavior. We also demonstrate b-dLDS's interpretability benefits on a task-driven RNN dataset featuring a nonlinear relationship between behavior and activations. We then show that b-dLDS can further scale up to tens of thousands of neurons by applying our model to a large-scale recording of a zebrafish hindbrain during the complex positional homeostasis behavior, wherein b-dLDS highlights asymmetry in behavior-related dynamic connectivity networks.

Figures & tables

Figure 1: behavior-dLDS architecture. A: The latent, lower-dimensional neural manifold represents the processes that generate observable neural activity, mapped to the ambient space via $\bm{D}$ , and behavior, via $\bm{\Psi}$ . B: The established dLDS model, inside the dashed lines, is updated here with a mapping from dynamics coefficients $\bm{c}$ to behavior $\bm{b}$ via $\bm{\Psi}$ . This contributes another regularization term to the joint inference of latent states $\bm{x}$ and dynamics coefficients $\bm{c}$ .

Figure 2: Simulated system composed of two independent sub-systems with 10 behavior traces. Example trial (of 50 randomly sampled trials); $R^{2}$ calculated over all 50 samples. A,B: “Neural” ground truth and reconstruction by b-dLDS. Signal is generated from a set of 6 ground truth dynamics operators and corresponding dynamics coefficients, with $\bm{D}=\bm{I}$ . C,D: Simulated behavior and reconstruction from b-dLDS, where behavior is simulated from the ground truth dynamics coefficients. E: True dynamics coefficients used to generate the data. F: Learned dynamics coefficients (absolute value). G: Learned dynamics operators (1-15). H: Learned $\bm{\Psi}$ , where each column corresponds to a dynamics operator and dynamics coefficient. Only column 3 is nonzero, and it perfectly reconstructs column 1 of the ground truth $\bm{\Psi}$ . This is the correct column to reconstruct because the first ground truth dynamics coefficient was used to generate the simulated behavior traces.

Figure 3: b-dLDS vs. CLDS and PSID. CLDS learns only behavior-conditioned dynamics, while b-dLDS learns both behavior-generating and intrinsic ones. PSID learns only a shared latent neural-behavioral space, while b-dLDS learns a hierarchical relationship between latent neural states, latent dynamics, and behavior. (Note: CLDS requires an 80/20 train/test split for optimization, so all models (CLDS, b-dLDS, and PSID) were only trained/learned on 40 out of 50 simulated trials (where CLDS then saw the remaining 10 trials for testing). Results shown here are on the same 40 “train” trials across models.) A-G: b-dLDS, 2 independent systems simulation, 1 trial out of 40 shown, $\bm{D}=\bm{I}$ , with simulated behavior generated as a linear combination of ground truth dynamics coefficients. $R^{2}$ calculated across 40 trials. A, B: 1 trace out of 8 shown, data and reconstruction. E: True dynamics coefficients $\bm{c}$ . Stars match selected time points in H . Note that at the first two time points, one top-left (purple) dynamics operator and one bottom-right (yellow) dynamics operator are active, while at the third time point, only a top-left dynamics operator is active. D: b-dLDS-inferred $\bm{c}$ . The b-dLDS model is initialized with more than enough dynamics operators (15 learned vs. 6 ground truth operators). Noisy dynamics operators in F correspond to unused (black) dynamics coefficients in D . E: Behavior and reconstruction (1 trace). G: $\bm{\Psi}$ is a row vector with one dynamics coefficient mapping to one behavior (true value: 1, learned value: 0.8765, other true values 0, learned near 0). H: CLDS learns an 8x8 dynamics matrix $\widehat{\bm{A}}_{t}$ , while b-dLDS learns $\widehat{\bm{f}}$ and $\widehat{\bm{c}}$ , which can be combined via $\sum_{m=1}^{M}\widehat{\bm{f}}_{m}\widehat{\bm{c}}_{mt}$ to create an equivalent matrix. We show 3 example time points here as described above. b-dLDS learns the block-diagonal dynamics matrices, while CLDS does not. The full comparison across time is shown in Figure 0.4 . PSID learns a dynamics matrix $\widehat{\bm{A}}$ that describes the relationship between latent states. I: CLDS neural data reconstruction (1st trace). J: PSID neural data reconstruction (1st trace). K: b-dLDS achieves lower relative mean squared error than CLDS on dynamics reconstruction, regardless of the number of dynamics or the number of behaviors. CLDS used more than 1 TB of RAM when we tried to run it on 10 behavior traces.

Model	Iterations	$\lambda_{\Psi}$	$L_{1}$ / $L_{2}$ norm	$R^{2}_{b}$
b-dLDS	500	5	2.6	0.99
		50	0.66	0.99
		100	0.39	0.99
		200	0.07	0.96
b-dLDS-x	500	0.9644	3.88	0.95
		5	4.44	0.96

Table 1: b-dLDS vs. b-dLDS-x

\bm{\Psi}

sparsity and behavior reconstruction across levels of regularization on the Frobenius norm of

\bm{\Psi}

. For b-dLDS, we start with 5 as this is the setting used in the model shown in Figure 4 B. For b-dLDS-x, we start with 0.9644 as this is the BayesOpt-identified setting used in the model shown, and then compare to other stronger regularization settings.

Figure 4: b-dLDS vs. b-dLDS-x on delayanti task-driven RNN data . One seed of each model shown. $\bm{D}=\bm{I}$ . For behavior data reconstruction, b-dLDS $R^{2}=0.99$ , while b-dLDS-x $R^{2}=0.95$ . The b-dLDS-x behavior reconstruction is constrained to be a linear combination the latent states, while b-dLDS learns a sparser, more interpretable set of dynamics coefficients based on the behavior. A: TaskRNN model activations, “behavior” traces (23 total inputs and outputs), and latent states (equivalent to the activations because there is no noise in the observations; the activations are known directly). B: b-dLDS $\widehat{\bm{c}}$ and $\widehat{\bm{b}}$ example. C: Corresponding b-dLDS $\widehat{\bm{\Psi}}$ . D: b-dLDS-x $\widehat{\bm{c}}$ and $\widehat{\bm{b}}$ example. E: Corresponding b-dLDS-x $\widehat{\bm{\Psi}}$ . F: Comparison of $\widehat{\bm{\Psi}}$ across regularization strengths on the Frobenius norm for b-dLDS vs. b-dLDS-x.

Figure 5: Four strongest $\bm{\Psi}$ connections between dynamics coefficients and motor activity in a zebrafish model with one behavior trace. b-dLDS $R^{2}=0.91$ for neural data reconstruction (over whole time series concatenated; per neuron varies); $R^{2}=0.97$ for behavior data reconstruction. Dynamic connectivity maps for dynamics operators and corresponding dynamics coefficients shown. Motor-aligned dynamics are utilized throughout the recording or a mix of earlier and/or later in the recording.

Criteria	“Left” side	“Right” side	% asymmetry
# neuron centroids	2550	2400	6%
Variance of dF/F over time per trace	mean 0.0099, median 0.0052, std 0.0153	mean 0.0109, median 0.0057, std 0.0174	9% (means)
$R^{2}$ per neuron	mean 0.38, median 0.35, std 0.23	mean 0.46, median 0.43, std 0.27	17% (means)
# start/end points for largest connections			6 of 25 DOs: 18%-118%

Table 2: b-dLDS connectivity asymmetry in zebrafish hindbrain dynamics. To form a baseline of asymmetry that might stem from other sources (e.g., imaging artifacts) we compute the asymmetry in the geometry and variance of the fluorescence data itself. These baselines indicate that up to 17% asymmetry might be attributed to other sources. Of the 25 DOs, 6 displayed stronger asymmetry, with more start and/or end points of their connectivity on the left side than on the right.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

List of Supplementary Figures 0.1 Two-independent-systems simulation with 10 behaviors generated from the first two ground truth states, with some overlap in their combined effects. A: Ground truth signal fed into the model. B: Signal reconstruction from b-dLDS. C: True dynamics coefficients used to generate the data. D: Learned dynamics coefficients (absolute value). E: Learned dynamics operators (1-15). F: Learned $\bm{\Psi}$ . G: Best reconstruction of the true $\bm{\Psi}$ (2 columns, other columns all 0). H: Highlighted dynamics coefficient used to simulate behavior data. I: Highlighted dynamics coefficient corresponding to best match of learned $\bm{\Psi}$ . J: Highlighted corresponding dynamics operator. K: Simulated behavior ground truth. L: Behavior reconstruction.

List of Supplementary Figures 0.2 Two-independent-systems simulation with 10 behaviors generated from a combination of all 6 ground truth states. A: Ground truth signal fed into the model. B: Signal reconstruction from b-dLDS. C: True dynamics coefficients used to generate the data - one was randomly initialized as all zeros. D: Learned dynamics coefficients (absolute value). E: Learned dynamics operators (1-15). F: True $\bm{\Psi}$ . G: Best reconstruction of the true $\bm{\Psi}$ (6 columns). H: True behavior. I: Behavior reconstruction.

List of Supplementary Figures 0.3 Two-independent-systems simulation with 10 behaviors generated from the first ground truth state, with added Gaussian noise in the “neural” data. A: Ground truth signal fed into the model, with Gaussian noise scaled by $\sqrt{0.1}$ . B: Signal reconstruction from b-dLDS. C: True dynamics coefficients used to generate the data - two were randomly initialized as all zeros. D: Learned dynamics coefficients (absolute value). E: Learned dynamics operators (1-15). F: Learned $\bm{\Psi}$ . G: Best reconstruction of the true $\bm{\Psi}$ (1 column, other columns all 0). H: Highlighted dynamics coefficient used to simulate behavior data. I: Highlighted dynamics coefficient corresponding to best match of learned $\bm{\Psi}$ . J: Highlighted corresponding dynamics operator. K: Simulated behavior ground truth. L: Behavior reconstruction.

List of Supplementary Figures 0.4 CLDS vs. b-dLDS learned dynamics, continued. CLDS learns an 8x8 dynamics matrix $\bm{A}_{t}$ , while b-dLDS learns $\bm{f}$ and $\bm{c}$ , which can be combined via $\sum_{m=1}^{M}f_{m}c_{mt}$ to create an equivalent matrix. We compare each block of the matrix (top left 4x4 through bottom right 4x4) to the ground truth dynamics from the simulation. b-dLDS closely matches the ground truth and correctly learns that the two off-diagonal corners are 0, indicating 2 independent systems; CLDS instead incorrectly follows the behavior in all dynamics. Each row of each heatmap is an entry of the 4x4 block of the overall 8x8 matrix (A12 means row 1 column 2, 16 rows per heatmap).

Model	Seed	$R^{2}_{y}$	$R^{2}_{b}$	Max	Idx	Max	Idx	Max	Idx	Time (s)	RAM (KB)
				$\widehat{\bm{\Psi}}$	max	$R^{2}$ ,	max	$R^{2}$	max
				value	$\widehat{\bm{\Psi}}$	$\widehat{\bm{c}}$ vs.	$\widehat{\bm{c}}$	$\widehat{\bm{x}}$ vs.	$\widehat{\bm{x}}$
						$\bm{b}$		$\bm{b}$
b-dLDS	1	0.91	0.96	0.68	25	0.82	25	0.36	3	6.11E+04	-2.17E+08
	2	0.9	0.95	0.29	20	0.84	25	0.36	43	6.09E+04	-2.18E+08
	3	0.91	0.97	0.74	3	0.84	3	0.33	21	4.74E+04	7.91E+07

Appendix

Table 3: b-dLDS, dLDS, and b-dLDS-x model comparison, with runtimes and memory utilization. b-dLDS-x represents an alternative to b-dLDS where the behavior maps to the latent states rather than to the dynamics coefficients. With additional optimization, b-dLDS-x can match b-dLDS in overall behavior reconstruction. However, b-dLDS provides better interpretability: its max

\bm{\Psi}

values are larger, and so are its max

R^{2}

values between the dynamics coefficients and behavior, compared to the max

R^{2}

values between the dynamics coefficients and behavior for b-dLDS-x. In the two versions of b-dLDS shown here, two times out of three, these two indices (max

\bm{\Psi}

and max

\bm{c}

vs.

\bm{b}

R^{2}

also corresponded; in contrast, they only corresponded in one version of b-dLDS-x,“b-dLDS-x with reg on

\bm{\Psi}

”, which also had worse behavior reconstruction (see “Max

\bm{\Psi}

value” and “Idx max

\bm{\Psi}

” vs. “Max

R^{2}

\bm{c}

vs.

\bm{b}

” and “Idx max

\bm{c}

” for b-dDLS models, and “Max

\bm{\Psi}

value” and “Idx max

\bm{\Psi}

” vs. “Max

R^{2}

\bm{x}

vs.

\bm{b}

” and “Idx max

\bm{x}

” for b-dLDS-x models). Please note that runtime and memory utilization are only meant to be a rough estimate, as these were dependent on other processes happening in parallel on the server, and there is no built-in function for calculating memory utilization for MATLAB on Linux machines; instead we rely on subtracting start from end memory used, sometimes resulting in negative values ( Dvorak, 2026 ) .

Parameter	Meaning	How to set	Settings used for zebrafish models (in order of appearance in Table 3 )

nF	number of dynamics operators (DOs)	Based on models used in zebrafish figures. Also tried matching nF to N for fairness of $\widehat{\bm{x}}$ and $\widehat{\bm{c}}$ vs. $\bm{b}$ $R^{2}$ comparisons.	25 (first 7 model types); 50 (last 3 model types, “nF=50”)
N	number of latent dimensions	Based on models used in zebrafish figures.	50
lambda_val	regularization: sparsity of states ( $\widehat{\bm{x}}$ )	Based on models used in zebrafish figures.	0.05
lambda_history	regularization: smoothness of states ( $\widehat{\bm{x}}$ ), but more specifically, reconstruction of states from dynamics operators and dynamics coefficients	Based on models used in zebrafish figures (with exceptions for b-dLDS-x BayesOpt).	0.23; 0.1154 (b-dLDS-x BayesOpt model types)
lambda_b	regularization: sparsity of dynamics coefficients ( $\widehat{\bm{c}}$ )	Based on models used in zebrafish figures.	0.05

Appendix

Table 4: Meaning of key b-dLDS parameters, heuristics for setting them, and settings used for model comparisons in Table 3 : one motor signal input as behavior to b-dLDS and b-dLDS-x.

List of Supplementary Figures 0.5 Ablation of regularization on $\widehat{\bm{\Psi}}$ update step. b-dLDS with these settings optimized (same simulation setup as shown in Figure 3 K) achieves lower relative mean squared error on dynamics reconstruction than ablated models, regardless of the number of behavior traces (1 or 10) or number of dynamics tied to the behavior traces (behavior traces are generated as a linear combination of the simulated dynamics traces).

Parameter	Meaning	How to set	Settings used for 4 simulation types

nF	number of dynamics operators (DOs)	First set to 2x the number of PCs required for 95% variance explained. Then decrease until the median number of active dynamics coefficients starts decreasing. (Goal: minimize the number of DOs while maintaining the inherent dimensionality of the data.) Alternative heuristic: use the number of known behavioral states as a benchmark and double it to get the starting point for the number of DOs, and then start decreasing from there.	15
N	number of latent dimensions	Known based on data, given that $\bm{D}=\bm{I}$ .	8
lambda_val	regularization: sparsity of states $\widehat{\bm{x}}$	Increase to reduce the number of states active at the same time (moderately sensitive)	1.0000e-04
lambda_history	regularization: smoothness of states $\widehat{\bm{x}}$ , but more specifically, reconstruction of states from dynamics operators and dynamics coefficients	Increase to encourage states to change smoothly from time point to time point (less sensitive)	1.0000e-04
lambda_b	regularization: sparsity of dynamics coefficients (c)	Increase to reduce the number of DOs active at the same time (sensitive)	0.25

Appendix

Table 5: Meaning of key b-dLDS parameters, heuristics for setting them, and settings used for 4 simulation model types: one dynamics coefficient maps to all 10 behaviors; 2 dynamics coefficients map to all 10 behaviors; all dynamics coefficients map to all 10 behaviors; and one dynamics coefficient maps to all 10 behaviors, with added Gaussian noise in the simulated data.

Parameter	Meaning	How to set	Settings used (if differ, first b-dLDS setting shown; then b-dLDS-x)

nF	number of dynamics operators (DOs)	Set to 2x the number of PCs required for 95% variance explained.	16
N	number of latent dimensions	Known based on data, given that $\bm{D}=\bm{I}$ .	128
lambda_val	regularization: sparsity of states ( $\widehat{\bm{x}}$ )	Optimize with BayesOpt.	0.9772;0.0293
lambda_history	regularization: smoothness of states ( $\widehat{\bm{x}}$ ), but more specifically, reconstruction of states from dynamics operators and dynamics coefficients	Optimize with BayesOpt.	0.8702;0.0636
lambda_b	regularization: sparsity of dynamics coefficients ( $\widehat{\bm{c}}$ )	Optimize with BayesOpt.	0.25908;0.0315

Appendix

Table 6: Meaning of key b-dLDS and b-dLDS-x parameters, heuristics for setting them, and settings used for the TaskRNN experiments.

Parameter	Meaning	How to set	Settings used for zebrafish models (1 vs. 4 behavior inputs)

nF	number of dynamics operators (DOs)	First set to 2x the number of PCs required for 95% variance explained. Then decrease until the median number of active dynamics coefficients starts decreasing. (Goal: minimize the number of DOs while maintaining the inherent dimensionality of the data.) Alternative heuristic: use the number of known behavioral states as a benchmark and double it to get the starting point for the number of DOs, and then start decreasing from there.	25
N	number of latent dimensions	Set to at least the number of PCs required for 80-90% variance explained.	50
lambda_val	regularization: sparsity of states $\widehat{\bm{x}}$	Increase to reduce the number of states active at the same time (moderately sensitive)	0.05
lambda_history	regularization: smoothness of states ( $\widehat{\bm{x}}$ ), but more specifically, reconstruction of states from dynamics operators and dynamics coefficients	Increase to encourage states to change smoothly from time point to time point (less sensitive)	0.23
lambda_b	regularization: sparsity of dynamics coefficients $\widehat{\bm{c}}$	Increase to reduce the number of DOs active at the same time (sensitive)	0.05

Appendix

Table 7: Meaning of key b-dLDS parameters, heuristics for setting them, and settings used for two zebrafish model types in Figure 5 and Supplementary Figures 0.6 , 0.7 , 0.8 , 0.9 , and 0.10 : one motor signal input as behavior, or one motor signal plus three trial types as one-hot encodings.

List of Supplementary Figures 0.6 Four weakest $\bm{\Psi}$ connections between dynamics coefficients and motor activity in a model with one behavior. b-dLDS $R^{2}=0.91$ for neural data reconstruction (over whole time series concatenated; per neuron varies); $R^{2}=0.97$ for reconstruction of smoothed behavior data input to the model. Dynamic connectivity maps and dynamics coefficients shown. The weakest relationships to behavior are those dynamics operators that are rarely utilized throughout the recording.

List of Supplementary Figures 0.7 Four strongest $\bm{\Psi}$ connections between dynamics coefficients and motor activity in a model with four behaviors (motor and 3 trial types based on stimulation patterns, i.e., fish offset by experimenter). Overall model neural data reconstruction $R^{2}=0.93$ ; reconstruction of smoothed behavior data input to the model $R^{2}=0.997$ (all 4 behavior traces concatenated). Dynamic connectivity maps and dynamics coefficients shown. Motor-aligned dynamics are utilized throughout the recording or later in the recording.

List of Supplementary Figures 0.8 Four strongest $\bm{\Psi}$ connections between dynamics coefficients and swim trial type 1 in a model with four behaviors (motor and 3 trial types based on stimulation patterns between swims). Overall model neural data reconstruction $R^{2}=0.93$ ; reconstruction of smoothed behavior data input to the model $R^{2}=0.997$ (all 4 behavior traces concatenated). Dynamic connectivity maps and dynamics coefficients shown. Swim trial-aligned dynamics cover different time segments of the recording.

List of Supplementary Figures 0.9 Four strongest $\bm{\Psi}$ connections between dynamics coefficients and swim trial type 2 in a model with four behaviors (motor and 3 trial types based on stimulation patterns between swims). Overall model neural data reconstruction $R^{2}=0.93$ ; reconstruction of smoothed behavior data input to the model $R^{2}=0.997$ (all 4 behavior traces concatenated). Dynamic connectivity maps and dynamics coefficients shown. Swim trial-aligned dynamics cover different time segments of the recording.

List of Supplementary Figures 0.10 Four strongest $\bm{\Psi}$ connections between dynamics coefficients and swim trial type 3 in a model with four behaviors (motor and 3 trial types based on stimulation patterns between swims). Overall model neural data reconstruction $R^{2}=0.93$ ; reconstruction of smoothed behavior data input to the model $R^{2}=0.997$ (all 4 behavior traces concatenated). Dynamic connectivity maps and dynamics coefficients shown. Swim trial-aligned dynamics cover different time segments of the recording.

List of Supplementary Figures 0.11 Hyperparameter sensitivity experiments related to number of dynamics coefficients and model size. Data from Table 8 . 3 seeds shown for each point. A-D: Zebrafish model parameter sweeps: N, nF, regularization for sparsity of $\widehat{\bm{c}}$ , regularization for $\bm{b}$ reconstruction. E,F: Simulation (Fig. 3 ) parameter sweeps: regularization for sparsity of $\widehat{\bm{c}}$ , regularization for $\bm{b}$ reconstruction.

Model	Param.	Val.	Median $\widehat{\bm{c}}$ use	Mean	$R_{b}^{2}$	Mean	$R_{y}^{2}$	Mean
Zebrafish	N	25	7	6.67	0.98	0.97	0.73	0.72
hindbrain		25	6		0.98		0.70
		25	7		0.95		0.72
		35	6	6.00	0.97	0.97	0.78	0.78
		35	6		0.99		0.78
		35	6		0.97		0.79

Appendix

Table 8: Hyperparameter sensitivity experiments. * indicates settings used in paper figures. Values to test were chosen based on values used in figures, plus 0.01, 0.1, and 1, at least. The results shown are not necessarily converged to their maximum potential

R^{2}

, for example, because only 500 iterations were run for each of these tests (rather than the 5000 used in zebrafish figures).

Latent Dynamics Modeling

Latent dynamics modeling learns compact hidden-state representations and transition rules to predict, explain, and control evolving systems.

See all 253 papers

Machine Learning Methods for Studying Latent Neural Activity Dynamics
Jun 9, 2026Shufeng Kong, Fumei Deng, Xinyi Dong +7Dynamical Systems Latent Variable Models
School of Software Engineering, Sun Yat-sen University, Zhuhai, China · School of Computer Science and Artificial Intelligence, Foshan University, Foshan, China · Department of Ecology and Evolutionary Biology, Cornell University, Ithaca, NY, USA +2
Inferring Multi-Timescale Neural Dynamics with Switching Linear Dynamical Systems
Oct 1, 2026Lulu Gong, Yongxu Zhang, Shreya SaxenaStochastic Linear Dynamical Systems Latent Dynamics Modeling
Center for Neurocomputation and Machine Intelligence, Wu Tsai Institute, Department of Biomedical Engineering, Yale University, New Haven, CT 06510, USA
Integrative neurocybernetic modeling in the era of large-scale neuroscience
Apr 26, 2026Il Memming Park, Gonzalo G. de Polavieja, Ayesha Vermani +12Computational Cognitive Modeling Latent Dynamics Modeling
Champalimaud Centre for the Unknown, Champalimaud Foundation, Lisbon, Portugal · RyvivyR Inc, NY, USA · Harvard University, Cambridge, MA, USA +3

Jun 9, 2026J/K move · Enter open · S save

Machine Learning Methods for Studying Latent Neural Activity Dynamics

Shufeng Kong, Fumei Deng, Xinyi Dong, Caihua Liu+6

School of Software Engineering, Sun Yat-sen University, Zhuhai, China · School of Computer Science and Artificial Intelligence, Foshan University, Foshan, China · Department of Ecology and Evolutionary Biology, Cornell University, Ithaca, NY, USA+2

Dynamical Systems Latent Variable Models Latent Dynamics Modeling Neural Processes Neural Decoding

Open paper AskPDF arXiv

Recent developments in brain recording are driving a demand for machine learning tools capable of decoding the latent structure of large populations of neurons. In this paper, we provide a comprehensive survey that outlines the trajectory of Latent Variable Models (LVMs) from early state-space models to more recent deep generative models. We organize the literature into three closely related domains: (1) Single-Region Latent Dynamics, which includes models such as linear dynamical systems to more complex dynamics represented by Recurrent Neural Networks (RNNs) and Neural Ordinary Differential Equations (ODEs); (2) Multi-Region Communication, which employs probabilistic as well as subspace methods to study how information is transferred across different brain areas considering synaptic propagation delays and network connectivity; and (3) Behavior-Aligned Modeling, which seeks to disentangle neural activity related to task performance from other internal states via supervised or contrastive learning. This survey also includes large-scale neural foundation models, such as Transformers and diffusion models, that rely on large-scale pre-training for optimal performance across subjects. Finally, we conclude and discuss benchmarks, evaluation criteria, and open challenges, such as the ability to identify causal links or directionality of communication, to facilitate future research for bridging interpretable brain dynamics with reliable neural decoding.

Abstract

Latent Dynamics Modeling

Explore similar work

Machine Learning Methods for Studying Latent Neural Activity Dynamics

Inferring Multi-Timescale Neural Dynamics with Switching Linear Dynamical Systems

Integrative neurocybernetic modeling in the era of large-scale neuroscience