Scaling Neural Motor Decoding via Decoupled Behavioral Pretraining

Divyansha Lachi; Vinam Arora; Shivashriganesh P. Mahato; Alexandre Andre; Eva L. Dyer

cs.LGNov 5, 2026

Scaling Neural Motor Decoding via Decoupled Behavioral Pretraining

Authors: Divyansha Lachi, Vinam Arora, Shivashriganesh P. Mahato, Alexandre Andre, Eva L. Dyer

Organizations: University of Pennsylvania, Philadelphia, PA, USA

Abstract

Recent neural decoding approaches achieve strong performance by training on large collections of paired neural-behavioral recordings, but such datasets are expensive, invasive, and difficult to scale. In contrast, behavioral data is abundant and easy to collect, raising the question of whether neural decoding can benefit from decoupling behavioral and neural representation learning. We introduce BEEMO, a flexible multi-session framework that enables training with arbitrary mixtures of paired neural-behavioral recordings and unpaired behavioral data. A behavior decoder learns movement dynamics directly from behavioral trajectories, while neural activity is incorporated through cross-attention layers when available, allowing behavioral and neural supervision to scale independently. Across multiple intracortical motor datasets and tasks in nonhuman primates, we show that incorporating unpaired behavioral data improves decoding performance, particularly when transferring to new tasks, and that adding even a small amount of behavioral data from the target task further improves transfer. By decoupling neural and behavioral data, we unlock a new dimension for scaling: unpaired behavioral data, which is substantially cheaper and easier to collect than paired neural recordings, can be used directly to improve decoding performance. We further show that behavior-only pretraining, without any neural pretraining, achieves performance comparable to models pretrained on substantially larger neural datasets. Together, our results suggest that scaling behavioral data offers a practical and cost-effective path toward neural decoding models that generalize across subjects and tasks. Code is available at https://github.com/nerdslab/beemo.

Figures & tables

Figure 1: Overview of BeeMO : (A) Architecture: Behavioral context and query tokens pass through alternating self-attention and cross-attention layers. The behavioral and neural components are decoupled: the behavior decoder operates independently on unpaired behavioral data, and neural latents are incorporated via cross-attention when available. (B) Neural Encoders: POYO (left) represents neural activity as spike-level tokens, where each spike is embedded with its timing and unit identity. NDT* (right) represents neural activity as bin-level tokens, aggregating spikes into fixed time bins across units.

Single-Session
	Within-Task	Cross-Task
Method	CO	RT	Isometric Wrist
Linear	$0.6816\pm 0.0011$	$0.6146\pm 0.0003$	$0.5826\pm 0.0006$
MLP	$0.8995\pm 0.0014$	$0.7477\pm 0.0014$	$0.7392\pm 0.0035$
GRU	$0.9100\pm 0.0015$	$0.7839\pm 0.0016$	$0.7888\pm 0.0011$
POYO-SS	$0.8438\pm 0.0030$	$0.7219\pm 0.0101$	$0.6967\pm 0.0063$

Table 1: Asymmetric training results. All multi-session models are pretrained on center-out (CO) reaching data and evaluated on within-task (CO) and cross-task (RT, Isometric Wrist) transfer. BeeMO variants incorporate

\sim

6.5 hours of additional unpaired behavioral data during pretraining, while base models use paired neural-behavioral data only. Performance is reported as mean

\pm

SEM over three seeds. Paired

\Delta

reports the mean paired difference in

R^{2}

between each BeeMO variant and its matched backbone across held-out sessions. Bold indicates a significant improvement (

p<0.05

, two-sided paired

t

-test). Session-wise mean

\pm

SD and exact

p

-values are reported in Table A7 .

Figure 2: Scaling Analysis: Average cross-task decoding performance (

R^{2}

averaged across RT and IW tasks) as a function of the number of paired neural sessions used during pretraining (models pretrained on CO). Top: BEEMO-POYO versus POYO. Bottom: BEEMO-NDT* versus NDT*. BeeMO outperforms both POYO and NDT* at matched neural data budgets by leveraging unpaired behavioral data during pretraining. Notably, BeeMO matches the performance of POYO and NDT* using approximately 3

\times

less neural data. Shaded regions denote SEM over three seeds.

Figure 3: Behavioral pretraining improves neural decoding. Error bars denote SEM over three seeds. (A) Task-specific behavioral data: Adding task-matched behavioral data (IW) to the pretraining corpus consistently improves downstream decoding performance on the IW task on a new animal for both BeeMO -POYO and BeeMO -NDT*. (B) Behavior-only pretraining: Decoding performance across three tasks (CO, RT, IW) for single-session models and multi-session neural-pretrained models, compared against BeeMO with no neural pretraining (behavior-only).

Figure 4: Behavioral context improves neural decoding. Predicted velocity trajectories (

v_{x}

v_{y}

) under neural-only (blue) and joint neural-behavior (orange) context, vs. ground truth (black), across 1 sec decoding windows sorted by

\Delta R^{2}

(largest to smallest improvement). Additional trajectory examples are provided in Figure A2 in the Appendix.

	Cross-Task
Model Variant	RT	IW
BeeMO -POYO	$0.7609{\scriptscriptstyle\,\pm\,0.0014}$	$0.7361{\scriptscriptstyle\,\pm\,0.0020}$
w/o dropout	$0.6925{\scriptscriptstyle\,\pm\,0.0133}$	$0.6898{\scriptscriptstyle\,\pm\,0.0035}$
w/o unpaired	$0.7476{\scriptscriptstyle\,\pm\,0.0012}$	$0.6849{\scriptscriptstyle\,\pm\,0.0017}$
BeeMO -NDT*	$0.7734{\scriptscriptstyle\,\pm\,0.0018}$	$0.7802{\scriptscriptstyle\,\pm\,0.0001}$
w/o dropout	$0.7101{\scriptscriptstyle\,\pm\,0.0083}$	$0.7266{\scriptscriptstyle\,\pm\,0.0019}$

Table 2: Ablation study: Values are reported as mean

\pm

SEM across three seeds.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Motor tasks and representative behavioral trajectories. Center-out reaching (CO) and random-target reaching (RT) are performed with a planar manipulandum; isometric wrist (IW) uses a force transducer with no arm movement.

Hyperparameter	Search Space	Sampling
Model
Bin size (ms)	$\{1,5,10,20,40\}$	Categorical
Training
Number of epochs	$\{100,300,500\}$	Categorical
Batch size	$[16,128]$	Discrete ( $\log_{2}$ )
Weight decay	$[10^{-6},10^{-1}]$	Loguniform

Appendix

Table A1: Hyperparameter search space for the linear baseline, tuned for 100 trials.

Hyperparameter	Search Space	Sampling
Model
Bin size (ms)	$\{5,10,20,40\}$	Categorical
Depth	$\{1,2,3\}$	Categorical
Hidden dim	$[32,512]$	Discrete ( $\log_{2}$ )
Dropout	$\{0.0,0.2,0.4,0.6\}$	Categorical
Batch norm	$\{\text{True},\text{False}\}$	Categorical

Appendix

Table A2: Hyperparameter search space for the MLP baseline, tuned for 100 trials.

Hyperparameter	Search Space	Sampling
Model
Bin size (ms)	$\{1,5,10,20,40\}$	Categorical
Depth	$\{1,2,3\}$	Categorical
Hidden dim	$[32,512]$	Discrete ( $\log_{2}$ )
Dropout	$\{0.0,0.2,0.4,0.6\}$	Categorical
Bidirectional	$\{\text{True},\text{False}\}$	Categorical

Appendix

Table A3: Hyperparameter search space for the GRU baseline, tuned for 100 trials.

Hyperparameter	Search Space	Sampling
Model
Embedding dim	$\{64,96,128,192,256\}$	Categorical
Number of heads	$\{1,2,4,8\}$	Discrete ( $\log_{2}$ )
Depth	$\{0,1,\ldots,8\}$	Discrete
Dropout	$[0.0,0.9]$	Discrete (step 0.1)
Training

Appendix

Table A4: Hyperparameter search space for the POYO baseline, tuned for 100 trials.

Hyperparameter	Search Space	Sampling
Model
Embedding dim	$\{64,96,128,192,256\}$	Categorical
Number of heads	$\{1,2,4,8\}$	Discrete ( $\log_{2}$ )
Depth	$\{0,1,\ldots,8\}$	Discrete
Dropout	$[0.0,0.9]$	Discrete (step 0.1)
Bin size	$\{1,5,10,20,40\}$ ms	Categorical

Appendix

Table A5: Hyperparameter search space for the NDT* baseline, tuned for 100 trials.

Single-session tuning (per model)
Stage	Runs	GPU-time / run	Total GPU-hours
Linear	2,300	$59.9\pm 102.7$ min (22.3)	$\sim$ 2,300
MLP	2,300	$53.7\pm 122.5$ min (20.8)	$\sim$ 2,100
GRU	2,300	$46.8\pm 80.9$ min (16.9)	$\sim$ 1,800
POYO-SS	2,300	$55.8\pm 142.4$ min (17.0)	$\sim$ 2,100
NDT*-SS	2,300	$23.6\pm 19.2$ min (19.0)	$\sim$ 900

Appendix

Table A6: Estimated computational cost. For single-session baselines, run times are reported as mean

\pm

SD across tuning runs, with the median in parentheses, and total GPU-hours are estimated over all 2,300 runs. For pretrained models, we report the one-time pretraining cost and the cost of fine-tuning across the 23 evaluation sessions.

Figure A2: Behavioral context improves neural decoding. Predicted velocity trajectories ( $v_{x}$ , $v_{y}$ ) under neural-only (blue) and joint neural-behavior (orange) context, vs. ground truth (black), across 1 sec decoding windows sorted by $\Delta R^{2}$ (largest to smallest improvement). The gray trace shows the preceding behavioral context provided to the joint model. $R^{2}$ values are reported as $R^{2}_{\text{joint}}-R^{2}_{\text{neural}}=\Delta R^{2}$ . Behavioral context consistently produces smoother, better-calibrated trajectories, with the largest gains occurring when neural-only decoding fails to constrain trajectory amplitude and timing

Backbone	Task	BeeMO	Baseline	Paired $\boldsymbol{\Delta R^{2}}$	Paired $t$ -test ( $\boldsymbol{p}$ )
NDT*	CO	$0.9027\pm 0.0138$	$0.9010\pm 0.0175$	$+0.0017$	$0.4240$
	RT	$0.7734\pm 0.0716$	$0.7600\pm 0.0789$	$\mathbf{+0.0134}$	$\mathbf{0.0123}$
	IW	$0.7802\pm 0.0460$	$0.7767\pm 0.0435$	$+0.0035$	$0.1920$
POYO	CO	$0.9031\pm 0.0178$	$0.8973\pm 0.0185$	$+0.0058$	$0.1875$
	RT	$0.7609\pm 0.0726$	$0.7498\pm 0.0742$	$\mathbf{+0.0112}$	$\mathbf{0.0001}$
	IW	$0.7361\pm 0.0644$	$0.7237\pm 0.0680$	$\mathbf{+0.0124}$	$\mathbf{0.0089}$

Appendix

Table A7: Session-wise paired comparisons of BeeMO and matched backbones. Values report mean

R^{2}

\pm

standard deviation across held-out sessions (each session averaged over three seeds). Paired

t

-tests (two-sided) compare each BeeMO variant with its matched multi-session backbone and were conducted separately for each task (

n=6

for CO and RT,

n=11

for IW). Significant results (

p<0.05

) are bolded.

Method	CO	RT
Kalman Filter (SS)	$0.4000\pm 0.0381$	$0.4066\pm 0.0399$
POYO + Kalman Smoother (SS)	$0.8982\pm 0.0062$	$0.7515\pm 0.0018$
POYO + Kalman Smoother (MS)	$0.9004\pm 0.0015$	$0.7508\pm 0.0017$
POYO	$0.8973\pm 0.0013$	$0.7498\pm 0.0015$
BeeMO -POYO	$0.9031\pm 0.0003$	$0.7609\pm 0.0014$

Appendix

Table A8: Kalman-based baselines. Models pretrained on CO and evaluated on CO and RT. Values are reported as mean

\pm

SEM across three seeds.

Unpaired behavioral data	RT
None	0.7511
+ 6.5 h CO	0.7637
+ 6.5 h CO + 2 h RT	0.7705

Appendix

Table A9: Task-specific behavioral data on RT. BeeMO -POYO evaluated on held-out RT sessions with increasing amounts of unpaired behavioral data during pretraining (single seed).

	Within-Task	Cross-Task
Model Variant	CO	RT	IW
BeeMO -POYO	$0.9031\pm 0.0003$	$0.7609\pm 0.0014$	$0.7361\pm 0.0020$
w/o dropout	$0.8179\pm 0.0035$	$0.6925\pm 0.0133$	$0.6898\pm 0.0035$
w/o unpaired	$0.8992\pm 0.0012$	$0.7476\pm 0.0012$	$0.6849\pm 0.0017$
BeeMO -NDT	$0.9027\pm 0.0016$	$0.7734\pm 0.0018$	$0.7802\pm 0.0001$
w/o dropout	$0.8110\pm 0.0062$	$0.7101\pm 0.0083$	$0.7266\pm 0.0019$

Appendix

Table A10: Ablation study: Values are reported as mean

\pm

SEM across three seeds.

Neural Decoding

Neural decoding maps brain activity or error-correction syndromes to stimuli, behavior, language, or corrective actions.

See all 192 papers

NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations
Aug 5, 2026Luyao Jin, Yonghao Song, Huan Zhao +2Representation Learning Brain-Computer Interfaces
Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong · Department of Biomedical Engineering, School of Medicine, Tsinghua University · School of Biomedical Sciences, The Chinese University of Hong Kong
Neural-Behavioral Representation of Natural Whole-body Movement in Monkeys
May 28, 2026Jieshi He, Puzhe Li, Yanan Sui +1Neural Decoding
Center for Excellence in Brain Science and Intelligence Technology, CAS · 2Tsinghua University
Leveraging unlabelled data for generalizable neural population decoding
Jul 15, 2026Ximeng Mao, Nanda H. Krishna, Avery Hee-Woon Ryoo +2Masked Autoencoders Self-Supervised Pre-Training
Mila – Quebec AI Institute · Université de Montréal · Canada CIFAR AI Chair

Aug 5, 2026J/K move · Enter open · S save

NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations

Luyao Jin, Yonghao Song, Huan Zhao, Vincent C. K. Cheung+1

Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong · Department of Biomedical Engineering, School of Medicine, Tsinghua University · School of Biomedical Sciences, The Chinese University of Hong Kong

Representation Learning Brain-Computer Interfaces Neural Decoding Cross-Domain Transfer Learning

Open paper AskPDF arXiv

Decoding continuous motor trajectories from neural activity is essential for developing practical brain-computer interfaces (BCIs). However, current neural decoders are constrained by the limited scale and heterogeneity of neural recordings. In contrast, behavioral data can be collected more readily and at substantially larger scale from humans, animals, simulations, and robotic systems. Here, we introduce NeuroPB, a framework that scales neural decoding by transferring knowledge from pretrained behavioral representations. NeuroPB first pretrains a motor encoder on large-scale motor behavior data and then aligns neural activity with the resulting behavioral representation space using a limited set of paired neural-behavioral recordings. A neural encoder and lightweight motor decoder are subsequently optimized to reconstruct continuous movement from the aligned neural representations. Across multiple macaque motor datasets, behavioral pretraining improves trajectory decoding, including an 11%

R^2

increase on center-out and 8% on random-target compared with training the motor encoder from scratch. Notably, pretraining on robotic trajectories achieves performance comparable to pretraining on macaque trajectories, demonstrating that transferable kinematic structure is shared across biological and artificial models. Moreover, decoding performance improves as the scale and diversity of robotic pretraining data increase, when the amount of neural data is fixed. Pretraining also enhances generalization across recording sessions, subjects, and motor tasks, with only 10% calibration needed to match training from scratch. Overall, these results establish behavioral pretraining as a scalable source for neural decoding and provide a promising route toward high-performance and calibration-efficient BCIs under limited neural data.

Abstract

Neural Decoding

Explore similar work

NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations

Neural-Behavioral Representation of Natural Whole-body Movement in Monkeys

Leveraging unlabelled data for generalizable neural population decoding