cs.LGSep 18, 2026

BrainWideBench: Benchmarking large-scale pretraining and across-animal transfer in multi-region neural recordings

Authors: Alexandre Andre, Shivashriganesh P. Mahato, Vinam Arora, Keshav Balaji, Divyansha Lachi, Nanda H. Krishna, Jingyun Xiao, Yizi Zhang, 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Advances in large-scale neural recording have made it possible to collect data across many animals and distributed brain regions, raising the question of whether this scale can be exploited to learn general-purpose neural representations transferable across diverse downstream tasks. Yet, progress toward this goal has been limited by fragmented evaluation protocols and a narrow focus on individual task domains. Here, we present BrainWideBench, a benchmark for evaluating across-animal transfer on multi-region neural recordings, built on the International Brain Laboratory Brainwide Map dataset of neural and behavioral recordings spanning 276 brain regions from 139 mice performing a sensory-guided decision-making task. The benchmark is organized around three complementary task suites that evaluate whether learned representations support downstream decoding of behavior, can predict masked or future neural activity, and can recover biologically meaningful anatomical organization. With this benchmark, we systematically evaluate pretraining methods across transfer settings, including finetuning on downstream objectives and zero-shot generalization to unseen animals. Our results confirm pretraining improves performance over matched single-session baselines, but we show current methods exhibit heterogeneity in transfer capabilities: gains depend strongly on the alignment between pretraining objectives and downstream tasks. No single approach performs uniformly well across all three suites, and most methods are designed to only address a subset of them. Together, these findings suggest that learning representations that jointly generalize across behavior, dynamics, and anatomy remains an open challenge. By providing a unified and reproducible evaluation suite, BrainWideBench establishes a framework for measuring progress toward general-purpose models of the mouse brain.

Figures & tables

Figure 1: Overview of the benchmark. (Left) During pretraining, models have access to a large pretraining set with 126 mice that includes behavior, spike sorted neural recordings, and anatomical labels (e.g., brain region), to learn generalized representations. Models are pretrained and evaluated (Right) on a held-out set of 13 animals across three task suites: TS1 (decoding), TS2 (neural activity prediction), and TS3 (brain region classification).
Figure 2: Overview of IBL Brainwide Map Dataset. (A) Multi-region probes are inserted into the brain and mapped onto a common coordinate framework (CCF) atlas to map individual neural spike sorted units to different anatomical regions. (B) An example of neural and behavioral data captured simultaneously while the mouse tries to control a wheel to move a visual stimulus from either the left or right side of the screen to the center. We capture paws and wheel speed, whisker motion, and licking from the water spout. After a correct trial, the mouse is rewarded with water. (C) Data from individual probes and sessions are aggregated which in total amounts to over 800,000 neuron hours.
  Frame-level TasksSequence-level TasksAvg
 MethodLicks ( D2D^{2} )Whisker ( R2R^{2} )Wheel ( R2R^{2} )RPaw ( R2R^{2} )LPaw ( R2R^{2} )Reward (Acc)Choice (Acc)Contrast (Acc)Rank
Single-SessionLinear0.151\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.175\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.219\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.091\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.084\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.813\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.614\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.219\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}7.73
MLP0.348\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.268\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.329\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.157\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.135\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.877\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}0.636\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.224\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}5.46
GRU0.510\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.358\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.300\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}0.214\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.176\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.885\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.644\raisebox{0.0pt}{\scriptsize{,\pm,0.007}}0.214\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}3.96
CNN0.549\raisebox{0.0pt}{\scriptsize{,\pm,0.008}}0.389\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.330\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.225\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.194\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.887\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.621\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.205\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}3.74
CEBRA0.327\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.280\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.200\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.164\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.121\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.780\raisebox{0.0pt}{\scriptsize{,\pm,0.008}}0.521\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.198\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}7.44
Table 1: Task Suite 1: Behavior decoding performance. Single-session models are trained and tested on trials from the same session. Pretrained models are finetuned on the new sessions. Sequence-level tasks are reported as Balanced Accuracy (%); frame-level task performance is reported as R2R^{2} , except for Licks, which is an event stream, where we report D2D^{2} instead. All metrics are reported as the average over evaluation sessions (  ± \,\pm\, SEM over 5 finetuning seeds). Rankings incorporate statistical significance, see Appendix E.4 for more details. Green shading denotes a TSS baseline, while magenta denotes TSU . No shading denotes a single session baseline tuned and tested on the same session.
 MethodCo-smoothingForecastingAvg
  D2D^{2}bpsD2D^{2}bpsRank
SSAutoencoder0.088\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.200\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.037\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.065\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}4.81
LFADS0.176\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.421\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.144\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.325\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}2.34
NDT0.132\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.304\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.158\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.343\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}2.79
PreNDT-Stitch0.157\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.369\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.177\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.384\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}1.64
MtM0.191\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.459\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.131\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.288\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}2.57
Table 2: Task Suite 2: Co-smoothing and forecasting performance. Model performance for both tasks is reported in terms of D2D^{2} and bps. All metrics are reported as the average over evaluation sessions (  ± \,\pm\, SEM over 5 finetuning seeds). Rankings incorporate statistical significance, see Appendix E.4 for more details. Green shading denotes a TSS baseline, while magenta denotes TSU . No shading denotes a single session baseline tuned and tested on the same session.
  Single Unit (F1)Multi-Unit (F1)
RegimeMethodLinear ProbeMLP ProbeLinear ProbeMLP Probe
TransductivePOYO+0.099\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.120\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.072\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.132\raisebox{0.0pt}{\scriptsize{,\pm,0.008}}
POSSM0.103\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.123\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.104\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.133\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}
NDT-Stitch0.136\raisebox{0.0pt}{\scriptsize{,\pm,0.007}}0.167\raisebox{0.0pt}{\scriptsize{,\pm,0.007}}0.155\raisebox{0.0pt}{\scriptsize{,\pm,0.011}}0.187\raisebox{0.0pt}{\scriptsize{,\pm,0.008}}
MtM0.137\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.140\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}0.159\raisebox{0.0pt}{\scriptsize{,\pm,0.009}}0.158\raisebox{0.0pt}{\scriptsize{,\pm,0.007}}
InductiveISI Baseline0.2630.2630.380\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.3550.3550.519\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}
Table 3: Task Suite 3: Brain region prediction performance. Results are reported as the macro-average F1 score (  ± \,\pm\, SEM over 5 seeds). For learned embeddings, seeds parameterize embedding generation, i.e., through target-session calibration (finetuning) for transductive models, and through pretraining or supervised training for inductive models. Methods are evaluated at both Single Unit and Multi-Unit levels across Linear and MLP probes. All models are evaluated in zero-shot transfer with respect to the region labels after pretraining. Green shading denotes a TSS baseline, while magenta denotes TSU . We omit shading for the ISI baseline since its embeddings are not trained, and omit SEM on its linear probe since it is fully deterministic.
Appendix figures & tables55 assets

Supplementary material from the paper’s appendix.

Appendix

Filtering Stage# SessionsUnitsUnit-hours% Retained
Original Brain Wide Map459621,733——
Unfiltered BrainWideBench452610,701830,324100%
+ Probe QC452601,124816,37198.3%
+ Firing Rate452446,311606,72873.1%
+ Unit QC45267,16090,39011.0%
Appendix
Table A1: Neural Unit Attrition Across Filtering Stages
Figure A1: Behavior QC. A : Top : Example session with qc_behavior_paws=PASS , showing paw trajectories averaged separately for correct and incorrect trials, aligned to stimulus onset. No sessions had WARNING or FAIL labels for paws. B : Top left : Example session with qc_behavior_licks=PASS , showing lick times for correct trials aligned to feedback onset. Bottom left : Example session with qc_behavior_licks=FAIL , where there is no discernible increase in lick rate during the reward period, likely due to poor tongue tracking. Right : Two example sessions with qc_behavior_licks=WARNING , showing weak but present lick rate increase ( Top ) and non-stationary lick rate pattern over the course of the session ( Bottom ). C : Top : Example session with qc_behavior_motion_energy=PASS , showing whisker pad motion energy averaged over correct trials, aligned to stimulus onset. Bottom : Example session with qc_behavior_motion_energy=FAIL , where the signal is dominated by noise. No sessions were marked WARNING .
BuildRegimeSessionsSubjectsProbesUnitsSpikesDuration (h)
all_unitspretrain423126643567,51419.21 B561.5
all_unitseval29134543,1871.47 B40.7
selected_unitspretrain42312664263,3823.75 B561.5
selected_unitseval2913373,7780.21 B40.7
Appendix
Table A2: Composition of the two dataset builds. Sessions, subjects, and recording duration are identical across builds; only the stored unit population differs. Probes counts insertions that still contribute at least one unit. Duration is total session recording time.
Figure A2: Pretrain/Evaluation Mice Split. Selection pipeline for the 13 evaluation mice from the full cohort of 139 mice. Starting from all 139 mice, 28 candidates are identified for whom every session has at least one probe passing both neural and alignment quality control and full behavioral quality control. From this pool, 10 mice are selected via a seeded greedy search optimizing for broad brain region coverage and minimal within-cohort session discards. Separately, 3 mice with confirmed cerebellum coverage are drawn from a curated pool of 8 cerebellum candidates, yielding 13 evaluation mice in total. All remaining 126 mice are assigned to pretraining. Within the evaluation cohort, 7 individual sessions failing quality control are discarded, resulting in 29 evaluation sessions and 423 pretraining sessions.
Figure A3: Brain Region Distributions between Pretrain and Evaluation Sets. Bar plots show the number of mice, sessions, and probes per Cosmos-level brain region, split by pretrain versus evaluation set. The top row shows the all_units build used for Task Suite 1, while the bottom row shows the selected_units build used for Task Suites 2 and 3.
    Tokens
BuildSessionsUnitsDuration (h)SpikeUnit-binTime-step
all_units423567,514561.519.21 B138.42 B101,082,400
selected_units42363,382561.53.75 B15.36 B101,080,400
Appendix
Table A3: Pretraining corpus size over the full spike domain of all 423 sessions. Spike tokens count one token per spike; unit-bin tokens count one per (unit, 20 ms bin); time-step tokens count one per 20 ms bin with units in the channel dimension. Binned counts tile the domain with non-overlapping 1 s windows at 20 ms resolution.
Figure A4: Probe locations across 139 subjects in BrainWideBench . Each brain shows the histology-aligned location of neurons in one mouse subject’s brain across all acquisitions performed with that subject. Number of probe insertions ranges from 1 to 16 across subjects. Probe locations include added jitter (mean 0 s.d. 300 μ\mu m Gaussian) for better visualization.
Figure A5: Train/Validation/Test Splits Across Pretraining and Evaluation Sessions. Each bar represents a single session, with colored segments indicating the temporal assignment of trials to training (green), validation (yellow), test (red) and inference (blue) splits. Splits are separated by 20-second gaps to prevent leakage and to accommodate longer context windows. Trial windows are shown in gray. Pretraining sessions use a 90/10 train/val split with no test set. Each evaluation task suite uses a split configuration tailored to its objectives: TS1 uses a causal 40/20/40 train/val/test split; TS2 uses interleaved 5-minute blocks; and TS3 uses the full session for inference.
Figure A6: TS2 Performance Under Causal (TS1) versus Interleaved (TS2) Split Strategies. Each panel shows validation and test performance for a single-session NDT-Stitch model finetuned on 10 randomly selected evaluation sessions, across four task and metric combinations: forecasting ( D2D^{2} ), forecasting (BPS), co-smoothing ( D2D^{2} ), and co-smoothing (BPS). Each line connects the validation and test performance of a single session, with the left half of each panel showing the interleaved (TS2) split and the right half showing the causal (TS1) split. Under the interleaved split, validation and test performance are broadly consistent, reflecting the stationarity of the estimation problem. Under the causal split, performance drops sharply from validation to test across nearly all sessions and metrics, illustrating the detrimental effect of neural non-stationarity over long timescales. The slight reduction in validation performance between the TS2 and TS1 regimes is attributable to hyperparameters having been tuned for the interleaved split setting.
Figure A7: Effect of Block Duration on Test D2D^{2} Score. Mean test D2D^{2} ( ±\pm SEM) across 10 fine-tuned NDT models as a function of block duration, shown for the Forecasting (left) and Co-smoothing (right) tasks. Block duration is the length, in seconds, of the contiguous recording segments used to define the non-causal train/val/test splits for TS2. The dashed line marks the geometric elbow, defined as the block duration whose mean D2D^{2} is at maximum perpendicular distance from the line connecting the first and last points.
LevelVariable NameInterval StartInterval EndNum ClassesMetric
Framewhisker_motion_energymovement_onset_time{\color[rgb]{0.8828,0.4258,0.2461}\texttt{movement\_onset\_time}}+1.0–R2\boldsymbol{R^{2}} , Pearson’s rr
wheel_speedmovement_onset_time{\color[rgb]{0.8828,0.4258,0.2461}\texttt{movement\_onset\_time}}+1.0–R2\boldsymbol{R^{2}} , Pearson’s rr
left_paw_speedmovement_onset_time{\color[rgb]{0.8828,0.4258,0.2461}\texttt{movement\_onset\_time}}+1.0–R2\boldsymbol{R^{2}} , Pearson’s rr
right_paw_speedmovement_onset_time{\color[rgb]{0.8828,0.4258,0.2461}\texttt{movement\_onset\_time}}+1.0–R2\boldsymbol{R^{2}} , Pearson’s rr
licking_ratefeedback_time{\color[rgb]{0.6641,0.3125,0.5938}\texttt{feedback\_time}}+1.0–D2\boldsymbol{D^{2}}
Seq.stimulus_contrast{\color[rgb]{0.3203,0.6563,0.3281}\texttt{stim\_on\_time}}-0.9{\color[rgb]{0.3203,0.6563,0.3281}\texttt{stim\_on\_time}}+0.15Balanced Acc , F1F1 Score, AP
Appendix
Table A4: Summary of decoding tasks with associated intervals, class counts, and evaluation metrics for TS1 (Behavior Prediction). Note: D2D^{2} is defined in Appendix E .
Figure A8: Task-Aligned Decoding Windows for Behavior Prediction (TS1). Illustration of the time windows used for each behavioral decoding target, aligned to three key trial events: stimulus onset (green), movement onset (orange), and feedback onset (purple). Sequence-level classification targets (top) include stimulus_contrast , decoded from a window around stimulus onset, and choice and reward , decoded from windows ending at and starting from movement onset and feedback onset, respectively. Frame-level regression targets (bottom) include whisker_motion_energy , wheel_speed , left_paw_speed , and right_paw_speed , decoded from a 1 second window starting at movement onset, and licking_rate , decoded from a 1 second window starting at feedback onset. Windows are designed to avoid leakage between task phases while capturing the most task-relevant period of neural and behavioral activity for each target.
SplitChoiceContrastReward
 Right ( c=0c=0 )Left ( c=1c=1 )0% ( c=0c=0 )6% ( c=1c=1 )12.5% ( c=2c=2 )25% ( c=3c=3 )100% ( c=4c=4 )False ( c=0c=0 )True ( c=1c=1 )
Pretrain0.4970.5030.1160.2190.2190.2210.2250.1680.832
Eval Train0.4990.5010.1140.2170.2190.2250.2240.1580.842
Eval Val0.4620.5380.1110.2330.2280.2050.2230.1470.853
Appendix
Table A5: Class distribution of sequence-level tasks per split.
    Tokens
SplitUnitsSamplesDuration (h)SpikeUnit-binTime-step
train43,1877,7952.1766.3 – 81.8 M583.9 M389,750
val43,1873,4710.9631.2 – 37.9 M254.6 M173,550
test43,1875,8941.6455.7 – 67.1 M424.8 M294,700
Appendix
Table A6: TS1 split statistics on the all_units build. Samples are 1 s task-aligned windows, disjoint within a task. Spike tokens count one token per spike; unit-bin tokens count one per (unit, 20 ms bin); time-step tokens count one per 20 ms bin with units in the channel dimension.
TaskWindow TypeInterval StartInterval EndMask RatioMask DimensionCriterionMetric
co-smoothingrolling (20 ms){\color[rgb]{0.3203,0.6563,0.3281}\texttt{stim\_on\_time}}-1.0{\color[rgb]{0.6641,0.3125,0.5938}\texttt{feedback\_time}}+1.010%neuronrandom held-out †D2\boldsymbol{D^{2}} , bps
forecastingrolling (20 ms){\color[rgb]{0.3203,0.6563,0.3281}\texttt{stim\_on\_time}}-1.0{\color[rgb]{0.6641,0.3125,0.5938}\texttt{feedback\_time}}+1.010%timedeterministic last timestepD2\boldsymbol{D^{2}} , bps
Appendix
Table A7: Summary of pretraining evaluation task associated with TS2 (Neural Activity Prediction).
Figure A9: Evaluation Windows for TS2. Each trial spans from 1 s before stimulus onset to 1 s after feedback, with key events (stimulus, movement, feedback) marked along the timeline. Context windows of 1 s are extracted using a rolling procedure with a 20 ms stride across the full trial interval, and serve as inputs for both the co-smoothing and forecasting tasks.
    Tokens
SplitUnitsSamplesDuration (h)SpikeUnit-binTime-step
train3,77819,0336.0528.7 M122.5 M951,650
val3,77816,9522.5725.8 M109.8 M847,600
test3,778700,4715.591067.5 M4529.5 M35,023,550
Appendix
Table A8: TS2 split statistics on the selected_units build. A sample is a 1 s window drawn by the split’s sampler, and duration is the total span of the sampling intervals the sampler draws from. Spike tokens count one token per spike; unit-bin tokens count one per (unit, 20 ms bin); time-step tokens count one per 20 ms bin with units in the channel dimension. All three are counted as sampled, so overlapping windows count the same spike more than once.
HyperparameterValue
PreprocessingStandardScaler
Solverlbfgs
Inverse regularization CC1.01.0
Max iterations10001000
Tolerance10−510^{-5}
Class weightbalanced
Appendix
Table A9: Hyperparameters used for TS3 Linear probe. Implemented as LogisticRegression classifier from sklearn . No hyperparameter tuning is performed as the solver is deterministic given fixed inputs.
HyperparameterSearch SpaceSampling type
Model parameters
Bin size (ms){5, 10, 20, 40}\{5,\,10,\,20,\,40\}Categorical
Depth{1, 2, 3}\{1,\,2,\,3\}Discrete
Hidden dimension{32, 64, 128, 256, 512}\{32,\,64,\,128,\,256,\,512\}Discrete ( log⁡2\log_{2} )
Dropout{0.0, 0.2, 0.4, 0.6}\{0.0,\,0.2,\,0.4,\,0.6\}Discrete (step 0.20.2 )
Batch norm{True, False}\{\text{True},\,\text{False}\}Categorical
Appendix
Table A10: Hyperparameter search space for TS3 MLP probe. Tuned for 100100 trials with 44 concurrent trials at a time.
Figure A10: Context Window Selection and Data Leakage Prevention. (A) While our benchmark uses a fixed 1s default for both input and target windows, the framework supports variable input context lengths (e.g., 1.5s, 2s). (B) When extending the context, a temporal gap is required to prevent data leakage from outside the valid trial boundaries.
HyperparameterSearch SpaceSampling type
Model parameters
Bin size (ms){1, 5, 10, 20, 40}\{1,\,5,\,10,\,20,\,40\}Categorical
Training parameters
Number of epochs{100, 300, 500}\{100,\,300,\,500\}Discrete (step 200200 )
Batch size{16, 32, 64}\{16,\,32,\,64\}Discrete ( log⁡2\log_{2} )
Weight decay[10−6, 10−1][10^{-6},\,10^{-1}]Loguniform
Appendix
Table A11: Hyperparameter search space for linear decoding baselines. Linear models are tuned on individual sessions and tasks for 100100 trials with 44 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters
Bin size (ms){5, 10, 20, 40}\{5,\,10,\,20,\,40\}Categorical
Depth{1, 2, 3}\{1,\,2,\,3\}Discrete
Hidden dimension{32, 64, 128, 256, 512}\{32,\,64,\,128,\,256,\,512\}Discrete ( log⁡2\log_{2} )
Dropout{0.0, 0.2, 0.4, 0.6}\{0.0,\,0.2,\,0.4,\,0.6\}Discrete (step 0.20.2 )
Batch norm{True, False}\{\text{True},\,\text{False}\}Categorical
Appendix
Table A12: Hyperparameter search space for MLP decoding baselines. MLP’s are tuned on individual sessions and tasks for 100100 trials with 44 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters
Bin size (ms){1, 5, 10, 20, 40}\{1,\,5,\,10,\,20,\,40\}Categorical
Depth †{1, 2, 4, 6, 8, 10}\{1,\,2,\,4,\,6,\,8,\,10\}Discrete
Hidden dimension{32, 64, 128, 256}\{32,\,64,\,128,\,256\}Discrete ( log⁡2\log_{2} )
Kernel size{3, 5, 7}\{3,\,5,\,7\}Discrete (step 22 )
Dropout{0.0, 0.2, 0.4, 0.6}\{0.0,\,0.2,\,0.4,\,0.6\}Discrete (step 0.20.2 )
Appendix
Table A13: Hyperparameter search space for CNN decoding baselines. CNN’s are tuned on individual sessions and tasks for 100100 trials with 44 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters
Bin size (ms){1, 5, 10, 20, 40}\{1,\,5,\,10,\,20,\,40\}Categorical
Depth{1, 2, 3}\{1,\,2,\,3\}Discrete
Hidden dimension †{32, 64, 128, 256, 512}\{32,\,64,\,128,\,256,\,512\}Discrete ( log⁡2\log_{2} )
Dropout{0.0, 0.2, 0.4, 0.6}\{0.0,\,0.2,\,0.4,\,0.6\}Discrete (step 0.20.2 )
Bidirectional{True, False}\{\text{True},\,\text{False}\}Categorical
Appendix
Table A14: Hyperparameter search space for GRU decoding baselines. GRU’s are tuned on individual sessions and tasks for 100100 trials with 44 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters
Output dimension{8, 16, 32, 64, 128, 256}\{8,\,16,\,32,\,64,\,128,\,256\}Discrete ( log⁡2\log_{2} )
Time offsets{1, 2, 3, 4, 5}\{1,\,2,\,3,\,4,\,5\}Discrete
Model learning rate[10−4, 10−3][10^{-4},\,10^{-3}]Loguniform
Temperature[0.5, 2.0][0.5,\,2.0]Discrete (step 0.10.1 )
Mode{time, behavior}\{\text{time},\,\text{behavior}\}Categorical
Appendix
Table A15: Hyperparameter search space for CEBRA baselines. CEBRA is tuned on individual sessions and tasks for 100100 trials with 44 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters
Latent step{0.0625, 0.125}\{0.0625,\,0.125\}Discrete (step 0.06250.0625 )
Num. latents per step{8, 16, 32}\{8,\,16,\,32\}Discrete ( log⁡2\log_{2} ; sampled as 2k2^{k} , k∈{3,4,5}k\in\{3,4,5\} )
Depth{2, 4, 6, 8, 10}\{2,\,4,\,6,\,8,\,10\}Discrete (step 22 )
Dropout †{0.0, 0.2, 0.4, 0.6}\{0.0,\,0.2,\,0.4,\,0.6\}Discrete (step 0.20.2 )
Dimension{32, 64, 128, 256}\{32,\,64,\,128,\,256\}Discrete ( log⁡2\log_{2} ; sampled as 2k2^{k} , k∈{5,6,7,8}k\in\{5,6,7,8\} )
Appendix
Table A16: Hyperparameter search space for POYO decoding baselines. POYO models are tuned on individual sessions and tasks for 5050 trials with 22 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters
Dimension{32, 64, 128, 256}\{32,\,64,\,128,\,256\}Discrete ( log⁡2\log_{2} )
Depth{2, 4, 6, 8, 10}\{2,\,4,\,6,\,8,\,10\}Discrete (step 22 )
Num. heads{1, 2, 4}\{1,\,2,\,4\}Discrete ( log⁡2\log_{2} )
FFN factor{1, 2, 4}\{1,\,2,\,4\}Discrete ( log⁡2\log_{2} )
Dropout{0.2, 0.4, 0.6}\{0.2,\,0.4,\,0.6\}Discrete (step 0.20.2 )
Appendix
Table A17: Hyperparameter search space for NDT supervised decoding baselines. NDT supervised’s are tuned on individual sessions and tasks for 5050 trials with 22 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Population coupling 
Smoothing width σ\sigma (bins){1, 1.5, 2, 2.5}\{1,\,1.5,\,2,\,2.5\}Categorical
Coupling exponent γ\gamma{0.5, 0.75, 1, 1.25, 1.5, 2}\{0.5,\,0.75,\,1,\,1.25,\,1.5,\,2\}Categorical
RRR and RRR (w/ ISI features) 
Rank kk{1, 2, 3, 4, 5, 8, 12, 16}†\{1,\,2,\,3,\,4,\,5,\,8,\,12,\,16\}^{\dagger}Categorical
Ridge strength λ\lambda{100,…,107}\{10^{0},\dots,10^{7}\}Categorical ( log⁡10\log_{10} )
Appendix
Table A18: Hyperparameter search space for the statistical baselines. Each method is fit per session and task by exhaustive grid search on validation, with no gradient-based training.
HyperparameterSearch SpaceSampling type
Model parameters 
Bin size (ms){1,5,10,20,40}\{1,5,10,20,40\}Categorical
Training parameters 
Number of epochs{100, 300, 500}\{100,\,300,\,500\}Discrete (STEP 200)
Batch size[16,128][16,128]Discrete ( log⁡2\log_{2} )
Weight decay[10−6,10−1][10^{-6},10^{-1}]Loguniform
Appendix
Table A19: Hyperparameter search space for Autoencoder baselines. Autoencoders are tuned on individual sessions and tasks for 100100 trials with 44 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters
Unit emb. dim{0, 1, 2}\{0,\,1,\,2\}Discrete
Num. attention heads ‡{1, 2}\{1,\,2\}Discrete ( log⁡2\log_{2} )
FFN factor{1, 2}\{1,\,2\}Discrete ( log⁡2\log_{2} )
Num. encoder layers{1, 2, 3, 4, 5, 6}\{1,\,2,\,3,\,4,\,5,\,6\}Discrete
Dropout †[0.2, 0.6][0.2,\,0.6]Uniform
Appendix
Table A20: Hyperparameter search space for NDT baselines. NDT is tuned on individual sessions for 5050 trials with 22 concurrent trials at a time.
HyperparameterSearch SpaceSampling type
Model parameters 
Generator dimension{100, 200, 400}\{100,\,200,\,400\}Categorical
Factor dimension{20, 40, 80}\{20,\,40,\,80\}Categorical
Controller output dimension{1, 2, 4, 8}\{1,\,2,\,4,\,8\}Categorical
Dropout[0,0.6][0,0.6]Discrete (step 0.050.05 )
Coordinated dropout rate †[0,0.7][0,0.7]Discrete (step 0.050.05 )
Appendix
Table A21: Hyperparameter search space for LFADS baselines. LFADS is tuned on individual sessions and tasks for 5050 trials with 44 concurrent trials at a time.
Figure A11: ISI histogram representations under linear and log-spaced binning
HyperparameterValue
Model parameters
Number of ISI bins128128
ISI range (s)[10−3,3][10^{-3},3]
Encoder hidden dims[128,64,64,32][128,64,64,32]
Encoder activationReLU
Encoder dropout0.50.5
Appendix
Table A22: LOLCAT training configuration.
HyperparameterValue
Model parameters
WVF base channels6464
ACG base channels3232
WVF residual blocks[2,3,3,3][2,3,3,3]
ACG residual blocks[2,2,2,3][2,2,2,3]
WVF representation dim512512
Appendix
Table A23: NEMO training configuration.
Figure A12: Input and Target Modalities for Pretrained Baselines . We illustrate which data types (Spikes, Behavior, Anatomy) serve as model inputs (purple) or prediction targets (pink) for each baseline model during pretraining.
Figure A13: Visualizing Masking Strategies for Pretrained Neural Models . Comparison of masking schemes used during pretraining for NDT, MtM, and NDT2. Gray shaded areas represent masked spikes (targets for reconstruction).
HyperparameterValue
Model parameters
Backbone parameters10.4210.42 M
Total parameters43.0443.04 M
Bin size (s)0.020.02
Hidden dimension256256
Encoder num. layers1313
Appendix
Table A24: Pretraining configuration.
HyperparameterValue
Model parameters
Bin size (s)0.020.02
Hidden dimension256256
Encoder num. layers1212
Encoder num. heads88
Encoder FF dimension10281028
Appendix
Table A25: Model and Training Configuration.
SuiteModelStageGPUTrials/SeedsRuntimeTotal (GPU-hr)
TS1LinearTuningA101001.2 hr (sweep)339.6
MLPTuningA101001.3 hr (sweep)378.5
GRUTuningA101001.5 hr (sweep)429.9
TCNTuningA101001.3 hr (sweep)386.1
CEBRATuningA101009.1 hr (sweep)2638.7
POYOTuningA10501.2 hr (sweep)352.6
Appendix
Table A26: Computational resources used across the three task suites. Hyperparameter tuning sweeps are run on A10 GPUs (AMD EPYC 7502); all other stages are run on B200 GPUs (Xeon Platinum 8570). Tuning runtimes are estimated by sampling 200 runs; all other figures are measured from W&B run logs for the released projects, excluding crashed and failed runs. Training/finetuning runtimes are reported as the mean over all runs in the block, where a block spans all tasks ×\times sessions ×\times seeds.
Figure A14: Per-Session Behavior Decoding Performance Across Baselines (TS1). Each panel shows performance for a single decoding target, Whisker Motion Energy, Wheel Speed, Right Paw Speed, and Left Paw Speed ( R2R^{2} , top row), and Licking Rate ( D2D^{2} ), Reward, Choice, and Stimulus Contrast (balanced accuracy, bottom row), broken down by model and evaluation session. Each dot represents a single session, with dot size proportional to the standard error of the mean (SEM) across 5 random seeds; larger dots indicate higher variability across seeds. The vertical bar indicates the median performance across sessions, and the shaded region spans the interquartile range. Dots are jittered vertically for visibility.
Figure A15: Alternative Per-Session Behavior Decoding Performance Across Baselines for Regression Tasks (TS1). Each panel shows performance for a single regression target, Whisker Motion Energy, Wheel Speed, Right Paw Speed, and Left Paw Speed, using Pearson’s rr , broken down by model and evaluation session. Plotting conventions as in Figure A14 .
Figure A16: Alternative Per-Session Behavior Decoding Performance Across Baselines for Classification Tasks (TS1). Each panel shows performance for a single classification target, Reward, Choice, and Stimulus Contrast, using F1, broken down by model and evaluation session. Plotting conventions as in Figure A14 .
 MethodLicksWhiskerWheelRPawLPawRewardChoiceStimAvg. Rank ↓\downarrow
Single-SessionLinear10.109.668.529.109.386.315.073.697.73
MLP7.767.344.216.346.035.173.723.105.46
GRU4.553.834.414.033.523.694.033.623.96
CNN3.482.863.483.592.483.905.244.863.74
CEBRA8.287.218.246.557.036.868.866.487.44
NDT8.247.038.727.837.384.455.345.416.80
Appendix
Table A27: Average per-task rankings in TS1. Comparison of average ranks across all tasks for single-session and pretrained models. Ranks are determined by pairwise one-sided Welch’s t-tests ( α=0.05\alpha=0.05 ) using standard competition (1224) ranking. Lower ranks indicate better performance.
 MethodWhiskerWheelRPawLPaw
Single-SessionLinear0.431\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.504\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.325\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.325\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
MLP0.529\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.583\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.406\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.393\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
GRU0.625\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.597\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.499\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.460\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
CNN0.652\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.616\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.502\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.472\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
CEBRA0.559\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.497\raisebox{0.0pt}{\scriptsize{,\pm,0.000}}0.420\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.384\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}
NDT0.569\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.507\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.424\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.392\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
Appendix
Table A28: Pearson’s rr on the TS1 regression tasks. Session-averaged Pearson correlation between predicted and observed behavior for the four continuous targets, reported as mean ±\pm SEM over 5 seeds. Licking rate is excluded, as Pearson’s rr is not appropriate for a count target. Higher is better.
  RewardChoiceStim
 MethodF1APF1APF1AP
Single-SessionLinear0.808\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.934\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.582\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.675\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.160\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.242\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}
MLP0.847\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}0.967\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.600\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.692\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.170\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.248\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}
GRU0.874\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.961\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.600\raisebox{0.0pt}{\scriptsize{,\pm,0.008}}0.686\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}0.151\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.242\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}
CNN0.883\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.969\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.575\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.652\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.145\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.228\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
CEBRA0.775\raisebox{0.0pt}{\scriptsize{,\pm,0.010}}0.898\raisebox{0.0pt}{\scriptsize{,\pm,0.007}}0.414\raisebox{0.0pt}{\scriptsize{,\pm,0.007}}0.549\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.118\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.223\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}
Appendix
Table A29: Secondary metrics on the TS1 classification tasks. Session-averaged F1-score and average precision (AP) for the three classification targets, reported as mean ±\pm SEM over 5 seeds. Both are macro-averaged over classes. AP is not comparable across tasks, as each has a different number of classes and class prior; chance-level macro AP is approximately the mean class prevalence ( 0.50.5 for the binary tasks under a balanced prior, 0.20.2 for the five-way Stimulus Contrast task). Higher is better.
 MethodCo-smoothingForecastingAverage
  D2D^{2}bpsD2D^{2}bpsRank
Single-SessionPopulation coupling0.0030.0030.0100.010–––
RRR0.1120.1120.3040.304–––
RRR (w/ ISI features)0.1280.1280.3280.328–––
Trailing mean––0.0020.0020.0250.025–
Shrinkage––0.0770.0770.1610.161–
Appendix
Table A30: Task Suite 2: Co-smoothing and forecasting performance. Model performance for both tasks is reported in terms of D2D^{2} and bps. All metrics are reported as the average over evaluation sessions (  ± \,\pm\, SEM over 5 finetuning seeds). Rankings incorporate statistical significance, see Appendix E.4 for more details.
Figure A17: Per-Session Neural Activity Prediction Performance Across Baselines (TS2). Left panels: D2D^{2} metric; Right panels: BPS metric. Each panel shows performance for a single task (co-smoothing left subplot or forecasting right subplot) broken down by model and evaluation session. Plotting conventions as in Figure A14 .
 MethodCo-smoothingForecastingAvg. Rank ↓\downarrow
SSAE4.694.934.81
NDT3.552.032.79
LFADS1.762.932.34
Pre.NDT-Stitch2.281.001.64
MtM1.343.792.57
Appendix
Table A31: Average per-task rankings in TS2, excluding stats baselines. Comparison of average ranks for the co-smoothing and forecasting tasks. Ranks are determined by pairwise one-sided Welch’s t-tests ( α=0.05\alpha=0.05 ) using standard competition (1224) ranking. Lower ranks indicate better performance.
 MethodCo-smoothingForecastingAvg. Rank ↓\downarrow
Single-SessionPopulation coupling7.76–7.76
RRR5.14–5.14
RRR (w/ ISI features)4.62–4.62
Trailing mean–7.597.59
Shrinkage–5.725.72
Ridge autoregression–5.035.03
Appendix
Table A32: Average per-task rankings in TS2, including stats baselines. Comparis H on of average ranks for the co-smoothing and forecasting tasks, including stats baselines. Ranks are determined by pairwise one-sided Welch’s t-tests ( α=0.05\alpha=0.05 ) using standard competition (1224) ranking. Lower ranks indicate better performance.
MethodLevelProbeCBCNUCTXspHBHPFHYIso.MBOLFTHAvg
Transductive 
POYO+SULinear0.1050.0520.0460.0820.0240.0230.0590.1410.0930.3420.097
MLP0.0830.0570.0390.1100.1020.0090.1250.2060.1780.2910.120
MULinear0.1270.0350.0690.0230.0010.0110.0320.1040.0720.4270.090
MLP0.0860.0510.0280.0850.0790.0030.1340.2460.2160.3890.132
POSSMSULinear0.0440.0650.0230.1160.1570.0280.0920.1910.0820.2300.103
Appendix
Table A33: Brain region prediction performance across held-out sessions. F1-scores are reported per region and as macro-average. Methods are evaluated on Single Unit (SU) and Multi-Unit (MU) data with both Linear and MLP probes. CB = Cerebellum, CNU = Caudate-Putamen, CTXsp = Cortical Subplate, HB = Hindbrain, HPF = Hippocampal Formation, HY = Hypothalamus, Iso. = Isocortex MB = Midbrain, OLF = Olfactory Areas, TH = Thalamus.
MethodLevelProbeCBCNUCTXspHBHPFHYIso.MBOLFTHAvg
Transductive 
POYO+SULinear0.2150.0660.2330.0590.0130.0890.0320.1020.0750.4130.130
MLP0.1780.0770.0790.1090.0760.0300.0800.1580.2170.3030.131
MULinear0.2490.0420.2830.0130.0010.0370.0170.0670.0460.7040.146
MLP0.1650.0610.0500.0760.0510.0070.0870.2030.2270.5260.145
POSSMSULinear0.0360.0570.0330.1970.1940.1300.0560.1460.0650.2510.117
Appendix
Table A34: Per-region recall on TS3 neuron identity prediction. Recall for each of the 10 Cosmos-level brain regions, reported per method, unit level (SU: single-unit; MU: multi-unit) and probe type. Values are averaged over seeds. The rightmost column is the macro-average over regions, which for recall is exactly balanced accuracy. Higher is better.
MethodLevelProbeCBCNUCTXspHBHPFHYIso.MBOLFTHAvg
Transductive 
POYO+SULinear0.0690.0430.0260.1360.1510.0130.3500.2280.1230.2920.143
MLP0.0550.0490.0270.1140.1560.0060.3050.2990.1540.2860.145
MULinear0.0860.0310.0390.0960.0330.0070.4680.2440.1750.3070.149
MLP0.0580.0510.0190.1030.1700.0020.3380.3190.2250.3170.160
POSSMSULinear0.0590.0760.0180.0830.1320.0160.2570.2770.1110.2130.124
Appendix
Table A35: Per-region precision on TS3 neuron identity prediction. Precision for each of the 10 Cosmos-level brain regions, reported per method, unit level (SU: single-unit; MU: multi-unit) and probe type. Values are averaged over seeds. The rightmost column is the macro-average over regions. Higher is better.
ModelLicksWhiskerWheelRPawLPawRewardChoiceContrast
 ( D2D^{2} )( R2R^{2} )( R2R^{2} )( R2R^{2} )( R2R^{2} )(Acc)(Acc)(Acc)
POYO+0.664\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.390\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.351\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.247\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.197\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.905\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.719\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.259\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}
POYO-ST0.680\raisebox{0.0pt}{\scriptsize{,\pm,0.006}}0.319\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.344\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.236\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.132\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.568\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.497\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.200\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}
POYO0.536\raisebox{0.0pt}{\scriptsize{,\pm,0.008}}0.282\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.309\raisebox{0.0pt}{\scriptsize{,\pm,0.005}}0.192\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.145\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.861\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.633\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.222\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
Appendix
Table A36: POYO Models Performance Summary. Results are reported as in Table 1 except avg. rank, which is not calculated. POYO+ is a multi-task supervised pre-trained model; POYO-ST is a single-task supervised pre-trained model; POYO is a single-session baseline.
ModelLicksWhiskerWheelRPawLPawRewardChoiceContrast
 ( D2D^{2} )( R2R^{2} )( R2R^{2} )( R2R^{2} )( R2R^{2} )(Acc)(Acc)(Acc)
Linear0.152\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.177\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.234\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.096\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.090\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.833\raisebox{0.0pt}{\scriptsize{,\pm,0.010}}0.624\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.217\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
NDT20.127\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.229\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.211\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.103\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.100\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}0.847\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.618\raisebox{0.0pt}{\scriptsize{,\pm,0.002}}0.214\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
NEDS0.220\raisebox{0.0pt}{\scriptsize{,\pm,0.012}}0.243\raisebox{0.0pt}{\scriptsize{,\pm,0.012}}0.068\raisebox{0.0pt}{\scriptsize{,\pm,0.010}}0.136\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.075\raisebox{0.0pt}{\scriptsize{,\pm,0.003}}0.659\raisebox{0.0pt}{\scriptsize{,\pm,0.009}}0.547\raisebox{0.0pt}{\scriptsize{,\pm,0.004}}0.206\raisebox{0.0pt}{\scriptsize{,\pm,0.001}}
Appendix
Table A37: NEDS and NDT2 Models Performance Summary. Results are reported as in Table 1 except avg. rank, which is not calculated.
BenchmarkHoursSubjectsSessionsUnitsAcross-RegionsAcross-Subject# TasksBehaviorDynamicsAnatomy
NLB [ 31 ]1.75 h47887×\times×\times4✓✓×\times
FALCON [ 32 ]50 h668789 ch.×\times×\times5✓×\times×\times
Sensorium [ 33 ]49 h7728k+×\times×\times1✓×\times×\times
Dynamic Sensorium [ 119 ]20 h101078.9k×\times×\times1✓✓×\times
BrainWideBench (Ours)600+ h139452611k✓(brain-wide)✓3 suites, 12 tasks✓✓✓
Appendix
Table A38: Comparison of neural data benchmarks across scale and evaluation axes. We summarize dataset scale using reported recording hours, subjects, sessions/recordings, and recorded units. For FALCON, “Units” denotes recording channels rather than spike-sorted units. For BrainWideBench , we report both well-isolated units and total detected units. “n/r” denotes not reported as a unified benchmark-level quantity. Prior benchmarks focus on isolated aspects of neural modeling, whereas BrainWideBench jointly evaluates behavior, dynamics, and anatomy across brain-wide recordings and animals.
More in this topic

Transfer Learning

Transfer learning studies how models reuse knowledge across tasks, domains, or modalities when target data or computation are limited.

See all 420 papers

Explore similar work

CardsList
May 8, 2026

NeuralBench: A Unifying Framework to Benchmark NeuroAI Models

Deep learning and large public datasets have recently catalyzed the proliferation of AI models for processing brain recordings. However, systematically evaluating these models remains a challenge: not only do the preprocessing pipelines, training and finetuning approaches largely vary across studies, but their downstream evaluation is often limited to small sets of tasks and/or datasets. Here, we present NeuralBench: a unified framework for benchmarking AI models of brain activity. We accompany this framework with NeuralBench-EEG v1.0 -- a large EEG benchmark that includes 36 electroencephalography (EEG) tasks and 14 deep learning architectures, and is evaluated on 94 datasets accessed through a standardized interface. This first EEG-focused release already highlights two main findings. First, current foundation models only marginally outperform task-specific models. Second, a large set of tasks (e.g. cognitive decoding, clinical predictions) remain highly challenging, even for the best models. Critically, NeuralBench is designed for the integration of new tasks, datasets, models, and neuroimaging modalities, as illustrated by preliminary extensions to MEG and fMRI datasets and models. Through this white paper, we invite the community to expand this open-source framework and work together toward a unified benchmarking standard for neuroimaging models.

Hubert Banville, Stéphane d'Ascoli, Simon Dahan +12

Lucy · Brain & AI team, Meta FAIR · MIND, Inria +2

Benchmark DesignEEG DecodingEEG Foundation Models
AskPDF
Aug 5, 2026

NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations

Decoding continuous motor trajectories from neural activity is essential for developing practical brain-computer interfaces (BCIs). However, current neural decoders are constrained by the limited scale and heterogeneity of neural recordings. In contrast, behavioral data can be collected more readily and at substantially larger scale from humans, animals, simulations, and robotic systems. Here, we introduce NeuroPB, a framework that scales neural decoding by transferring knowledge from pretrained behavioral representations. NeuroPB first pretrains a motor encoder on large-scale motor behavior data and then aligns neural activity with the resulting behavioral representation space using a limited set of paired neural-behavioral recordings. A neural encoder and lightweight motor decoder are subsequently optimized to reconstruct continuous movement from the aligned neural representations. Across multiple macaque motor datasets, behavioral pretraining improves trajectory decoding, including an 11% R2R^2 increase on center-out and 8% on random-target compared with training the motor encoder from scratch. Notably, pretraining on robotic trajectories achieves performance comparable to pretraining on macaque trajectories, demonstrating that transferable kinematic structure is shared across biological and artificial models. Moreover, decoding performance improves as the scale and diversity of robotic pretraining data increase, when the amount of neural data is fixed. Pretraining also enhances generalization across recording sessions, subjects, and motor tasks, with only 10% calibration needed to match training from scratch. Overall, these results establish behavioral pretraining as a scalable source for neural decoding and provide a promising route toward high-performance and calibration-efficient BCIs under limited neural data.

Luyao Jin, Yonghao Song, Huan Zhao +2

Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong · Department of Biomedical Engineering, School of Medicine, Tsinghua University · School of Biomedical Sciences, The Chinese University of Hong Kong

Representation LearningBrain-Computer InterfacesNeural DecodingCross-Domain Transfer Learning
AskPDF
Apr 20, 2026

OmniMouse: Scaling properties of multi-modal, multi-task Brain Models on 150B Neural Tokens

Scaling data and artificial neural networks has transformed AI, driving breakthroughs in language and vision. Whether similar principles apply to modeling brain activity remains unclear. Here we leveraged a dataset of 3.1 million neurons from the visual cortex of 73 mice across 323 sessions, totaling more than 150 billion neural tokens recorded during natural movies, images and parametric stimuli, and behavior. We train multi-modal, multi-task models that support three regimes flexibly at test time: neural prediction, behavioral decoding, neural forecasting, or any combination of the three. OmniMouse achieves state-of-the-art performance, outperforming specialized baselines across nearly all evaluation regimes. We find that performance scales reliably with more data, but gains from increasing model size saturate. This inverts the standard AI scaling story: in language and computer vision, massive datasets make parameter scaling the primary driver of progress, whereas in brain modeling -- even in the mouse visual cortex, a relatively simple system -- models remain data-limited despite vast recordings. The observation of systematic scaling raises the possibility of phase transitions in neural modeling, where larger and richer datasets might unlock qualitatively new capabilities, paralleling the emergent properties seen in large language models. Code available at https://github.com/enigma-brain/omnimouse.

Konstantin F. Willeke, Polina Turishcheva, Alex Gilbert +18

Department of Ophthalmology, Byers Eye Institute, Stanford University · 2Stanford Bio-X, Stanford University · 3Wu Tsai Neurosciences Institute, Stanford University +6

Neural DecodingVisual NeuroscienceNeural Scaling Laws
AskPDF