NeuroHorizon: Long-Horizon Forecasting of Neural Population Activity with Hierarchical Memory
Organizations: Center for Life Sciences and Artificial Intelligence, School of Life Sciences, Tsinghua University, Beijing, China. · IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China. · Tsinghua-Peking Center for Life Sciences, Tsinghua University, Beijing, China.
Abstract
Long-horizon forecasting of neural population activity tests whether a model has captured dynamics that remain predictive well beyond the observed history. Prediction quality typically declines as the horizon grows, yet neither short-horizon accuracy nor teacher-forced evaluation, which supplies the true activity preceding each predicted step, reveals how well models sustain predictions from an initial history alone. We compare direct multi-step prediction and autoregressive rollout on spiking recordings across horizons and introduce NeuroHorizon, which pairs event-based spike encoding with a causal decoder for population firing rates. Its hierarchical prediction memory keeps recent predictions at full temporal resolution and compresses earlier ones into segment summaries, giving the decoder both local detail and longer-range context. Under teacher-forced training this memory holds ground-truth activity, whereas during rollout it holds the model's own predictions. To reduce this mismatch, we train with scheduled sampling, progressively exposing the decoder to its own predictions. From 500 ms of macaque motor-cortex activity, NeuroHorizon achieves the highest forward-prediction score among the evaluated autonomous models at all four horizons from 100 to 1000 ms, and its per-bin predictions remain above a mean-firing-rate baseline across nearly the full 1 s window. Complementary experiments on macaque sensorimotor and mouse multi-region recordings show that performance generally increases with model and data size, alongside improved prediction after cross-subject pretraining and fine-tuning. Together, these results clarify how forecasting performance changes across horizons and offer an effective approach to long-horizon forecasting of neural population activity.
Figures & tables
| Dataset | Sessions | Subjects | Region(s) | Task |
|---|---|---|---|---|
| Perich–Miller dataset [ 16 ] | 10 | 3 | M1 + PMd | Instructed-delay center-out reaching |
| O’Doherty–Sabes dataset [ 25 ] | 47 | 2 | M1 + S1 | Continuous hand-controlled 2D cursor reaching |
| Allen Neuropixels dataset [ 26 ] | 32 | 32 | VIS, CA1, etc. | Passive natural movie viewing |
| Model (mode) | 100 ms | 240 ms | 500 ms | 1000 ms |
|---|---|---|---|---|
| Autonomous forecasting | ||||
| IBL-MTM (Direct) | 0.226 (0.019) | 0.200 (0.013) | 0.120 (0.011) | -0.018 (0.025) |
| POCO (Direct) | 0.177 (0.020) | 0.136 (0.019) | 0.070 (0.017) | 0.035 (0.007) |
| Neuroformer (RO) | -0.707 (0.059) | -0.779 (0.066) | -2.478 (0.241) | -5.473 (0.117) |
| LFADS (RO) | 0.212 (0.017) | 0.204 (0.014) | 0.172 (0.014) | 0.139 (0.011) |
| Causal NDT2 (RO) | 0.235 (0.019) | 0.156 (0.015) | 0.095 (0.017) | 0.035 (0.021) |
| 500 ms | 1000 ms | |||
| Configuration | TF | RO | TF | RO |
| (a) Prediction memory | ||||
| No memory | 0.174 (0.014) | 0.174 (0.014) | 0.139 (0.012) | 0.139 (0.012) |
| Only tail memory | 0.278 (0.024) | 0.182 (0.015) | 0.291 (0.026) | 0.126 (0.013) |
| Only segment memory | 0.268 (0.025) | 0.169 (0.015) | 0.274 (0.027) | 0.126 (0.012) |
| NeuroHorizon (tail and segment memory) | 0.281 (0.025) | 0.188 (0.015) | 0.295 (0.027) | 0.143 (0.011) |
| 500 ms | 1000 ms | |||
|---|---|---|---|---|
| Setting | TF | RO | TF | RO |
| From scratch (5 Loco) | 0.058 (0.008) | 0.026 (0.007) | 0.060 (0.010) | 0.018 (0.008) |
| Transfer full fine-tune (37 Indy 5 Loco) | 0.082 (0.008) | 0.035 (0.009) | 0.076 (0.010) | 0.015 (0.010) |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Value | Hyperparameter | Value |
|---|---|---|---|
| Encoder (POYO+) | Training | ||
| Number of layers ( ) | 6 | Optimizer | SparseLamb |
| Hidden dimension ( ) | 128 | Peak learning rate | batch size |
| Attention heads | 8 self, 2 cross | LR schedule | OneCycleLR with cosine annealing |
| Head dimension ( ) | 64 | Weight decay | |
| Dropout (attention, FFN) | 0.2 | Batch size | 32 |
| Horizon (ms) | Model (mode) | C1 | C2 | C3 | C4 | J1 | J2 | J3 | M1 | M2 | M3 | Mean (SEM) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 100 | IBL-MTM (Direct) | 0.171 | 0.147 | 0.139 | 0.198 | 0.266 | 0.290 | 0.299 | 0.273 | 0.243 | 0.237 | 0.226 (0.019) |
| POCO (Direct) | 0.133 | 0.104 | 0.091 | 0.147 | 0.245 | 0.254 | 0.268 | 0.163 | 0.168 | 0.199 | 0.177 (0.020) | |
| LFADS (RO) | 0.163 | 0.159 | 0.121 | 0.186 | 0.264 | 0.275 | 0.262 | 0.253 | 0.207 | 0.225 | 0.212 (0.017) | |
| Causal NDT2 (RO) | 0.176 | 0.165 | 0.144 | 0.193 | 0.270 | 0.297 | 0.303 | 0.269 | 0.270 | 0.259 | 0.235 (0.019) | |
| NeuroHorizon (RO) | 0.192 | 0.185 | 0.156 | 0.222 | 0.294 | 0.345 | 0.331 | 0.283 | 0.267 | 0.262 | 0.254 (0.020) | |
| Causal NDT2 (TF) | 0.222 | 0.219 | 0.204 | 0.237 | 0.364 | 0.385 | 0.376 | 0.322 | 0.324 | 0.310 | 0.296 (0.022) |
| Model (mode) | 100 ms | 240 ms | 500 ms | 1000 ms |
|---|---|---|---|---|
| Autonomous forecasting | ||||
| IBL-MTM (Direct) | 0.245 (0.017) | 0.237 (0.018) | 0.211 (0.019) | 0.175 (0.018) |
| POCO (Direct) | 0.230 (0.017) | 0.217 (0.018) | 0.200 (0.019) | 0.183 (0.017) |
| Neuroformer (RO) | 0.128 (0.018) | 0.096 (0.014) | -2.722 (1.341) | -1.079 (0.449) |
| LFADS (RO) | 0.242 (0.017) | 0.238 (0.018) | 0.226 (0.019) | 0.209 (0.016) |
| Causal NDT2 (RO) | 0.248 (0.017) | 0.217 (0.018) | 0.197 (0.015) | 0.174 (0.021) |
| Model (mode) | Global fp-bps | |
|---|---|---|
| Autonomous forecasting | ||
| IBL-MTM (Direct) | 0.2475 | 0.1067 |
| POCO (Direct) | 0.2162 | 0.1036 |
| NeuroHorizon (RO) | 0.2722 | 0.1114 |
| TF evaluation | ||
| NeuroHorizon (TF) | 0.3357 | 0.1226 |
| Model | fp-bps | Rate | Latent | Trajectory corr. |
|---|---|---|---|---|
| NeuroHorizon | 0.1368 | 0.5535 | 0.7225 | 0.9517 |
| Causal NDT2 | 0.1595 | 0.6835 | 0.7078 | 0.9491 |
| Observation history | Rollout fp-bps | vs. 0.5 s |
|---|---|---|
| 0.5 s | — | |
| 1.0 s | ||
| 2.0 s |
| Tail | Visible tokens | Rollout fp-bps | Last-five-bin fp-bps |
|---|---|---|---|
| 1 | 14 | 0.0561 | |
| 4 | 26 | 0.0770 | |
| 8 | 42 | 0.0720 |
| Size | Hidden dim | Encoder depth | Decoder depth | Backbone params |
|---|---|---|---|---|
| Tiny | 64 | 3 | 1 | 0.45M |
| Small | 96 | 4 | 2 | 1.30M |
| Base | 128 | 6 | 2 | 4.52M |
| Large | 192 | 8 | 3 | 8.33M |
| XLarge | 256 | 12 | 4 | 20.00M |