Model- and Layer-Level Closed-Loop Forecasting with SSM-Based Spiking Neural Networks

Erik Stolt; Sanja Karilanova; Ayca Ozcelikkale

cs.LGOct 20, 2026

Model- and Layer-Level Closed-Loop Forecasting with SSM-Based Spiking Neural Networks

Authors: Erik Stolt, Sanja Karilanova, Ayca Ozcelikkale

Organizations: Department of Electrical Engineering, Uppsala University, Uppsala, Sweden

Abstract

Deep State-Space Models (Deep SSMs) have shown promising performance and computational benefits in processing of time-series data with complex temporal dependencies. Spiking neural networks (SNNs) constitute a biologically inspired class of deep SSMs, where computing blocks communicate through spikes, discrete single-bit events rather than continuous activation functions. A particularly important task in time-series processing is forecasting, i.e. predicting future values of a sequence from past observations alone. However, how to best perform forecasting with SNNs is an open question, where the standard autoregressive nature of the data processing with SNNs requires values that are not available during the forecasting phase. In this article, we address this challenge. We explore two closed-loop forecasting strategies, in which the network's own generated values are fed back into the network as if they were observed past values: (i) model-level and (ii) layer-level closed-loop forecasting. We evaluate these approaches on the ETT-small datasets, a standard benchmark for time-series forecasting, under both spike-based and continuous activation functions. We compare the proposed closed-loop approaches against state-of-the-art methods such as SpaceTime, as well as the zero-masking baseline in S4. Both of the proposed approaches perform comparably to or better than zero-masking with S4, and in most cases match SpaceTime. These findings highlight the trade-off between continuous and low-bit quantized activations, and demonstrate that the proposed Deep SSM architectures provide a competitive framework for forecasting.

Figures & tables

Fig. 1 : Deep SSM architecture with an input layer consisting of

m

input features, followed by

L

SSM layers, and an output layer consisting of

q

output features.

TABLE I : Univariate time-series forecasting results on three ETT-small datasets. Results are averaged over five random initializations; subscripts denote the standard deviation. Results for S4 and SpaceTime are taken from [ 6 ] . Bold denotes the best and underline the second-best model per row.

Fig. 2 : Average elapsed time in seconds per epoch when training on the ETTh1 dataset. All models scale approximately linear with context length.

Panel of Figure 4 — Fig. 3 : Example forecasting samples for all strategies on the ETT-small datasets with context length $P=720$ , forecast horizon $F=96$ , and with Signed-Spk. activation .

Description	Parameter Value
Input dimension per SSM neuron	$n_{\text{in}}=1$
Output dimension per SSM neuron	$n_{\text{out}}=1$
Number of stacked SSM neurons	$h=64$
Number of SSM layers	$L=3$
State dimension per neuron	$n=128$
Dropout rate	$0.0$

TABLE II : Model parameters used in the univariate forecasting of ETT-small.

Parameter	Description
$m$	Number of input features in the input layer
$q$	Number of output features in the output layer
$n_{\text{in}}$	Input dimension per SSM neuron
$n_{\text{out}}$	Output dimension per SSM neuron
$h$	Number of parallel SSM neurons
$L$	Number of SSM layers

TABLE III : Notation used in the model architecture and in network layer

\ell

Parameter	Number of Trainable Parameters
$\bm{W}^{(\ell)}\in\mathbb{R}^{d_{\text{in}}\times d_{\text{out}}}$	$d_{\text{in}}d_{\text{out}}$
$\vec{b}^{(\ell)}\in\mathbb{R}^{d_{\text{in}}}$	$d_{\text{in}}$
$\bm{A}\in\mathbb{R}^{n}$ (diag)	$hn\,/\,2$
$\bm{A}\in\mathbb{C}^{n}$ (diag)	$hn$
$\bm{B}\in\mathbb{R}^{n\times n_{\text{in}}}$	$n_{\text{in}}n\,/\,2$
$\bm{C}\in\mathbb{R}^{n_{\text{out}}\times n}$	$n_{\text{out}}n\,/\,2$

TABLE IV : Number of trainable parameters for the components of the Deep SSM.

\bm{W}^{(\ell)}

denotes the mixing matrix between layer

\ell

and

\ell+1

with the corresponding bias vector

\vec{b}^{(\ell)}

Panel of Figure 8 — Fig. 4 : Example forecasting samples for all strategies on the ETT-small datasets with context length $P=720$ , forecast horizon $F=96$ , and with GELU activation .

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 858 papers

Sep 28, 2026

SpikeLite: Lightweight Spiking Neural Networks for Time-Series Forecasting

Spiking neural networks (SNNs) offer an energy-efficient paradigm for time-series forecasting through spike-driven computation. However, recent SNN forecasters often pursue higher accuracy through increasingly complex attention mechanisms, or specialized neuronal dynamics, weakening the lightweight motivation of SNNs. We introduce SpikeLite, a spiking forecasting framework built around two modules: a Frequency-Selective Spiking Encoder (FSSE) for frequency-sensitive temporal encoding and a Sparse Spiking Channel Attention (SSCA) module for selective cross-channel interaction. FSSE exploits the low-pass filtering behavior of LIF dynamics to reorganize each input sequence into frequency-sensitive components while collectively preserving the input at the decomposition stage. SSCA then learns a binary mask from encoded channel representations and uses it to selectively exchange information within spike-driven self-attention, retaining informative cross-channel interactions while suppressing redundant ones. When explicit channel interaction is unnecessary, SpikeLite uses the lighter FSSE-only channel-independent path. Experiments under the SeqSNN and SpikF protocols cover four standard multivariate and eight long-term forecasting benchmarks. SpikeLite achieves the best aggregate performance under both protocols, with an average

R^2

of 0.790 and RSE of 0.440, and lowest average MSE/MAE of 0.343/0.345 in long-term forecasting. Moreover, evaluation on the ECL dataset shows that SpikeLite achieves the lowest reported energy consumption, further demonstrating its potential for energy-efficient time-series forecasting.

Bang Hu, Changze Lv, Mingjie Li +3

School of Computer Science, Fudan University, Shanghai, China

Multivariate Time Series Forecasting Efficient Neural Network Inference Time Series Forecasting Spiking Neural Networks Sparse Attention

Ask PDF

May 15, 2026

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

State Space Models (SSMs) are increasingly deployed in the Edge because they offer, at comparable performance, a smaller memory/training/inference footprint, compared to Large Language Models (LLMs). These three advantages are a direct consequence of the time recurrence inherent in the SSMs architecture. Here, we further improve this recurrent architecture by positively answering two previously underexplored, orthogonal questions: (1) Can we reduce SSMs memory-footprint without any performance penalty, by also employing depth recurrence? (2) Can we increase SSMs performance by using a fixed and consistent time-granularity across all tasks? The first question is somewhat unexpected, given that SSMs are already recurrent. However, the orthogonal depth recurrence further decreases SSMs memory footprint. We show that a looped SSM with

k

parameters adaptively iterated

M

times, achieves a performance comparable to a standard SSM with

k \cdot L

independent parameters, where

M \leq L

. The second question is also unexpected given the time-recurrent nature of the SSMs architecture. However, it makes perfect sense for the time-parallel training of SSMs on the entire input sequence. We show that concatenating time steps for lower-dimensional sequence elements, or flattening and re-chunking the joint feature-time dimension for high-dimensional ones, can improve the baseline by enhancing the way information is presented to the model. Our results for both extensions lead to consistent benefits across four representative SSM architectures: LRU, S5, LinOSS, LrcSSM.

Mónika Farsang, Ramin Hasani, Daniela Rus +1

1TU Wien · MIT CSAIL · Liquid AI

Parameter Sharing State Space Models

Ask PDF

Apr 3, 2026

Adversarial Robustness of Deep State Space Models for Forecasting

State-space model (SSM) for time-series forecasting have demonstrated strong empirical performance on benchmark datasets, yet their robustness under adversarial perturbations is poorly understood. We address this gap through a control-theoretic lens, focusing on the recently proposed Spacetime SSM forecaster. We first establish that the decoder-only Spacetime architecture can represent the optimal Kalman predictor when the underlying data-generating process is autoregressive - a property no other SSM possesses. Building on this, we formulate robust forecaster design as a Stackelberg game against worst-case stealthy adversaries constrained by a detection budget, and solve it via adversarial training. We derive closed-form bounds on adversarial forecasting error that expose how open-loop instability, closed-loop instability, and decoder state dimension each amplify vulnerability - offering actionable principles towards robust forecaster design. Finally, we show that even adversaries with no access to the forecaster can nonetheless construct effective attacks by exploiting the model's locally linear input-output behavior, bypassing gradient computations entirely. Experiments on the Monash benchmark datasets highlight that model-free attacks, without any gradient computation, can cause at least 33% more error than projected gradient descent with a small step size.

Sribalaji C. Anand, George J. Pappas

Department of Electrical and Systems Engineering, University of Pennsylvania, United States · KTH Royal Institute of Technology, Sweden

Adversarial Robustness State Space Models Black-Box Adversarial Attacks

Ask PDF

Abstract

Time Series Forecasting

Explore similar work

SpikeLite: Lightweight Spiking Neural Networks for Time-Series Forecasting

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

Adversarial Robustness of Deep State Space Models for Forecasting