Rethinking Reservoir Pruning: A Dynamical Perspective for Echo State Networks

Understanding how training shapes the geometry of recurrent network dynamics is a central problem in time-series modeling. We study the emergence of low-dimensional dominant manifolds in the training of Reservoir Computing (RC) networks for temporal forecasting tasks. For a general linear continuous-time reservoir in the infinite-data limit, we show that the training data generate an invariant subspace of the trained reservoir, whose dimension equals the number of dominant modes. We then specialize to a simplified diagonal linear reservoir, where we link the dominant eigenvalues and eigenvectors to the spectrum of a backward Dynamic Mode Decomposition matrix, which yields a finite-dimensional approximation of the backward-time Koopman operator of the system generating the training data. We illustrate the emergence of these dominant modes during training in simulation, and discuss how the analysis may be extended to nonlinear RC via tangent dynamics and differential p-dominance.

Reservoir computing has emerged as an efficient machine learning framework for predicting time series generated by dynamical systems. In contrast to other machine and deep learning approaches, a reservoir computing trains only the output layer via linear regression, leaving the reservoir (recurrent layer) untrained. This simplification makes reservoir computers easier to train and more amenable to experimentation. However, because current reservoirs consist of networks of randomly connected nodes and require the optimization of numerous hyperparameters, a framework that precisely explains how reservoir computing operates and how it can be optimized remains missing. Here, we propose a frequency-based reservoir inspired by the brain's oscillatory dynamics and its hierarchy of timescales. The frequency-based reservoir can be interpreted as an ensemble of independent oscillatory units, each processing a portion of the input's frequency content. This allows us to understand the reservoir's internal behavior by modeling it as a single unit driven by an external input. Borrowing from the theory of a nonlinear oscillator forced by complex periodic inputs, we found that units of the frequency-based reservoir selectively amplify and store specific input frequencies, which are then used for prediction. The frequency-based reservoir performs as well as or better than equivalent random reservoirs. Furthermore, the frequency-based approach can be optimized to improve short-term prediction, a property that random reservoirs lack. Finally, we show that the frequency-based reservoir can also predict complex spatiotemporal dynamics. Our results show that reservoir computing can be designed using brain properties and theoretical insights borrowed from the physics of forced nonlinear oscillators.

Echo-state networks enable efficient temporal learning by fixing the recurrent dynamics and training only a linear readout. However, conventional reservoirs typically accommodate signal mixing, memory retention, and stability within a single random recurrent matrix. Existing structured designs improve topology, norm preservation, leakage, or depth, but generally do not provide separate modal control of reversible mixing and irreversible forgetting together with a direct global stability guarantee. We introduce a classical Lindblad-inspired multi-timescale reservoir that bridges open-system dynamical principles with structured state-space modeling. The recurrent operator is assembled from exactly discretized damped rotational modes, so rotation and decay become independent design variables governing phase mixing and memory loss. Orthogonal mode mixing preserves normality, while the decay spectrum directly determines the echo-state stability margin without post-hoc spectral-radius rescaling. We evaluate the method over ten aligned seeds against standard, leaky, deep, orthogonal, cycle, and next-generation reservoirs, together with a compact trained gated recurrent unit, across linear memory, nonlinear recurrence, chaotic forecasting, delayed logic, and real sensor calibration. Across the benchmark suite, the proposed reservoir achieves the best fixed-reservoir performance on bounded NARMA-20 and the lowest mean error on Lorenz-63, matches the strongest linear-memory result, and remains broadly competitive across broad range of benchmarks. Ablation studies show that rotation increases state diversity, whereas dissipation provides controlled forgetting and improves predictive conditioning. The resulting framework offers an interpretable recurrent architecture in which mixing, memory, and stability are explicit and independently tunable design variables.