Free-Probability Kernels for Zero-Rollout Hyperparameter Selection in Reservoir Computing

Sara Malacarne; Andrea Ceni; Claudio Gallicchio

cs.LGAug 21, 2026

Free-Probability Kernels for Zero-Rollout Hyperparameter Selection in Reservoir Computing

Authors: Sara Malacarne, Andrea Ceni, Claudio Gallicchio

Organizations: Telenor Research & Innovation, Oslo, Norway · Department of Computer Science, University of Pisa, Pisa, Italy

Abstract

Reservoir computing (RC) couples a fixed recurrent dynamical system with a trained lightweight readout, but this efficiency is partly lost during hyperparameter selection: the recurrent gain, input scale, and leakage rate determine the reservoir's stability and temporal processing regime and are usually tuned through many rollouts. We introduce a deterministic, pilot-informed selector for leaky linear reservoirs followed by coordinate-wise nonlinear features. Free probability yields cross-lag propagation coefficients that summarize how the reservoir mixes past inputs. In the large-width limit, these coefficients define a deterministic temporal kernel that approximates the finite-reservoir feature geometry. Kernel ridge regression on a short labelled pilot sequence therefore ranks candidate operating regimes without instantiating or rolling out a reservoir, and the selected configuration transfers across widths. Across ten synthetic temporal benchmarks, zero-rollout selection obtains a mean deployment score of $0.772$ , compared with $0.774$ for exhaustive simulation-based search, while avoiding $156\,600$ selection rollouts. With a small rollout budget, the proposed ranking provides the strongest mean performance at every tested budget and reaches the exhaustive reference using $4.8\%$ of its rollout cost. On four public electricity-transformer-temperature (ETT) forecasting datasets, five retained candidates recover the exhaustive operating point on three datasets. On multivariate cellular-traffic forecasting, 15 rollouts per cell reach the 462-rollout exhaustive reference and outperform random search and Bayesian optimization at low budgets. These results position free-probability kernels as deterministic surrogates for selecting reservoir operating regimes when validation rollouts are scarce.

Figures & tables

Task	Washout	Deploy train	Test	Input, target, and task parameter
NARMA10	100	800	800	$u_{t}\!\sim\!U(0,0.5)$ ; NARMA target, order $k=10$
NARMA20	100	1200	1200	$u_{t}\!\sim\!U(0,0.5)$ ; NARMA target, order $k=20$
NARMA30	100	1200	1200	$u_{t}\!\sim\!U(0,0.5)$ ; NARMA target, order $k=30$
NARMA50	100	1600	1600	$u_{t}\!\sim\!U(0,0.5)$ ; NARMA target, order $k=50$
MC	100	2000	2000	$u_{t}\!\sim\!U(-1,1)$ ; delayed targets at lags $1{:}200$
Inubushi	100	1000	1000	$u_{t}\!\sim\!U(-1,1)$ ; $y_{t}=\sin(2u_{t-5})$

TABLE I: Synthetic task protocols. Deployment train/test lengths are counted after washout. Forecast horizons are given in task time steps.

Method	Selection rollouts	$n=1\,000$	$n=3\,000$	$n=5\,000$	$n=10\,000$	$n=20\,000$
FP	0	0.692 $\pm$ 0.171	0.744 $\pm$ 0.159	0.756 $\pm$ 0.158	0.768 $\pm$ 0.158	0.772 $\pm$ 0.158
Memory 500	156,600	0.619 $\pm$ 0.194	0.668 $\pm$ 0.193	0.683 $\pm$ 0.197	0.698 $\pm$ 0.188	0.703 $\pm$ 0.186
Direct 500	156,600	0.720 $\pm$ 0.178	0.749 $\pm$ 0.164	0.762 $\pm$ 0.158	0.769 $\pm$ 0.157	0.774 $\pm$ 0.156
ESN Direct 500	156,600	0.708 $\pm$ 0.203	0.725 $\pm$ 0.195	0.727 $\pm$ 0.188	0.731 $\pm$ 0.188	0.734 $\pm$ 0.186

TABLE II: Average deployment score across ten synthetic benchmarks under the holdout-ridge protocol. Scores are

1-\operatorname{NRMSE}

; higher is better. Mean

\pm

standard deviation is across tasks. Best and second-best results are bold and underlined, respectively.

Fig. 1: Deployment score at

n=20\,000

across the ten-task synthetic suite under the holdout-ridge protocol. Scores are

1-\operatorname{NRMSE}

; higher is better. Error bars show the standard deviation over deployment seeds.

Fig. 2: Average ranks across the ten synthetic tasks at

n=20\,000

. Lower is better. Horizontal cliques indicate no significant pairwise difference under Wilcoxon signed-rank tests with Holm correction (

\alpha=0.05

Fig. 3: Measured aggregate selection CPU time over the ten synthetic tasks as a function of selection width $n_{\rm select}$ . All finite-reservoir selector points are measured; connecting lines are guides to the eye and do not represent fitted complexity laws. FP does not take $n_{\rm select}$ as an input, so its repeated measured runtimes are summarized by a horizontal mean line, with individual markers and a shaded $\pm 1$ standard-deviation band showing run-to-run timing variability. FP uses the closed-form erf surrogate employed for synthetic selection; see Sec. III-C . Lower is better.

$K$	Ten-task rollouts	FP-K	Random-K	TPE-K
25	750 (0.5%)	0.774 $\pm$ 0.159	0.669 $\pm$ 0.234	0.660 $\pm$ 0.244
50	1,500 (1.0%)	0.779 $\pm$ 0.162	0.701 $\pm$ 0.232	0.716 $\pm$ 0.214
100	3,000 (1.9%)	0.775 $\pm$ 0.168	0.732 $\pm$ 0.219	0.742 $\pm$ 0.209
250	7,500 (4.8%)	0.777 $\pm$ 0.166	0.750 $\pm$ 0.194	0.769 $\pm$ 0.167
Full Direct 500 reference		0.774 $\pm$ 0.156

TABLE III: Budgeted synthetic comparison at

n=20\,000

. Rows indicate the retained candidate budget

K

; scores are deployment

1-\operatorname{NRMSE}

(higher is better), averaged across the ten tasks. Percentages are relative to exhaustive Direct500 selection.

Dataset	FP	Memory 500	Direct 500	ESN Direct 500
ETTh1	0.556 $\pm$ 0.16	0.522 $\pm$ 0.11	0.567 $\pm$ 0.17	0.564 $\pm$ 0.18
ETTh2	0.664 $\pm$ 0.15	0.525 $\pm$ 0.05	0.664 $\pm$ 0.15	0.662 $\pm$ 0.15
ETTm1	0.505 $\pm$ 0.19	0.498 $\pm$ 0.18	0.499 $\pm$ 0.19	0.508 $\pm$ 0.20
ETTm2	0.587 $\pm$ 0.24	0.579 $\pm$ 0.16	0.587 $\pm$ 0.24	0.587 $\pm$ 0.24
Telco	0.572 $\pm$ 0.03	0.619 $\pm$ 0.05	0.635 $\pm$ 0.04	0.607 $\pm$ 0.03

TABLE IV: Three-horizon real-data forecasting summary at

n=20\,000

. Each entry is the mean

\pm

standard deviation over the three reported physical horizons (1, 6, and 12 h). Horizon-level scores, whose deviations are instead over deployment seeds (ETT) or cells (Telco), are given in Sec. S-VII of the SM. The Telco aggregate over horizons

1{:}12

is reported in Table V . Scores are

1-\operatorname{NRMSE}

; higher is better.

Dataset	Selector	$K=5$ 15 rollouts	$K=10$ 30 rollouts	$K=25$ 75 rollouts	$K=50$ 150 rollouts	Full Direct 500
ETTh1	FP-K	0.567 $\pm$ 0.00035	0.567 $\pm$ 0.00035	0.567 $\pm$ 0.00035	0.567 $\pm$ 0.00035	0.567 $\pm$ 0.00035
	Random-K	0.561 $\pm$ 0.008	0.563 $\pm$ 0.009	0.568 $\pm$ 0.002	0.567 $\pm$ 0.002
	TPE-K	0.561 $\pm$ 0.013	0.571 $\pm$ 0.007	0.567 $\pm$ 0.005	0.567 $\pm$ 0.005
ETTh2	FP-K	0.665 $\pm$ 0.00049	0.665 $\pm$ 0.00049	0.665 $\pm$ 0.00049	0.665 $\pm$ 0.00049	0.665 $\pm$ 0.00049
	Random-K	0.609 $\pm$ 0.047	0.645 $\pm$ 0.015	0.654 $\pm$ 0.011	0.661 $\pm$ 0.003
	TPE-K	0.626 $\pm$ 0.039	0.635 $\pm$ 0.032	0.639 $\pm$ 0.026	0.660 $\pm$ 0.007

TABLE V: Budgeted real-data forecasting comparison at

n=20\,000

. Each selector evaluates

K

candidates using

3K

finite-reservoir rollouts.

\dagger

denotes FP-K significantly outperforming both Random-K and TPE-K after Holm correction.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Recurrent matrix	Coefficients	Complete-history control	Nonlinear-kernel status
Real Ginibre	$\eta_{k,\ell}=\tau_{k,\ell}$	Resolvent bound	Exact $\tanh$ by quadrature; optional erf surrogate for the synthetic and timing experiments.
Haar orthogonal	$\eta_{k,\ell}=\tau_{k,\ell}$	Direct normal-matrix bound	Same real-valued nonlinear-kernel construction.
Cyclic shift	$\eta_{k,\ell}=\tau_{k,\ell}$ for fixed lags and large $n$	Direct normal-matrix bound	Same real-valued nonlinear-kernel construction.
Circulant	Fourier integral ( 62 )	Uniform symbol bound	Same construction with the Fourier coefficients.
Gaussian skew-symmetric	Semicircle integral ( 63 )	Normal-matrix bound	Same construction with the skew-symmetric coefficients.
Complex Ginibre / Haar unitary	$\eta_{k,\ell}^{\mathbb{C}}=\tau_{k,\ell}$	Resolvent / normal-matrix bound	Linear covariance explicit; nonlinear kernel requires a complex feature convention.

Appendix

TABLE VI: Recurrent-matrix catalogue. The coefficient column gives the quantities substituted for

\tau_{k,\ell}

in ( 36 ); the nonlinear kernel is then formed as in ( 39 ).

Task	Selector	$n=1\,000$	$n=3\,000$	$n=5\,000$	$n=10\,000$	$n=20\,000$
MC	FP	0.322 $\pm$ 0.023	0.418 $\pm$ 0.004	0.439 $\pm$ 0.001	0.457 $\pm$ 0.006	0.463 $\pm$ 0.003
	Memory 500	0.184 $\pm$ 0.004	0.211 $\pm$ 0.004	0.214 $\pm$ 0.003	0.279 $\pm$ 0.005	0.283 $\pm$ 0.001
	Direct 500	0.322 $\pm$ 0.023	0.418 $\pm$ 0.004	0.439 $\pm$ 0.001	0.457 $\pm$ 0.006	0.463 $\pm$ 0.003
	ESN Direct 500	0.223 $\pm$ 0.004	0.268 $\pm$ 0.015	0.302 $\pm$ 0.003	0.318 $\pm$ 0.006	0.328 $\pm$ 0.002
NARMA10	FP	0.763 $\pm$ 0.014	0.850 $\pm$ 0.012	0.854 $\pm$ 0.004	0.871 $\pm$ 0.010	0.870 $\pm$ 0.034
	Memory 500	0.720 $\pm$ 0.014	0.782 $\pm$ 0.008	0.789 $\pm$ 0.002	0.804 $\pm$ 0.004	0.805 $\pm$ 0.030

Appendix

TABLE VII: Full task-level deployment scores under the holdout-ridge protocol. Values are

1-\operatorname{NRMSE}

mean

\pm

std over deployment seeds. Intermediate widths use three deployment seeds;

n=20\,000

uses ten deployment seeds. Best score per task and width is bolded; second-best is underlined. Identical FP and Direct 500 rows (e.g., MC) indicate that both selectors chose the same configuration.

Fig. 4: Deployment score vs. reservoir width $n$ through $n=20\,000$ for the ten-task, four-selector comparison. Shaded bands show $\pm 1$ s.d. over deployment seeds; the $n=20\,000$ column uses ten deployment seeds and the intermediate widths use three. Higher is better.

$T_{\rm select}$	FP $\uparrow$	Memory 500 $\uparrow$	Direct 500 $\uparrow$	ESN Direct 500 $\uparrow$
150	$0.627{\pm}0.196$	$0.727{\pm}0.165$	$0.662{\pm}0.176$	$0.640{\pm}0.261$
250	$0.744{\pm}0.148$	$0.727{\pm}0.165$	$0.688{\pm}0.188$	$0.626{\pm}0.273$
500	$0.771{\pm}0.158$	$0.727{\pm}0.165$	$0.705{\pm}0.147$	$0.698{\pm}0.191$
1000	$0.754{\pm}0.185$	$0.727{\pm}0.165$	$0.746{\pm}0.176$	$0.710{\pm}0.190$
2000	$0.749{\pm}0.202$	$0.727{\pm}0.165$	$0.761{\pm}0.169$	$0.727{\pm}0.192$

Appendix

TABLE VIII: Selection-length sensitivity in an auxiliary coarse-grid synthetic sweep. Scores are deployment

1-\operatorname{NRMSE}

n=20\,000

, averaged over the ten synthetic tasks;

\pm

denotes task-level standard deviation. The selection length

T_{\rm select}

is used only during hyperparameter selection. FP uses the closed-form erf-surrogate selection kernel from Remark 1 ; matched exact-

\tanh

checks are reported in Sec. F-A . Memory 500 uses

D=\min(T_{\rm select},1000)

delayed targets and selects the same candidate throughout this sweep. Best entries are bolded and second-best entries are underlined.

Dataset	Metric	FP	Memory 500	ESN Direct 500	Direct 500
ETTh1	1 h	$0.695\pm 0.00056$	$0.597\pm 0.00024$	$0.731\pm 0.00048$	$0.724\pm 0.00068$
	6 h	$0.587\pm 0.00012$	$0.576\pm 0.00024$	$0.586\pm 0.00156$	$0.589\pm 0.00044$
	12 h	$0.386\pm 0.00102$	$0.392\pm 0.00009$	$0.374\pm 0.00213$	$0.387\pm 0.00100$
ETTh2	1 h	$0.836\pm 0.00022$	$0.583\pm 0.00010$	$0.835\pm 0.00063$	$0.836\pm 0.00022$
	6 h	$0.620\pm 0.00135$	$0.509\pm 0.00005$	$0.609\pm 0.00165$	$0.620\pm 0.00135$
	12 h	$0.537\pm 0.00027$	$0.483\pm 0.00013$	$0.542\pm 0.00144$	$0.537\pm 0.00027$

Appendix

TABLE IX: Horizon-level forecasting results at

n=20\,000

. Scores are

1-\operatorname{NRMSE}

; higher is better. ETTh is sampled hourly, so horizon steps coincide with physical hours; ETTm is sampled every 15 minutes, and horizons are given in steps with the physical horizon in parentheses. ETT deviations are over three deployment seeds for deterministic selectors; Telco deviations are across ten cells. Telco selection optimizes the aggregate over horizons 1–12; the table shows three representative horizons, so their mean differs from the aggregate scores of Table IV of the main paper.

Reservoir Computing

Reservoir computing uses recurrent dynamical systems to transform temporal inputs into features that support efficient learning with simple readouts.

See all 107 papers

Oct 19, 2026

Sector-Ring Linear Reservoirs: Sector-Wise Decay-Timescale Allocation With State Reuse Across Candidate Settings

In reservoir computing, performance depends on whether the fixed dynamics of the reservoir suit the task, and the appropriate memory length and frequency response differ across tasks. The reservoir settings are found by a search on each task, and driving the reservoir with the input and collecting its states for every candidate setting dominates the cost of the search. In a reservoir with a linear recurrence, the dynamics decompose into modes, one per eigenvalue, and the eigenvalue distribution becomes a design variable. We propose the Sector-Ring Linear Reservoir (SRLR), in which the eigenvalue angles are fixed at evenly spaced positions around a circle and one decay timescale, chosen from a few values, is assigned to each sector of consecutive angles. To keep the scale of each mode's state equal across candidates, the input weights are scaled using the power spectrum of the training input. To reduce the cost of the search, the states of the modes for each decay-timescale value are computed once and reused across candidates. On prediction and reconstruction benchmarks built from chaotic systems, the SRLR achieved, in all eight cases, a lower mean test error than a reference model in which all modes share one decay timescale, and was competitive with nonlinear echo state network and leaky-integrator echo state network baselines. The selected decay-timescale profiles changed with the prediction target and horizon. The state reuse reduced the number of computed mode trajectories in the search by 99.8% relative to per-candidate recomputation.

Yudai Ebato, Sou Nobukawa, Yusuke Sakemi +3

Graduate School of Information and Computer Science, Chiba Institute of Technology, Narashino, Chiba 275-0016, Japan · Department of Computer Science, Chiba Institute of Technology, Narashino, Chiba 275-0016, Japan · Research Center for Mathematical Engineering, Chiba Institute of Technology, Narashino, Chiba 275-0016, Japan +4

Reservoir Computing Linear RNNs

Ask PDF

Oct 3, 2026

A KKL Observer Perspective on Reservoir Computing

Reservoir computing (RC) is a machine learning technique for data-driven modeling of dynamics for forecasting and control, primarily studied in computer science and physics literature with promising applications in neural network learning, physical computing, and neuroscience. Why reservoirs learn and how to choose good reservoir architectures are considered important open questions. We show that the RC problem is mathematically an extension of a classic problem in systems and control theory, the Kazantzis-Kravaris-Luenberger (KKL) observer design problem. As a consequence, many of the questions considered open for RC stand to benefit from a large body of theory in the mature KKL literature, non-exhaustively including on questions of embedding, transverse stability, local and global uniqueness guarantees, and effective data-driven solution constructions. Elaborating on this connection, we show that the surprising forecasting ability of reservoirs is in fact a direct consequence of the well-known observer internal model principle, derive an upper bound on the prediction error over a fixed forecast horizon, and provide a partial explanation for why linear readout training in RC works reasonably well. This work illustrates how classical ideas from systems and control can provide strong theoretical backing and open new questions for modern machine learning methods.

Anastasia Bizyaeva, Fernando Castaños, Jaime A. Moreno

Sibley School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, NY, USA · Department of Automatic Control, Cinvestav, Mexico · Instituto de Ingenier´ıa, Universidad Nacional Aut´onoma de M´exico, Mexico

Reservoir Computing Dynamical Systems

Ask PDF

Jul 27, 2026

Frequency-Based Reservoir computing

Reservoir computing has emerged as an efficient machine learning framework for predicting time series generated by dynamical systems. In contrast to other machine and deep learning approaches, a reservoir computing trains only the output layer via linear regression, leaving the reservoir (recurrent layer) untrained. This simplification makes reservoir computers easier to train and more amenable to experimentation. However, because current reservoirs consist of networks of randomly connected nodes and require the optimization of numerous hyperparameters, a framework that precisely explains how reservoir computing operates and how it can be optimized remains missing. Here, we propose a frequency-based reservoir inspired by the brain's oscillatory dynamics and its hierarchy of timescales. The frequency-based reservoir can be interpreted as an ensemble of independent oscillatory units, each processing a portion of the input's frequency content. This allows us to understand the reservoir's internal behavior by modeling it as a single unit driven by an external input. Borrowing from the theory of a nonlinear oscillator forced by complex periodic inputs, we found that units of the frequency-based reservoir selectively amplify and store specific input frequencies, which are then used for prediction. The frequency-based reservoir performs as well as or better than equivalent random reservoirs. Furthermore, the frequency-based approach can be optimized to improve short-term prediction, a property that random reservoirs lack. Finally, we show that the frequency-based reservoir can also predict complex spatiotemporal dynamics. Our results show that reservoir computing can be designed using brain properties and theoretical insights borrowed from the physics of forced nonlinear oscillators.

Arthur S Powanwe

Department of Mathematics, Western University, London, ON, Canada

Frequency-Domain Feature Learning Reservoir Computing Spatiotemporal Forecasting

Ask PDF

Abstract

Reservoir Computing

Explore similar work

Sector-Ring Linear Reservoirs: Sector-Wise Decay-Timescale Allocation With State Reuse Across Candidate Settings

A KKL Observer Perspective on Reservoir Computing

Frequency-Based Reservoir computing