Reservoir observer enhanced with residual calibration and attention mechanism

Yichen Liu; Wei Xiao; Tianguang Chu

cs.LGApr 1, 2026

Reservoir observer enhanced with residual calibration and attention mechanism

Authors: Yichen Liu, Wei Xiao, Tianguang Chu

Organizations: School of Advanced Manufacturing and Robotics, Peking University, Beijing 100871, China

Abstract

Reservoir observers provide a data-driven approach to the inference of unmeasured variables from observed ones for nonlinear dynamical systems. While previous studies have demonstrated wide applicability, their performance may vary considerably with different input variables, even compromising reliability in the worst cases. To enhance the performance of inference, we integrate residual calibration and attention mechanism into the reservoir observer design. The residual calibration module leverages information from the estimation residuals to refine the observer output, and the attention mechanism exploits the temporal dependencies of the data to enrich the representation of reservoir internal dynamics. Experiments on typical chaotic systems demonstrate that our method substantially improves inference accuracy, especially for the worst cases resulting from the traditional reservoir observers. We also invoke the notion of transfer entropy to explain the reason for the input-dependent observation discrepancy and the effectiveness of the proposed method.

Figures & tables

Figure 1: Schematic diagrams of (a) the traditional reservoir observer and (b) the proposed RORA.

Figure 2: Schematic diagram of the residual calibration approach.

Figure 3: Correlation matrices of the elements of (a) the reservoir state and (b) the corresponding principal components obtained via SVD for the input variable

x

in the Rössler system.

System	$T$	$\Delta t$	$d$	$D$	$\alpha$	$\xi$	$\rho$	$\gamma$	$\beta$	$\lambda$	$\sigma$	$N_{c}$
Rössler	400	0.1	400	0.05	1.0	1.0	1.0	1.0	$10^{-8}$	0.9	1.0	50
Lorenz	800	0.05	400	0.05	1.0	1.0	1.0	1.0	$10^{-8}$	0.9	1.0	50
Chua’s circuit	1000	0.1	400	0.05	1.0	1.0	1.0	1.0	$10^{-8}$	0.5	1.0	50
Kuramoto-Sivashinsky	30,000	0.25	1,000	0.06	0.5	0.0	0.9	0.5	$10^{-10}$	0.95	1.0	50

Table 1: Hyperparameters for the systems in experiments.

Method	Input $x(t)$		Input $y(t)$		Input $z(t)$
Method	$y$	$z$	$x$	$z$	$x$	$y$
RO	$8.12\times 10^{-4}$	$7.51\times 10^{-4}$	$5.74\times 10^{-4}$	$5.99\times 10^{-2}$	$14.56$	$76.44$
ROR	$2.71\times 10^{-4}$	$1.94\times 10^{-4}$	$2.85\times 10^{-4}$	$2.99\times 10^{-2}$	$6.33$	$27.68$
ROR	( $66.63\%$ )	( $74.17\%$ )	( $50.35\%$ )	( $50.08\%$ )	( $56.52\%$ )	( $63.79\%$ )
ROA	$2.88\times 10^{-4}$	$1.79\times 10^{-4}$	$2.71\times 10^{-4}$	$1.84\times 10^{-2}$	$0.67$	$8.28$
ROA	( $64.53\%$ )	( $76.17\%$ )	( $52.79\%$ )	( $69.28\%$ )	( $95.40\%$ )	( $89.17\%$ )

Table 2: MSE values and percentage reductions compared to RO for different methods on the Rössler system. Results are grouped by different input variables and the corresponding inferred outputs.

Method	Input $x(t)$		Input $y(t)$		Input $z(t)$
Method	$y$	$z$	$x$	$z$	$x^{2}$	$y^{2}$
RO	$5.04\times 10^{-3}$	$3.01\times 10^{-2}$	$1.52\times 10^{-6}$	$4.11\times 10^{-2}$	$9.82\times 10^{-7}$	$1.35\times 10^{-4}$
ROR	$2.45\times 10^{-3}$	$1.31\times 10^{-2}$	$4.68\times 10^{-7}$	$1.24\times 10^{-2}$	$3.77\times 10^{-7}$	$6.99\times 10^{-5}$
ROR	( $51.39\%$ )	( $56.48\%$ )	( $69.21\%$ )	( $69.83\%$ )	( $61.61\%$ )	( $48.22\%$ )
ROA	$8.11\times 10^{-4}$	$9.64\times 10^{-3}$	$7.57\times 10^{-7}$	$1.05\times 10^{-2}$	$5.59\times 10^{-8}$	$6.29\times 10^{-6}$
ROA	( $83.91\%$ )	( $67.97\%$ )	( $50.20\%$ )	( $74.45\%$ )	( $94.31\%$ )	( $95.34\%$ )

Table 3: MSE values and percentage reductions compared to RO for different methods on the Lorenz system. Results are grouped by different input variables and the corresponding inferred outputs.

Figure 4: Inference of variables in the Rössler system using RORA.

Figure 5: MSE values of RO (blue) and RORA (red) versus the reservoir hyperparameters. In each case, one hyperparameter is varied while all others are held fixed as given in the previous experiments.

Method	Input $x(t)$		Input $y(t)$		Input $z(t)$
Method	$y$	$z$	$x$	$z$	$x$	$y$
RO	$6.25\times 10^{-5}$	$2.14\times 10^{-3}$	68.31	70.82	$6.72\times 10^{-3}$	$2.47\times 10^{-5}$
ROR	$9.40\times 10^{-6}$	$2.51\times 10^{-4}$	21.25	23.60	$2.25\times 10^{-3}$	$8.21\times 10^{-6}$
ROR	( $84.96\%$ )	( $88.27\%$ )	( $68.89\%$ )	( $66.68\%$ )	( $66.52\%$ )	( $66.76\%$ )
ROA	$2.25\times 10^{-5}$	$5.42\times 10^{-4}$	9.59	10.66	$1.39\times 10^{-3}$	$8.11\times 10^{-6}$
ROA	( $64.00\%$ )	( $74.67\%$ )	( $85.96\%$ )	( $84.95\%$ )	( $79.32\%$ )	( $67.17\%$ )

Table 4: MSE values and percentage reductions compared to RO for different methods on the Chua’s circuit system. Results are grouped by different input variables and the corresponding inferred outputs.

Figure 6: Inference of variables in the Lorenz system using RORA.

Figure 7: Inference of variables in the Chua’s circuit system using RORA.

Initialization setting	Method	$n=2$		$n=4$		$n=6$		$n=8$
Initialization setting	Method	MSE	$\downarrow$	MSE	$\downarrow$	MSE	$\downarrow$	MSE	$\downarrow$
$\mathrm{IS}_{1}$	RO	1.0716		0.5294		0.1533		0.0385
	ROR	0.6521	$39.15\%$	0.3691	$30.28\%$	0.1051	$31.44\%$	0.0129	$66.49\%$
	ROA	0.6377	$40.50\%$	0.2414	$54.41\%$	0.0488	$68.19\%$	0.0059	$84.68\%$
	RORA	0.4757	$55.62\%$	0.1449	$72.64\%$	0.0385	$74.90\%$	0.0013	$96.62\%$
$\mathrm{IS}_{20}$	RO	1.2025		0.5682		0.1767		0.0439

Table 5: MSE values and percentage reductions (

\downarrow

) compared to RO for different methods on the Kuramoto-Sivashinsky equation. In the setting

\mathrm{IS}_{q}

, the data used in each of 100 runs are randomly selected from the datasets corresponding to

q

initial conditions.

Figure 8: Inference of the Kuramoto-Sivashinsky equation using RO and RORA. (a) Simulated data from a random initialization. (b–e) Inference results for cases with the number of input variables $n=2,4,6,$ and $8$ , respectively. In each case, the four panels (from top to bottom) show the RO inference, the RORA inference, and their respective differences from the simulated data. Black arrows to the right of the panels indicate the input locations.

Figure 9: MSE values of

R_{\mathrm{basic}}

and the percentage reduction of MSE brought by ROR versus the noise level

\eta

. (a) MSE values against the noise-free sequences under varying levels of measurement noise. (b) Percentage reduction in MSE achieved by ROR compared to RO in the presence of noise.

Figure 10: Transfer entropy between variables in three chaotic systems: (a) the Rössler system, (b) the Lorenz system, and (c) the Chua’s circuit system.

Method	Statistics	$x\to y$		$y\to z$		$z\to x$
Method	Statistics	$\left[t_{T/2},t_{T}\right]$	$\left[t_{T},t_{11T}\right]$	$\left[t_{T/2},t_{T}\right]$	$\left[t_{T},t_{11T}\right]$	$\left[t_{T/2},t_{T}\right]$	$\left[t_{T},t_{11T}\right]$
ROR	MAV	$9.23\times 10^{-5}$	$4.35\times 10^{-4}$	$1.73\times 10^{-4}$	$5.63\times 10^{-4}$	$9.70\times 10^{-4}$	$2.78\times 10^{-3}$
ROR	Variance	$4.55\times 10^{-8}$	$2.19\times 10^{-7}$	$4.68\times 10^{-7}$	$2.24\times 10^{-6}$	$1.03\times 10^{-5}$	$3.85\times 10^{-4}$
		$\left[t_{1},t_{T}\right]$	$\left[t_{T},t_{11T}\right]$	$\left[t_{1},t_{T}\right]$	$\left[t_{T},t_{11T}\right]$	$\left[t_{1},t_{T}\right]$	$\left[t_{T},t_{11T}\right]$
ROR-al	MAV	$6.87\times 10^{-5}$	$3.25\times 10^{-4}$	$8.28\times 10^{-5}$	$5.07\times 10^{-4}$	$7.93\times 10^{-4}$	$2.77\times 10^{-3}$
ROR-al	Variance	$9.71\times 10^{-9}$	$1.67\times 10^{-7}$	$7.35\times 10^{-8}$	$1.45\times 10^{-6}$	$1.75\times 10^{-6}$	$1.26\times 10^{-4}$

Table 6: Statistics of residuals from

R_{\mathrm{basic}}

during different time intervals for the Rössler system. The intervals

\left[t_{T/2},t_{T}\right]

for ROR and

\left[t_{1},t_{T}\right]

for ROR-al specify the time ranges from which residuals are drawn to train

R_{\mathrm{res}}

, and

\left[t_{T},t_{11T}\right]

denotes the common inference interval for both methods. The mean absolute value (MAV) and variance are calculated on the residuals for each setting “input

\rightarrow

target” and averaged over 100 runs.

Method	$x\to y$	$x\to z$	$y\to x$	$y\to z$	$z\to x$	$z\to y$
ROR	$2.71\times 10^{-4}$	$1.94\times 10^{-4}$	$2.85\times 10^{-4}$	$2.99\times 10^{-2}$	6.33	27.68
ROR-al	$3.52\times 10^{-4}$	$3.26\times 10^{-4}$	$3.31\times 10^{-4}$	$3.37\times 10^{-2}$	7.72	33.68

Table 7: MSE values of different methods on the Rössler system. Results are grouped by “input

\rightarrow

target” pairs, and averaged over 100 runs.

Figure 11: MSE values of RO and ROR versus the hyperparameter

\beta

when inferring

y

and

z

with inputting

x

of the Rössler system. MSE values are averaged over 100 runs during the inference stage of 2,000 time steps, with the same hyperparameters except

\beta

. For ROR, both reservoirs share the same value of

\beta

. The shaded area indicates the well-regularized region where

\beta

yields near-optimal performance. Beyond this region, the MSE values of RO and ROR vary over a wide range.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Schematic diagram of of an alternative residual-calibration scheme ROR-al.

Method	System	Input $x(t)$		Input $y(t)$		Input $z(t)$
Method	System	$y$	$z$	$x$	$z$	$x^{e}$	$y^{e}$
RO- $2d$	Rössler	$6.44\times 10^{-4}$	$6.65\times 10^{-4}$	$4.87\times 10^{-4}$	$5.46\times 10^{-2}$	12.22	60.11
	Lorenz	$4.94\times 10^{-3}$	$2.88\times 10^{-2}$	$9.23\times 10^{-7}$	$4.00\times 10^{-2}$	$8.77\times 10^{-7}$	$9.69\times 10^{-5}$
	Chua’s	$4.54\times 10^{-5}$	$1.18\times 10^{-3}$	$61.21$	$63.87$	$5.45\times 10^{-3}$	$1.31\times 10^{-5}$
P-RC	Rössler	$6.06\times 10^{-4}$	$3.92\times 10^{-4}$	$4.79\times 10^{-4}$	$2.57\times 10^{-2}$	$9.15$	$42.63$
	Lorenz	$2.73\times 10^{-3}$	$2.43\times 10^{-2}$	$1.12\times 10^{-6}$	$3.41\times 10^{-2}$	$1.71\times 10^{-7}$	$5.79\times 10^{-5}$

Appendix

Table 8: MSE values of different methods on the three systems. Results are evaluated using the same hyperparameters as in the main text, except for the doubled number of nodes in RO-

2d

, and are grouped by different input variables and the corresponding inferred outputs. For input

z(t)

e=2

for the Lorenz system and

e=1

otherwise.

Figure 13: MSE ratios (normalized by RO) for RO-2

d

, P-RC, and the proposed methods on the concerned systems. (a) Comparison of ROR against RO-2

d

. (b) Comparison of ROA and RORA against P-RC. Roman numerals denote the “input

\rightarrow

target” pairs: I:

x\rightarrow y

, II:

x\rightarrow z

, III:

y\rightarrow x

, IV:

y\rightarrow z

, V:

z\rightarrow x

, and VI:

z\rightarrow y

Reservoir Computing

Reservoir computing uses recurrent dynamical systems to transform temporal inputs into features that support efficient learning with simple readouts.

See all 107 papers

A KKL Observer Perspective on Reservoir Computing
Oct 3, 2026Anastasia Bizyaeva, Fernando Castaños, Jaime A. MorenoReservoir Computing Dynamical Systems
Sibley School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, NY, USA · Department of Automatic Control, Cinvestav, Mexico · Instituto de Ingenier´ıa, Universidad Nacional Aut´onoma de M´exico, Mexico
Inference of Unknown Dynamical Components Using Next Generation Reservoir Computing: From Chaotic Systems to Climate Data
Sep 21, 2026Jule Budnick, Andrew Keane, Serhiy YanchukNonlinear System Identification Reservoir Computing
School of Mathematical Sciences, University College Cork, Cork, Ireland · Potsdam Institute for Climate Impact Research, Potsdam, Germany
Inferring bifurcation diagrams of two distinct chaotic systems by a single machine
Apr 29, 2026Jianmin Guo, Yao Du, Yizhen Yu +2Dynamical Systems Nonlinear System Identification
School of Physics and Information Technology, Shaanxi Normal University, Xi’an 710062, China · College of Physics and Electronic Information Engineering, Qinghai Normal University, Xining 810008, China · School of Physics and Electronic Science, East China Normal University, Shanghai 200062, China

Oct 3, 2026J/K move · Enter open · S save

A KKL Observer Perspective on Reservoir Computing

Anastasia Bizyaeva, Fernando Castaños, Jaime A. Moreno

Sibley School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, NY, USA · Department of Automatic Control, Cinvestav, Mexico · Instituto de Ingenier´ıa, Universidad Nacional Aut´onoma de M´exico, Mexico

Reservoir Computing Dynamical Systems

Open paper AskPDF arXiv

Reservoir computing (RC) is a machine learning technique for data-driven modeling of dynamics for forecasting and control, primarily studied in computer science and physics literature with promising applications in neural network learning, physical computing, and neuroscience. Why reservoirs learn and how to choose good reservoir architectures are considered important open questions. We show that the RC problem is mathematically an extension of a classic problem in systems and control theory, the Kazantzis-Kravaris-Luenberger (KKL) observer design problem. As a consequence, many of the questions considered open for RC stand to benefit from a large body of theory in the mature KKL literature, non-exhaustively including on questions of embedding, transverse stability, local and global uniqueness guarantees, and effective data-driven solution constructions. Elaborating on this connection, we show that the surprising forecasting ability of reservoirs is in fact a direct consequence of the well-known observer internal model principle, derive an upper bound on the prediction error over a fixed forecast horizon, and provide a partial explanation for why linear readout training in RC works reasonably well. This work illustrates how classical ideas from systems and control can provide strong theoretical backing and open new questions for modern machine learning methods.

Abstract

Reservoir Computing

Explore similar work

A KKL Observer Perspective on Reservoir Computing

Inference of Unknown Dynamical Components Using Next Generation Reservoir Computing: From Chaotic Systems to Climate Data

Inferring bifurcation diagrams of two distinct chaotic systems by a single machine