Washout-free synchronization, parameter estimation and uncertainty quantification of echo state networks
Organizations: Aeronautics Department, Imperial College London, Exhibition Road, London, SW7 2AZ, UK · I-X AI in Science Centre, Imperial College London, White City Campus, London, W12 0BZ, UK
Abstract
Echo state networks (ESNs) forecast chaotic systems for several Lyapunov times. Like all recurrent networks, ESNs require a washout before forecasting, i.e., a phase in which the full state drives the network until the reservoir state synchronizes with the system. At deployment, however, sensors measure only part of the state, with noise and at a lower sampling rate than the training data. We propose a washout-free deployment of ESNs via ensemble data assimilation. In closed loop, the ESN is a Markovian dynamical system; hence, a sequential filter can correct its reservoir state whenever data are available. The filter synchronizes an ensemble of reservoir states, drawn from the training data, with the system, and quantifies the forecast uncertainty. We extend the method to parametric ESNs, to infer physical parameters, and to parallel ESNs with localization, to forecast high-dimensional systems. We test the method on the Lorenz-63 and Lorenz-96 systems. First, the filter synchronizes the ESN with a fraction of the data of a washout. Second, the filter deploys one full-state ESN from a subset of its components, which a washout cannot do. Third, the filter estimates a forcing that is not in the training data, and the estimated forcing lengthens the forecast. Fourth, localization synchronizes a parallel ESN with eight times fewer members than the global filter. Fifth, the ensemble spread signals, without the true state, when the ESN synchronizes and when the filter fails. This work enables the deployment of ESNs from partial, sparse and noisy observations.
Figures & tables
| Lorenz-96 | |||
| Description | Lorenz-63 | ||
| Time step of the record, | |||
| Noise in the data | |||
| ESN time step, ; steps per | ; | ; | ; |
| Upsampling, | |||
| Observation interval, | ; | ; | ; |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Lorenz-63 | Lorenz-96, | Lorenz-96, | |||||
| single | single | parallel | single, parametric | single | parallel | ||
| Description | Symbol | L63 | L96-10s | L96-10p | parametric L96-10s | L96-40s | L96-40p |
| Training regime | , , , , , , , , , , , , } | ||||||
| Training length (steps of ; ) | ; | ; | ; | per value of | ; | ; | |
| Reservoirs units | |||||||
| Sites per patch; halo | – | – | ; | – | – | ; | |
Chaotic Time-Series Prediction
Chaotic time-series prediction studies how to forecast nonlinear systems whose sensitive dependence on initial conditions limits long-range accuracy.
See all 59 papers