Operator Learning for Smoothing and Forecasting

Edoardo Calvello; Elizabeth Carlson; Nikola Kovachki; Michael N. Manta; Andrew M. Stuart

stat.MLMar 20, 2026

Operator Learning for Smoothing and Forecasting

Authors: Edoardo Calvello, Elizabeth Carlson, Nikola Kovachki, Michael N. Manta, Andrew M. Stuart

Organizations: California Institute of Technology, Pasadena, CA · Oregon State University, Corvallis, OR · NVIDIA Corporation, Santa Clara, CA · Stanford University, Stanford, CA

Abstract

Machine learning has opened new frontiers in purely data-driven algorithms for data assimilation in, and for forecasting of, dynamical systems; the resulting methods are showing some promise. However, in contrast to model-driven algorithms, analysis of these data-driven methods is poorly developed. In this paper we address this issue, developing a theory to underpin data-driven methods to solve smoothing problems arising in data assimilation and forecasting problems. The theoretical framework relies on two key components: (i) establishing the existence of the mapping to be learned; (ii) the properties of the operator learning architecture used to approximate this mapping. By studying these two components in conjunction, we establish novel universal approximation theorems for purely data driven algorithms for both smoothing and forecasting of dynamical systems. We work in the continuous time setting, hence deploying neural operator architectures. The theoretical results are illustrated with experiments studying the Lorenz 63, Lorenz 96 and Kuramoto-Sivashinsky dynamical systems.

Figures & tables

	Model-driven	Data-driven
Algorithm	Uses dynamical physics model	Uses data from dynamical model
		and/or direct observations
Limitation	Possibly expensive to evaluate and	Requires existence of direct mapping,
	may miss some dynamics due to	a sufficiently dense training dataset and may not
	unmodeled physics	enforce explicit physical constraints
Advantage	Interpretable and output satisfies	Possibly cheap to evaluate and

Table 1: Comparison between model-driven and data-driven approaches.

System	Avg. Rel. $L_{2}$	Med. Rel. $L_{2}$	Std. Rel. $L_{2}$	Min. Rel. $L_{2}$	Max. Rel. $L_{2}$
L63	0.012426	0.012402	0.001612	0.008761	0.017275
L96	0.047156	0.044635	0.014569	0.025633	0.258743
KSE	0.009433	0.009322	0.000755	0.007678	0.013157

Table 2: Smoothing performance on Lorenz ‘63 (L63), Lorenz ‘96 (L96), and the Kuramoto–Sivashinsky equation (KSE).

System	Avg. Rel. $L_{2}$	Med. Rel. $L_{2}$	Std. Rel. $L_{2}$	Rel. Improvement (%)
L63	0.050	0.033	0.100	95.53
L96	0.033	0.032	0.004	94.38
KSE	0.046	0.045	0.007	82.68

Table 3: Forecasting performance on Lorenz ‘63 (L63), Lorenz ‘96 (L96), and the Kuramoto–Sivashinsky equation (KSE). In addition to the relative

L^{2}

errors achieved on the test set, we also report the relative improvement of

L^{2}

errors achieved with the transformer neural operator compared to a constant forecast equal to the last input state.

Category	Smoothing	Forecasting
Parameters	$\sigma=10,\ \rho=28,\ \beta=\tfrac{8}{3}$
Observed	$x\in C\bigl([0,5];\mathbb{R}\bigr)$	$x\in C\bigl([0,2];\mathbb{R}\bigr)$
Unobserved	$(y,z)\in C\bigl([0,5];\mathbb{R}^{2}\bigr)$	$(y,z)\in C\bigl([2,4];\mathbb{R}^{2}\bigr)$
Observation time step	$\Delta t=0.02$	$\Delta t=0.01$

Table 4: Lorenz ‘63 system settings

Figure 1: Lorenz ‘63. Median and worst-case relative $L^{2}$ error samples for smoothing experiment involving prediction of $(y,z)$ from $x$ .

Figure 2: Lorenz ‘63. Median and worst-case relative $L^{2}$ error samples for smoothing experiment involving prediction of $(x,y)$ from $z$ .

Panel of Figure 3 — Figure 3: Lorenz ‘63. Left: median and worst-case relative $L^{2}$ error samples in forecasting. Right: distribution of trajectory predictions under composition of the learned forecasting map; the dark shade of red represents overlap between the blue ground truth distribution and the light red predicted distribution. The prevalence of the dark shade of red thus indicates matching of distributions.

Category	Smoothing	Forecasting
Parameters	$F=8$
Observed	$(u_{1},\ldots,u_{21},u_{23},u_{25},\ldots,u_{39})\in C([0,5];\mathbb{R}^{30})$	$(u_{1},u_{2},u_{3},u_{5},\ldots,u_{39})\in C([0,5];\mathbb{R}^{30})$
Unobserved	$(u_{22},u_{24},\ldots,u_{40})\in C([0,5];\mathbb{R}^{10})$	$(u_{1},u_{2},u_{3},u_{5},\ldots,u_{39})\in C([5,5.2];\mathbb{R}^{30})$
Obs. time	$\Delta t=0.02$

Table 5: Lorenz ‘96 system settings

Figure 4: Lorenz ‘96. Median and worst-case performance on the Lorenz ‘96 system for smoothing.

Figure 5: Lorenz ‘96. Pointwise errors relative to the root mean square of the ground truth (computed in space) for the sample in the test set achieving the median relative $L^{2}$ error.

Panel of Figure 6 — Figure 6: Lorenz ‘96. Left: forecast obtained by composing the learned map on one-step median and worst-case relative $L^{2}$ error samples. Right: distribution of trajectory predictions under composition of the learned forecasting map; the dark shade of red represents overlap between the blue ground truth distribution and the light red predicted distribution. The prevalence of the dark shade of red thus indicates matching of distributions.

Category	Smoothing	Forecasting
Parameters	$L=32\pi$
Spatial Grid	$x_{j}=jL/128,\hskip 8.50012ptj=1,\ldots,128$
Observed	$\bigl(\widetilde{u}(x_{1}),\ldots,\widetilde{u}(x_{128})\bigr)\!\in C([0,100];\mathbb{R}^{128})$	$\bigl(\widetilde{u}(x_{1}),\ldots,\widetilde{u}(x_{128})\bigr)\!\in C([0,100];\mathbb{R}^{128})$
Unobserved	$\bigl(u(x_{1}),\ldots,u(x_{128})\bigr)\!\in C([0,100];\mathbb{R}^{128})$	$\bigl(\widetilde{u}(x_{1}),\ldots,\widetilde{u}(x_{128})\bigr)\!\in C([100,102];\mathbb{R}^{128})$
Obs. time	$\Delta t=0.25$

Table 6: Kuramoto–Sivashinsky (KS) system settings. We recall that

u

denotes the solution, while

\widetilde{u}

is obtained by retaining the first

64

modes of

u

and zeroing out the rest.

Figure 7: Median and worst-case performance on the KS system for smoothing.

Figure 8: KS. Pointwise errors relative to the root mean square of the ground truth (computed in space) for the sample in the test set achieving the median relative $L^{2}$ error.

Panel of Figure 9 — Figure 9: KS. Left: forecast obtained by composing the learned map on one-step median and worst-case relative $L^{2}$ error samples. Right: distribution of trajectory predictions under composition of the learned forecasting map; the dark shade of red represents overlap between the blue ground truth distribution and the light red predicted distribution. The prevalence of the dark shade of red thus indicates matching of distributions.

Data Assimilation

Data assimilation combines observations with dynamical models to estimate hidden system states and their uncertainty over time.

See all 58 papers

Adversarial dynamical systems characterize when data-driven learning succeeds or fails
Jul 8, 2024Matthew J. Colbrook, Igor Mezić, Alexei StepanenkoKoopman Operator Learning Spectral Methods
DAMTP, University of Cambridge, Cambridge, CB3 0WA, UK. · University of California, Santa Barbara, CA 93106, USA.
A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems
Mar 16, 2026Jingqiao Tang, Ryan Bausback, Feng Bao +2Data Assimilation Scientific ML
PR-Smoother: Simulator-Preserving Non-Gaussian Smoothing for Data Assimilation
Sep 22, 2026Yuta TarumiAmortized Inference Data Assimilation
iTHEMS RIKEN Kobe, Japan

Jul 8, 2024J/K move · Enter open · S save

Adversarial dynamical systems characterize when data-driven learning succeeds or fails

Matthew J. Colbrook, Igor Mezić, Alexei Stepanenko

DAMTP, University of Cambridge, Cambridge, CB3 0WA, UK. · University of California, Santa Barbara, CA 93106, USA.

Koopman Operator Learning Spectral Methods Statistical Learning Theory Chaotic Time-Series Prediction

Open paper AskPDF arXiv

Many systems resist analytical modeling, making data-driven inference of dynamics important. Yet data-driven methods can fail to converge or generalize, leaving open a central question: When can system behavior be learned reliably from data, and when is such learning impossible? We answer this question using adversarial dynamical systems to identify the boundary between accessible and inaccessible regimes. In Koopman operator learning, a leading framework for representing nonlinear dynamics through linear spectral objects, we design optimal data-driven spectral algorithms with convergence and certification guarantees under conditions arising broadly in physical systems. This yields a convergence theory for Koopman-operator approximations and resolves a longstanding open problem in Koopman spectral analysis. Conversely, by constructing adversarial systems, we prove matching impossibility results: without these conditions, no single-sequence limiting procedure can guarantee learning, regardless of data quality. These results sharply characterize when data-driven spectral learning can succeed and when it must fail. We validate the framework on oscillators, chaotic fluid flows and Arctic sea ice concentration forecasting. In the latter, we uncover hidden modes of Arctic sea ice decline, deliver long-range forecasts with geographic error bounds, and outperform state-of-the-art dynamical and deep learning models at substantially lower computational cost, enabling real-time deployment on standard CPUs.

Abstract

Data Assimilation

Explore similar work

Adversarial dynamical systems characterize when data-driven learning succeeds or fails

A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems

PR-Smoother: Simulator-Preserving Non-Gaussian Smoothing for Data Assimilation