Gradient-free online learning of subgrid-scale dynamics with neural emulators

Hugo Frezat; Ronan Fablet; Guillaume Balarac; Julien Le Sommer

physics.comp-phOct 30, 2023

Gradient-free online learning of subgrid-scale dynamics with neural emulators

Authors: Hugo Frezat, Ronan Fablet, Guillaume Balarac, Julien Le Sommer

Organizations: Univ. Grenoble-Alpes, CNRS, Grenoble INP, LEGI, Grenoble, France · Univ. Grenoble-Alpes, CNRS, INRAE, IRD, Grenoble INP, IGE, Grenoble, France · IMT Atlantique, CNRS, Lab-STICC, Brest, France · Inria, Odyssey team, Brest, France ·

Abstract

In this paper, we propose a generic algorithm to train machine learning-based subgrid parametrizations online, i.e., with $\textit{a posteriori}$ loss functions, but for non-differentiable numerical solvers. The proposed approach leverages a neural emulator to approximate the reduced state-space solver, which is then used to allow gradient propagation through temporal integration steps. We apply this methodology on a chaotic two-timescales Lorenz-96 system and a single layer quasi-geostrophic system with zonal dynamics, known to be highly unstable with offline strategies. Using our algorithm, we are able to train a parametrization that recovers most of the benefits of online strategies without having to compute the gradient of the original solver. We found that training the neural emulator and parametrization components separately with different loss quantities is necessary in order to minimize the propagation of approximation biases. Experiments on emulator architectures with different complexities also indicates that emulator performance is key in order to learn an accurate parametrization. This work is a step towards learning parametrization with online strategies for climate models.

Figures & tables

Name	Symbol	Value
Simulation parameters
	$b$	$10$
	$c$	$1$
	$h$	$1$
Forcing	$F$	$18$
Slow variables	$K$	$8$

Table 1: Simulation (top) and learning (bottom) parameters used for the L96 demonstration.

Figure 1: Evolution of the deviation (based on the root mean squared error) between reference trajectory

X_{k}

and the perturbed ensemble

\delta X_{k}

for the L96 system (left). The dashed lines shows the approximate decorrelation time

t_{c}

. Evolution of the mean state

\langle X_{k}\rangle_{k}

throughout the 25 trajectories (right) for the reference solver

f

(blue), coarse solver

g

with restarts at each sub-trajectory (indicated by a circle), and neural emulator

\mathcal{E}

trained to reproduce the dynamics of

g

Figure 2: State-averaged cumulative error

E(t)

(left) and Wasserstein distance

W_{1}(X_{k}\sim P,\hat{X}_{k}\sim Q)

(right) for 100 decorrelation times

t_{c}

(or 30,000 iterations with coarse solver

g

) for the L96 system.

Figure 3: Evolution of the deviation (based on the root mean squared error) between reference trajectory $\omega(t)$ and the perturbed ensemble $\delta\omega(t)$ for the QG system (left). The dashed lines shows the approximate decorrelation time $t_{c}$ . Vorticity field $\omega$ (center) and filtered vorticity field $\bar{\omega}$ (right) at the end of the spin-up.

Name	Symbol	Value
Simulation parameters
Length of the domain	$L$	$2\pi$
Linear drag	$\mu$	$2\times 10^{-2}$
Kinematic viscosity	$\nu$	$10^{-5}$
Rossby parameter	$\beta$	30
Forcing amplitude	$\sigma$	10

Table 2: Parameters of the numerical setup for simulation (top) and learning (bottom). Note that coarse-resolution systems use the same parameters, except for grid resolution

\bar{N}=N/\Delta^{\prime}

, and time step

\overline{\delta t}=\Delta^{\prime}\delta t

. The quantities are given in numerical (dimensionless) as directly used in the solver for reproducibility.

Figure 4: A priori evaluation of the different models used in this study. The test set is composed of 2,500 filtered samples from a DNS performed after spin-up, i.e., prolonging the initial trajectory. Averaged

pdf

of the SGS term

\tau_{\omega}

(left) and energy fluxes due to the SGS term

\partial_{t}\partial E(k)_{\text{sgs}}

(right).

Figure 5: Evolution of kinetic energy

E(t)

(left) and enstrophy

Z(t)

(right) for the different models used in this study. Simulations are integrated for almost

300

turnover times

t_{L}

Figure 6: Vorticity fields at the end of the a posteriori evaluation ( $t=240$ ) for the DNS, filtered DNS and the models in this study. Note that $\tau\equiv\mathcal{M}_{\text{on}}^{\text{state}}(\text{sm})$ was unstable and is thus shown as empty. The offline model $\tau\equiv\mathcal{M}_{\text{off}}$ was also omitted from this plot because it crashed during the simulation.

Figure 7: Kinetic energy and enstrophy spectra

E(k)

(left) and

Z(k)

(right) averaged over 2,500 samples on the a posteriori evaluation horizon

t\in[0,240]

for the different models used in this study. The forcing wavenumber

k_{f}=15

is shown in dashed gray.

Figure 8: Probability distribution of vorticity

\omega

and kinetic energy

E(x,y)=1/2(u_{x}^{2}+u_{y}^{2})

at the end of the a posteriori evaluation at

t=240

for the different models used in this study.

	$\mathcal{S}\{E\}$	$\mathcal{S}\{Z\}$	$\mathcal{S}\{E(k)\}$	$\mathcal{S}\{Z(k)\}$	$\mathcal{S}\{pdf(\omega)\}$	$\mathcal{S}\{pdf(E)\}$
$\mathcal{M}_{\text{on}}^{\text{subgrid}}(\text{sm})$	$-6.586$	$0.309$	$-3.747$	$-0.170$	$-0.381$	$-4.064$
$\mathcal{M}_{\text{on}}^{\text{state}}(\text{lg})$	$-14.240$	$0.180$	$-8.170$	$-0.030$	$-0.380$	$-15.569$
$\mathcal{M}_{\text{on}}^{\text{subgrid}}(\text{lg})$	$-2.623$	$0.942$	$-1.048$	$0.537$	$0.592$	$-1.808$
$\mathcal{M}_{\text{on}}^{\text{state}}(\text{ref})$	$\mathbf{0.753}$	$\mathbf{0.965}$	$\mathbf{0.290}$	$\mathbf{0.790}$	$\mathbf{0.821}$	$\mathbf{0.550}$
$\mathcal{M}_{\text{on}}^{\text{subgrid}}(\text{ref})$	$-1.333$	$\mathbf{0.966}$	$-0.961$	$0.543$	$\mathbf{0.829}$	$-1.348$

Table 3: Similarity measures on the metrics used in the a posteriori evaluation. Note that the metric is systematically calculated against the DNS with respect to the under-resolved simulation

\tau\equiv 0

Climate Modeling

Climate modeling uses physical simulations and data-driven methods to understand, predict, and project Earth’s climate and its response to external forces.

See all 72 papers

Subgrid-Scale Parameterization in Burgers' Equation Using Structure-Preserving Neural Networks and Entropy Variables
Jul 16, 2026Aijaz Nazir, Ilya TimofeyevNeural Surrogate Modeling PDE Solving
Dept. of Mathematics, University of Houston, Houston, TX 77204
Learning the Entropic Relaxation: Stable Coarse-Grid Emulation with Neural Discrete Equilibrium
Nov 2, 2026Jose Antonio Lara Benitez, Kareem Hegazy, Anastasis Kratsios +3PDE Surrogate Modeling Neural PDE Solvers
Rice University and LBNL · ICSI and University of California at Berkeley · McMaster University and the Vector Institute +3
Learning Prognostic Variables for AI Convective Parameterizations via Symbolic Distillation
Sep 21, 2026Jurij Schönfeld, Tom Beucler, Julien Savre +2Latent Variable Models Climate Modeling
Deutsches Zentrum für Luft- und Raumfahrt, Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany · University of Bremen, Institute of Environmental Physics (IUP), Bremen, Germany · Faculty of Geosciences and Environment, University of Lausanne, Lausanne, Switzerland +3

Jul 16, 2026J/K move · Enter open · S save

Subgrid-Scale Parameterization in Burgers' Equation Using Structure-Preserving Neural Networks and Entropy Variables

Aijaz Nazir, Ilya Timofeyev

Dept. of Mathematics, University of Houston, Houston, TX 77204

Neural Surrogate Modeling PDE Solving Structure-Preserving Neural Networks Computational Fluid Dynamics

Open paper AskPDF arXiv

We present a machine learning approach for developing subgrid-scale (SGS) parametrizations in coarse simulations of partial differential equations. We utilize structure-preserving neural networks and entropy variables to learn subgrid fluxes in coarse simulations of the Burgers' equation. In particular, we employ a decoupled neural network architecture explicitly separating the subgrid corrections into two distinct components: a conservative Flux Potential network and an Eddy Viscosity network. We demonstrate that this reduced-order framework maintains high physical fidelity, accurately reproducing the energy spectrum, spatial and temporal correlation functions, and dynamical characteristics of the full-scale system. Furthermore, we show that our approach is robust and applicable to parameters outside the training regime.

Abstract

Climate Modeling

Explore similar work

Subgrid-Scale Parameterization in Burgers' Equation Using Structure-Preserving Neural Networks and Entropy Variables

Learning the Entropic Relaxation: Stable Coarse-Grid Emulation with Neural Discrete Equilibrium

Learning Prognostic Variables for AI Convective Parameterizations via Symbolic Distillation