In this paper, we propose a generic algorithm to train machine learning-based subgrid parametrizations online, i.e., with a posteriori loss functions, but for non-differentiable numerical solvers. The proposed approach leverages a neural emulator to approximate the reduced state-space solver, which is then used to allow gradient propagation through temporal integration steps. We apply this methodology on a chaotic two-timescales Lorenz-96 system and a single layer quasi-geostrophic system with zonal dynamics, known to be highly unstable with offline strategies. Using our algorithm, we are able to train a parametrization that recovers most of the benefits of online strategies without having to compute the gradient of the original solver. We found that training the neural emulator and parametrization components separately with different loss quantities is necessary in order to minimize the propagation of approximation biases. Experiments on emulator architectures with different complexities also indicates that emulator performance is key in order to learn an accurate parametrization. This work is a step towards learning parametrization with online strategies for climate models.
Figures & tables
Name
Symbol
Value
Simulation parameters
b
10
c
1
h
1
Forcing
F
18
Slow variables
K
8
Table 1: Simulation (top) and learning (bottom) parameters used for the L96 demonstration.
Figure 1: Evolution of the deviation (based on the root mean squared error) between reference trajectory Xk and the perturbed ensemble δXk for the L96 system (left). The dashed lines shows the approximate decorrelation time tc . Evolution of the mean state ⟨Xk⟩k throughout the 25 trajectories (right) for the reference solver f (blue), coarse solver g with restarts at each sub-trajectory (indicated by a circle), and neural emulator E trained to reproduce the dynamics of g .
Figure 2: State-averaged cumulative error E(t) (left) and Wasserstein distance W1(Xk∼P,X^k∼Q) (right) for 100 decorrelation times tc (or 30,000 iterations with coarse solver g ) for the L96 system.
Figure 3: Evolution of the deviation (based on the root mean squared error) between reference trajectory ω(t) and the perturbed ensemble δω(t) for the QG system (left). The dashed lines shows the approximate decorrelation time tc . Vorticity field ω (center) and filtered vorticity field ωˉ (right) at the end of the spin-up.
Name
Symbol
Value
Simulation parameters
Length of the domain
L
2π
Linear drag
μ
2×10−2
Kinematic viscosity
ν
10−5
Rossby parameter
β
30
Forcing amplitude
σ
10
Table 2: Parameters of the numerical setup for simulation (top) and learning (bottom). Note that coarse-resolution systems use the same parameters, except for grid resolution Nˉ=N/Δ′ , and time step δt=Δ′δt . The quantities are given in numerical (dimensionless) as directly used in the solver for reproducibility.
Figure 4: A priori evaluation of the different models used in this study. The test set is composed of 2,500 filtered samples from a DNS performed after spin-up, i.e., prolonging the initial trajectory. Averaged pdf of the SGS term τω (left) and energy fluxes due to the SGS term ∂t∂E(k)sgs (right).
Figure 5: Evolution of kinetic energy E(t) (left) and enstrophy Z(t) (right) for the different models used in this study. Simulations are integrated for almost 300 turnover times tL .
Figure 6: Vorticity fields at the end of the a posteriori evaluation ( t=240 ) for the DNS, filtered DNS and the models in this study. Note that τ≡Monstate(sm) was unstable and is thus shown as empty. The offline model τ≡Moff was also omitted from this plot because it crashed during the simulation.
Figure 7: Kinetic energy and enstrophy spectra E(k) (left) and Z(k) (right) averaged over 2,500 samples on the a posteriori evaluation horizon t∈[0,240] for the different models used in this study. The forcing wavenumber kf=15 is shown in dashed gray.
Figure 8: Probability distribution of vorticity ω and kinetic energy E(x,y)=1/2(ux2+uy2) at the end of the a posteriori evaluation at t=240 for the different models used in this study.
S{E}
S{Z}
S{E(k)}
S{Z(k)}
S{pdf(ω)}
S{pdf(E)}
Monsubgrid(sm)
−6.586
0.309
−3.747
−0.170
−0.381
−4.064
Monstate(lg)
−14.240
0.180
−8.170
−0.030
−0.380
−15.569
Monsubgrid(lg)
−2.623
0.942
−1.048
0.537
0.592
−1.808
Monstate(ref)
0.753
0.965
0.290
0.790
0.821
0.550
Monsubgrid(ref)
−1.333
0.966
−0.961
0.543
0.829
−1.348
Table 3: Similarity measures on the metrics used in the a posteriori evaluation. Note that the metric is systematically calculated against the DNS with respect to the under-resolved simulation τ≡0 .
Climate modeling uses physical simulations and data-driven methods to understand, predict, and project Earth’s climate and its response to external forces.
We present a machine learning approach for developing subgrid-scale (SGS) parametrizations in coarse simulations of partial differential equations. We utilize structure-preserving neural networks and entropy variables to learn subgrid fluxes in coarse simulations of the Burgers' equation. In particular, we employ a decoupled neural network architecture explicitly separating the subgrid corrections into two distinct components: a conservative Flux Potential network and an Eddy Viscosity network. We demonstrate that this reduced-order framework maintains high physical fidelity, accurately reproducing the energy spectrum, spatial and temporal correlation functions, and dynamical characteristics of the full-scale system. Furthermore, we show that our approach is robust and applicable to parameters outside the training regime.
Aijaz Nazir, Ilya Timofeyev
Dept. of Mathematics, University of Houston, Houston, TX 77204
The ability to emulate complicated and computationally-expensive physical systems in an efficient and stable manner is a popular motivation for scientific machine learning (SciML). Stable numerical solvers tend to be computationally expensive, as they attempt to resolve small (spatial and temporal) scales that receive and dissipate spectral energy. Learned coarse-grid forecasters remove the dissipative channel, allowing unresolved frequencies to re-emerge as grid-scale noise (aliasing). In this paper, we use the Neural Discrete Equilibrium (NeurDE) method to explore stable emulation at extremely coarse grids, demonstrating physically-principled control over entropy dynamics that reinforces the physical system's dissipative mechanism. Entropically-stabilized NeurDE produces stable long-time rollouts for thousands of forecasting steps on large Reynolds number turbulent flows (Re=100,000) with very coarse grids (down to R16×16). At these very coarse grid scales, entropically-stabilized NeurDE can be more stable than even our stabilized data-generation numerical method.
Jose Antonio Lara Benitez, Kareem Hegazy, Anastasis Kratsios +3
Rice University and LBNL · ICSI and University of California at Berkeley · McMaster University and the Vector Institute +3
Hybrid AI-physics climate modeling aims to improve coarse (~100km-resolution) Earth system models by learning to parameterize subgrid processes from high-fidelity data. However, this so far mostly involves local-in-time, diagnostic parameterizations, in which the subgrid state depends only on the current coarse state with no memory of previous states, which is unrealistic for processes such as convection that have intrinsic persistence. To address this, we enhance local-in-time parameterizations by learning prognostic variables that compactly carry important, additional past information where no explicit sub-grid information is available. First we compress past information into a low-dimensional latent space using an autoencoder, which then informs a neural network trained to parameterize targeted subgrid-scale processes. We then replace the autoencoder with symbolic equations that govern the time evolution of the latent variables, yielding additional prognostic memory variables that can be integrated alongside the resolved atmospheric state. We evaluate this approach on two systems: the Lorenz-96 model (online) and surface precipitation from high-resolution atmospheric simulations (offline). A forced multivariate linear ordinary differential equation recovers most of the added value achieved by the autoencoder-based approach in both experiments. Benchmarked against diagnostic parameterizations without memory, our memory-informed approach improves climate statistics and temporal structure, including a realistic diurnal cycle of tropical land precipitation.
Jurij Schönfeld, Tom Beucler, Julien Savre +2
Deutsches Zentrum für Luft- und Raumfahrt, Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany · University of Bremen, Institute of Environmental Physics (IUP), Bremen, Germany · Faculty of Geosciences and Environment, University of Lausanne, Lausanne, Switzerland +3