Calibration of a neural network ocean closure for improved mean state and variability

Pavel Perezhogin; Alistair Adcroft; Laure Zanna

physics.ao-phApr 7, 2026

Calibration of a neural network ocean closure for improved mean state and variability

Authors: Pavel Perezhogin, Alistair Adcroft, Laure Zanna

Organizations: Courant Institute School of Mathematics, Computing, and Data Science, New York University, New York, NY, USA · Program in Atmospheric and Oceanic Sciences, Princeton University, Princeton, NJ, USA

Abstract

Global ocean models exhibit biases in the mean state and variability, particularly at coarse resolution, where mesoscale eddies are unresolved. To address these biases, parameterization coefficients are typically tuned ad hoc. Here, we formulate parameter tuning as a calibration problem using Ensemble Kalman Inversion (EKI). We optimize parameters of a neural network parameterization of mesoscale eddies in two idealized ocean models at coarse resolution. The calibrated parameterization reduces errors by factors of 1.7-3.3 in the time-averaged fluid interfaces and their variability compared to the unparameterized model, depending on the metric and configuration. The EKI method is robust to noise in time-averaged statistics arising from chaotic ocean dynamics. Furthermore, we propose an efficient calibration protocol that bypasses integration to statistical equilibrium by carefully choosing an initial condition. These results demonstrate that systematic calibration can substantially improve coarse-resolution ocean simulations and provide a practical pathway for reducing biases in global ocean models.

Figures & tables

Figure 1: (a) Idealized wind-driven ocean model GFDL MOM6 in a double-gyre configuration. (b) The eddy kinetic energy (EKE) spectrum as a function of isotropic horizontal wavenumber in the upper fluid layer and domain $5^{\circ}\mathrm{E}$ $-$ $15^{\circ}\mathrm{E}$ $\times$ $35^{\circ}\mathrm{N}$ $-$ $45^{\circ}\mathrm{N}$ . The percentages represent the integral over the spectrum relative to that of the high-resolution simulation. Panel (c) shows how the Ensemble Kalman Inversion calibration algorithm interacts with the coarse ocean model in order to update the free parameters of the parameterization such that the time-mean sea surface height is as close as possible to the filtered and coarse-grained high-resolution simulation (”observation”).

Figure 2: Calibration of the eddy parameterization (eANN) in Double Gyre configuration. The upper row shows time-averaged sea surface height (SSH), and the second row shows the temporal standard deviation of SSH. On these panels, all simulations are 100 years long and results are averaged over 90 years. We show coarse ocean models (

1/2^{\circ}

) with: (a,e) no parameterization, (b,f) parameterization trained offline with manually adjusted scaling coefficient (

\gamma=1.4

), (c,g) calibrated parameterization. Panels (d,h) show the filtered and coarse-grained high-resolution simulation (

1/32^{\circ}

). The lowest row shows the convergence of the calibration process, as assessed by two metrics included in the loss function (i, j) and one metric excluded from the loss function (k). Blue markers show the ensemble median, and error bars show the 25% and 75% quantiles.

Figure 3: Evaluation of calibrated parameterizations in 30000-day simulations in configuration NeverWorld2. (a) Zonally- and time-averaged vertical coordinate of internal fluid interfaces. Lower row shows temporal standard deviation of sea surface height for simulations with: (b) unparameterized model, (c) parameterization trained offline with manually adjusted scaling coefficient (

\gamma=0.5

), (d) parameterization calibrated in Double Gyre with manually adjusted

\gamma=0.6

, (e) parameterization calibrated in short 5-year runs in NeverWorld2 configuration, (f) filtered and coarse-grained high-resolution model.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Comparison of the offline skill in the global ocean CM2.6 dataset Griffies et al. ((2015)) between the neural network used for online simulations in Perezhogin et al. ((2025)) (on the left) and the equivariant neural network used in this study (on the right). We slightly increased the size of the single hidden layer of the neural network from 20 neurons to 64 neurons and slightly modified the loss function to match the accuracy of predictions in both the divergence of stress tensor ( $\nabla\cdot\mathbf{T}$ , upper row) and the stress tensor ( $\mathbf{T}$ , lower row).

Figure 2: Sensitivity of the calibration in DG configuration to the scheduler step (

\delta t

), in the upper row, and random seed, in the lower row. The left and center columns show the components of the loss function, which track the convergence of the calibration algorithm. The right column shows the spread of the ensemble in parameter space, which tracks the ensemble consensus or collapse. The scheduler step controls the convergence: large values result in a non-monotonic decreasing of the loss function and too fast ensemble collapse. On the contrary, a small value of

\delta t

results in slower and more monotonic loss decreasing and slower ensemble collapse. Lower panels show that loss dynamics and ensemble spread follow similar trends for two additional random seeds. The ensemble size is

n_{e}=100

Figure 3: Calibration of the scaling coefficient

\gamma

in front of the offline-trained eANN parameterization via a grid search in the Double Gyre configuration in 100-year simulations. Upper panel: root mean squared error (RMSE) in the time-mean sea surface height (SSH MEAN) and temporal standard deviation of sea surface height (SSH STD) between coarse parameterized model and filtered and coarse-grained simulation. Middle panel: spatial pattern correlation. Lower panel: The loss function similar to that used for automatic calibration. Coefficient

\gamma=1.4

gives the best joint error in the mean state and variability of the surface ocean, and best spatial pattern correlation.

Figure 4: Assessment of the random noise originating from ocean chaotic dynamics in temporal averages. Results for the calibrated eANN parameterization are shown. Panels compare 10-year averages over non-overlapping time intervals to the 90-year average. Root Mean Squared Error (RMSE) is shown with respect to the 90-year average. The noise magnitudes in the temporal average and standard deviation (STD) of the sea surface height (SSH) are comparable (

\mathrm{RMSE}\approx 0.02

). Note that the noise is more significant for the SSH STD metric, as it amounts to approximately half of the parameterized model error. Noise in the time-averaged SSH looks less significant by visual inspection. However, we note that the noise is hidden by the strong mean pattern.

Figure 5: Sensitivity to the setup of the calibration protocol in NW2 configuration: spin-up time (0 or 1000 days) and averaging time (800 or 100 days). The metrics shown are obtained in long 30000-days integrations from the state of rest. Increasing the averaging time is crucial for improved variability (d,e). The spin-up time has almost no effect on both the mean state and variability (c,e).

Figure 6: Mean state and variability in NW2 configuration at resolution

1/2^{\circ}

for the offline-trained eANN parameterization. Sensitivity to the scaling coefficient (

\gamma

) in front of the parameterization, where

\gamma=0

corresponds to the unparameterized model. The mean state and variability can be improved by

13.4\%

(upper row) and

4.4\%

(lower row), respectively, compared to the unparameterized simulation when

\gamma=0.5

Figure 7: Generalization to unseen configuration NW2 for the eANN parameterization calibrated in different configuration (DG). Sensitivity to the scaling coefficient (

\gamma

) in front of the parameterization, where

\gamma=0

corresponds to the unparameterized model. The mean state and variability can be improved by

41.9\%

(upper row) and

20.9\%

(lower row), respectively, compared to the unparameterized simulation when

\gamma=0.6

Figure 8: Offline analysis of parameterizations on output from the CM2.6 climate model at coarse-graining factor 4 and depth 5 m. $\mathbf{S}=\frac{1}{2}(\nabla\mathbf{u}+(\nabla\mathbf{u})^{T})$ is the strain-rate tensor, and $\mathbf{u}\partial_{t}\mathbf{u}=\cdots-\mathbf{T}:\mathbf{S}$ is the subgrid KE transfer, where positive (red) values indicate backscatter and negative (blue) values are dissipation. The KE transfer is decomposed into the contributions from the along-gradient fluxes (left) and isotropic stress (center), where $\mathbf{T}^{\prime}=\mathbf{T}-\frac{1}{2}\mathrm{tr}\mathbf{T}$ is the deviatoric stress. The cross-gradient projection (right) uses $\mathbf{S}^{\prime}_{\perp}=\frac{1}{2}\begin{bmatrix}\sigma_{S}&-\sigma_{D}\\ -\sigma_{D}&-\sigma_{S}\end{bmatrix}$ which satisfies $\mathbf{S}^{\prime}{\perp}:\mathbf{S}^{\prime}=0$ . Rows show KE fluxes from the high-resolution data (top), the offline-trained eANN with $\gamma=1$ (second), and eANNs calibrated in DG and NW2 configurations (bottom two). The “corr” denotes pattern correlation with the offline-trained predictions. The scaling coefficient shows how to rescale the offline-trained parameterization to obtain the calibrated one.

	Perezhogin et al. 2025	eANN (this study)
Input features	27	27
Output features	3	3
Hidden layers	1	1
Neurons in hidden layer	20	64
Parameters in the first layer	560	1792
Parameters in the last layer	63	195

Appendix

Table 1: Comparison of the architecture of the neural networks used in Perezhogin et al. ((2025)) for online simulations and the equivariant neural network (eANN) used in this study.

Parameter	Weights												Biases	$\gamma$
Offline	0.68	0.69	1.15	0.15	0.60	-0.60	0.74	0.01	-0.36	0.31	-0.04	2.25	0.15	1.00
STD	0.18												0.04	0.25
DG	0.64	0.66	1.35	0.23	0.15	-0.50	0.40	0.29	-0.79	0.51	-0.19	2.06	0.16	1.08
NW2	0.85	0.73	1.29	0.18	0.35	-0.69	0.21	0.15	-0.44	0.48	0.03	2.18	0.17	0.88

Appendix

Table 2: Parameter set that was calibrated in the eANN parameterization, which represents the last layer of the neural network (12 weights and 1 bias) and scaling coefficient in front of the parameterization (

\gamma

). Offline-trained values (”Offline”) are obtained on the global ocean dataset CM2.6. The initial ensemble is drawn from the normal distribution, which has the offline-trained values as its mean and the standard deviation shown in the third row (”STD”). The weights share the same standard deviation at initialization, which is 25% of the standard deviation over the parameter dimension. The standard deviation of the bias and the scaling coefficient is 25% of their respective absolute values. Parameters of the calibrated parameterization in Double Gyre (”DG”) and NeverWorld2 (”NW2”) configurations are shown in the two bottom rows.

Model Calibration

Model calibration aligns predicted confidence or model outputs with observed outcomes and measurements to support more reliable decisions.

See all 218 papers

Calibrating subgrid parametrizations of single-column ocean models via simulation-based inference
Sep 3, 2026Luben M. C. Cabezas, Sacha Wendling, Aurèle Gallard +3Model Calibration Simulation-Based Inference
DEs, ICMC Federal Univ. of São Carlos, Univ. of São Paulo, Brazil · Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France · Univ. Grenoble Alpes, Inria, CNRS, IRD, Grenoble INP, INRAE, IGE, France
Samudra 2: Scaling Ocean Emulators across Resolutions
May 24, 2026Yuan Yuan, Jesse Rusak, Alexander Merose +5Neural Surrogate Modeling Climate Science
Courant Institute School of Mathematics, Computing, and Data Science, New York University, New York, NY, USA · Open Athena AI Foundation, Inc., New York, NY, USA · Program in Atmospheric and Oceanic Sciences, Princeton University, Princeton, NJ, USA
Njord: A Probabilistic Graph Neural Network for Ensemble Ocean Forecasting
May 14, 2026Daniel Holmberg, Joel Oskarsson, Erik Wikingsson +2Graph Neural Networks Uncertainty Quantification
University of Helsinki · ETH AI Center · Linköping University

Sep 3, 2026J/K move · Enter open · S save

Calibrating subgrid parametrizations of single-column ocean models via simulation-based inference

Luben M. C. Cabezas, Sacha Wendling, Aurèle Gallard, Gabriel Mouttapa+2

DEs, ICMC Federal Univ. of São Carlos, Univ. of São Paulo, Brazil · Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France · Univ. Grenoble Alpes, Inria, CNRS, IRD, Grenoble INP, INRAE, IGE, France

Model Calibration Simulation-Based Inference Neural Posterior Estimation Computational Fluid Dynamics

Open paper AskPDF arXiv

Subgrid parametrizations of vertical mixing in ocean models depend on free coefficients that cannot be measured directly and must be calibrated against high-fidelity references such as large-eddy simulations (LES). Existing approaches return point estimates and leave the associated uncertainty unquantified, a limitation when the inverse problem is ill-posed or when distinct parameter configurations fit the data comparably well. Simulation-based inference (SBI) addresses exactly this: given a prior and access to the simulator, it approximates the full posterior over parameters without requiring a tractable likelihood, at a cost set by the number of simulator evaluations. We apply it to tunax, a JAX-based single-column ocean model, to calibrate the coefficients of its

k

\varepsilon

closure. A blockwise PCA summary statistic compresses the simulator output along the depth axis while preserving its forcing--horizon--variable structure, making inference tractable at modest budgets. We compare neural posterior estimation and its sequential variants against a recent training-free approach built on a tabular foundation model. The latter recovers informative posteriors from a few hundred simulator calls, outperforming the trained estimators at every budget considered.

Abstract

Model Calibration

Explore similar work

Calibrating subgrid parametrizations of single-column ocean models via simulation-based inference

Samudra 2: Scaling Ocean Emulators across Resolutions

Njord: A Probabilistic Graph Neural Network for Ensemble Ocean Forecasting