Increased computing power has transformed computational science and improved predictive capabilities for complex systems, but many phenomena still exceed current simulation capabilities or require rapid evaluations for fast design iterations. Closure models address this challenge by approximating unresolved effects in reduced or incomplete dynamical systems. Data-driven closures offer a flexible mechanism for improving model accuracy and applicability, but approaches that embed the closure within a computational model for construction and training phases can be computationally expensive and cause numerical instabilities in the computational model during training. To avoid the challenges posed by embedded approaches, staged approaches to closure-model learning have been developed. The first stage uses data assimilation to infer state and closure trajectories from observations. The second stage trains a machine learning closure model offline using the assimilated data, thereby avoiding repeated simulator calls during training. In this work, we investigate the cost and accuracy tradeoffs between embedded and staged data-driven closure modeling approaches for a synthetic epidemiological SIQR model with a nonlinear state-dependent quarantine closure. The results illustrate a practical accuracy-efficiency trade-off: for this exemplar, embedded training provides the strongest average accuracy, while the staged approach yields comparable qualitative behavior with a moderate increase in error, substantially fewer model evaluations, and improved optimization robustness.
Figures & tables
Figure 1: True SIQR system dynamics and closure trajectory used to generate the synthetic observations.
Figure 2: Embedded and staged state trajectories across 50 replicates compared with the true SIQR trajectory.
Figure 3: Learned closure trajectories from embedded and staged replicates compared with the true synthetic closure.
Figure 4: Distribution of calibration errors over the 50 independent replicates for the embedded and staged approaches. Each violin shows the empirical distribution of the replicate-level error metric for one method, with width indicating relative density. Horizontal lines inside each violin indicate the 25th , 50th , and 75th percentiles of the replicate distribution.
Method
Training model evaluations
Failed starts
Embedded
2.41×104±7.41×103
215
STAGED DDCM
2.78×103±3.59×102
0
Table 1: Comparison of embedded and staged closure-training approaches over 50 replicates. Computational cost is measured by training-stage model evaluations, where results are presented as μ±σ over the 50 replicates. Robustness is measured by the total number of failed neural-network starts.
Component
Model evaluations
Process-noise MLE
2.68×103±3.59×102
Final EnKF assimilation
1.00×102±0
Total STAGED DDCM
2.78×103±3.59×102
Table 2: Training-stage model-evaluation accounting for the staged approach. The total is the sum of model evaluations used for process-noise maximum-likelihood estimation and the final EnKF assimilation. The subsequent supervised neural-network training step does not call the SIQR simulator and is therefore excluded from this simulator-dependent cost.
Figure 5: Calibration and validation scenario trajectories in the susceptible-infectious projection.
Figure 6: Embedded and staged state trajectories under validation scenarios.
Figure 7: Embedded and staged learned closure trajectories under validation scenarios.
Figure 8: Distribution of validation scenario errors over the 50 independent replicates for the embedded and staged approaches. Each violin shows the empirical distribution of the replicate-level error metric for one method, with width indicating relative density. Horizontal lines inside each violin indicate the 25th , 50th , and 75th percentiles of the replicate distribution.
Stochastic closure modeling is an emerging area in reduced-order modeling of large and complex (e.g., multiscale, multi-physics) systems, for which the computationally affordable reduced-order physical descriptions (e.g., digital twins for additive manufacturing, world models for AI agents) in numerical simulations inevitably lead to information losses that cannot be well addressed by classical deterministic closures. However, learning stochastic closures typically requires closure data that are difficult to obtain from measurements or often prohibitively expensive to compute using first-principles simulations. In this work, we develop a stochastic closure modeling framework that builds on score-based diffusion models, leveraging indirect offline training and online data assimilation without requiring the direct data of closures. Numerical experiments on two-dimensional turbulent flows demonstrate that indirect offline learning captures key closure statistics and improves predictions of system state. In addition, combining the learned closure with known physical dynamics leads to better generalization capability across different regimes than purely data-driven models that directly predict system state. Compared to standard data assimilation methods with a Gaussian assumption, the posterior-score assimilation better represents non-Gaussian features of the learned closure, and its combination with state-level data assimilation further improves prediction of the system state. The proposed framework enables stochastic closure modeling in realistic and challenging scenarios of complex dynamical systems in mechanics and engineering problems, where non-Gaussian statistics of model-form uncertainties are non-negligible (e.g., due to the nonlinearities of the system dynamics and/or the observations) and true closure data are not available, for both offline learning and online assimilation.
Xinghao Dong, Huchen Yang, Jin-Long Wu
Department of Mechanical Engineering, University of Wisconsin–Madison, Madison, WI 53706
We develop a closure modeling framework for identifying missing components of dynamical systems using Kernel Ridge Regression (KRR). The framework addresses two classes of closure problems: difference-equation closures arising in ODE and PDE settings, and algebraic closures arising from moment closure in kinetic equations. For the first class, we derive an error bound in an ODE setting that quantifies contributions from time integration, approximation of unresolved scales, and interpolation required to couple unresolved-scale effects to the resolved solver. Numerical experiments on the Lorenz-63 system and the Kuramoto-Sivashinsky equation demonstrate accurate long-horizon predictions and substantial improvements over an LSTM-based closure model. For the second class, we consider moment closure for a one-dimensional kinetic equation by modeling discrepancies between kinetic and macroscopic fluxes as a function of the resolved macroscopic variables. We compare global KRR models based on PCA coordinates with spatially local models. While the global model performs well for unimodal initial conditions, its accuracy deteriorates for bimodal initial conditions. Spatially local models with appropriate modeling inputs improve robustness and achieve higher predictive accuracy.
Evan Habbershaw, John Harlim, Senwei Liang
Department of Mathematics, The Pennsylvania State University · Department of Mathematics and Institute for Computational and Data Sciences, The Pennsylvania State University · Department of Mathematics and Statistics, Texas Tech University
The differentiable physics paradigm may be leveraged as an a-posteriori approach for discovering turbulence closure models by embedding a neural network parameterization directly inside the solver and optimizing it given potentially sparse target data. This addresses a key limitation of a-priori learning where direct numerical simulation (DNS) data is used to approximate the subgrid stress with the assumption of a low-pass filter. Closures trained in this a-priori manner frequently lead to unstable deployments due to the mismatch between the assumed filter and the effect of numerical discretizations and coarse-graining. In comparison, while typically stable during deployment, a-posteriori learning incurs high computational costs due to the need to backpropagate through a large eddy simulation (LES) solver. Furthermore, a-posteriori methods are challenging to apply broadly since they require significant modification of existing solvers. Finally, both approaches are limited when generalization is desired across different numerical schemes with their implicit filtering characteristics. In this work, we present a deep-learning approach for turbulence closure modeling built on the continuous data assimilation framework. Our approach enables the a-priori training of closures using sparsely observed DNS data without modifying or differentiating through the LES solver, while preserving stability during deployment for the recovery of invariant statistics. We focus on the model's ability to adapt to different discretizations by explicitly conditioning it on the numerical scheme. We use two- and three-dimensional canonical cases to test our framework and show that the learned correction systematically tracks the discretization error of the coarse solver.
School of Mechanical Engineering Purdue University West Lafayette, IN 47907 · College of Information Sciences and Technology Pennsylvania State University University Park, PA 16802