The computational cost of stiff chemical kinetics remains a dominant bottleneck in reacting-flow simulation, yet hybrid integration strategies are typically driven by hand-tuned heuristics or supervised predictors that make myopic decisions from instantaneous local state. We introduce a constrained reinforcement learning (RL) framework that autonomously selects between an implicit BDF integrator (CVODE) and a quasi-steady-state (QSS) solver during chemistry integration. Solver selection is cast as a Markov decision process. The agent learns trajectory-aware policies that account for how present solver choices influence downstream error accumulation, while minimizing computational cost under a user-prescribed accuracy tolerance enforced through a Lagrangian reward with online multiplier adaptation. Across sampled 0D homogeneous reactor conditions, the RL-adaptive policy achieves a mean speedup of approximately 3×, with speedups ranging from 1.11× to 10.58×, while maintaining accurate ignition delays and species profiles for a 106-species n-dodecane mechanism and adding approximately 1% inference overhead. Without retraining, the 0D-trained policy transfers to 1D counterflow diffusion flames over strain rates 10--2000s−1, delivering consistent ≈2.2× speedup relative to CVODE while preserving near-reference temperature accuracy and selecting CVODE at only 12--15% of space-time points. Overall, the results demonstrate the potential of the proposed reinforcement learning framework to learn problem-specific integration strategies while respecting accuracy constraints, thereby opening a pathway toward adaptive, self-optimizing workflows for multiphysics systems with spatially heterogeneous stiffness.
Figures & tables
Figure 1: Reinforcement learning (RL) framework for adaptive solver selection. The RL agent observes the evolving thermo-chemical state variables (e.g., T , OH, H 2 , O, etc.) from the combustion simulation and selects the optimal numerical solver (e.g., CVODE or QSS) at each step.
Parameter
Value
Rollout length Nrollout
2048
Minibatch size
64
PPO epochs Nepoch
4
Learning rate
3×10−4
Discount γ
0.99
GAE λGAE
0.95
Table 1: Summary of key hyperparameters
Figure 2: RL-adaptive performance across initial conditions: (a) speedup distribution and (b) dependence on (T0,p) . Accuracy is maintained within 2.6% ignition-delay error and < 110K temperature RMSE.
Figure 3: 0D trajectory comparisons across thermochemical conditions using the RL-adaptive solver policy. (a) Condition 1 (650K, 1.0atm) achieves a 3.25 × speedup with 0.41% ignition delay error and 5.4% CVODE usage. The temperature subplot shows solver selection via colored scatter (red=CVODE, green=QSS), with CVODE concentrated in a narrow ignition window ( t≈15 –22 ms). (b–d) As temperature and pressure increase (hence stiffness), CVODE usage rises from 14% (b) to 68.0% (c,d), demonstrating adaptive resource allocation. In all cases, the RL-adaptive approach maintains ignition delay errors below 2.62% and temperature RMSE below 108K.
Figure 4: Space-time evolution of temperature, HX2O , and OH illustrating agreement between the RL-adaptive solution and the CVODE reference, and deviations introduced by pure QSS.
Figure 5: Spatiotemporal solver deployment in the 1D counterflow diffusion flame at strain rate 2000s−1 , as selected by the RL policy. Red denotes CVODE and turquoise denotes QSS.
Reinforcement learning studies how agents learn sequential decisions from rewards and interaction, balancing exploration, performance, safety, and adaptation.
We present a physics-constrained machine learning framework for accelerating the direct numerical simulation (DNS) of turbulent reacting flows. The model replaces the direct evaluation of detailed chemical source terms with a surrogate that predicts reaction rates from a reduced thermochemical state. To improve physical consistency, the second law of thermodynamics is incorporated as a training constraint by enforcing non-negative entropy generation, which restricts the evolution of the thermochemical state to physically admissible directions and improves stability during time integration. The approach is demonstrated on DNS of a two-dimensional planar lean premixed methane-air flame interacting with a turbulent flow field. The model reproduces detailed-chemistry results with high fidelity while achieving more than an order-of-magnitude reduction in computational cost. Furthermore, a residual-based synthetic data augmentation strategy enables parametric exploration by constructing new training data from the original dataset, allowing accurate simulation at new inlet conditions without additional detailed-chemistry CFD runs. These results demonstrate that thermodynamically constrained machine learning can provide reliable and computationally efficient surrogates for detailed chemistry in high-fidelity combustion simulations.
Okezzi Ukorigho, Opeoluwa Owoyele
Department of Mechanical and Industrial Engineering Louisiana State University, Baton Rouge, LA 70810, USA
Hybrid domain decomposition methods provide a flexible framework for coupling full order models (FOMs) and reduced order models (ROMs), but typically assume the model assigned to each subdomain is fixed throughout a simulation. This is limiting for transient problems in which localized features propagate through the domain and the regions requiring high-fidelity resolution change over time. We introduce a reinforcement learning (RL)-based approach for online adaptation of FOM-ROM models coupled via the overlapping Schwarz alternating method (O-SAM), an iterative domain decomposition method that solves subdomain-local problems while exchanging solution information through transmission boundary conditions on overlapping interfaces. Deep Q-networks (DQNs) are trained offline to select among subdomain-local FOMs and pre-trained Operator Inference (OpInf) ROMs using a reward balancing accuracy, cost, and model-switching frequency. Once trained, the policies are deployed predictively on problem instances not seen during training, without requiring a reference FOM solution. We demonstrate the approach on two examples: a 1D advection-diffusion problem with a moving front, and a 3D linear elastic wave propagation problem implemented in the Norma.jl solid mechanics code. For the advection-diffusion benchmark, the learned policy dynamically allocates high-fidelity resolution as the front propagates and outperforms static FOM/ROM assignments; letting the agent also adapt the domain decomposition provides no further benefit. For the elastic wave benchmark, learned policies for two and three subdomain decompositions track the propagating wave by assigning FOMs to subdomains containing the wave and ROMs elsewhere, as expected. Our results demonstrate the potential of RL to enable predictive online adaptation of model fidelity within Schwarz-based hybrid simulations.
Turbulent combustion simulations are crucial for many scientific and engineering systems. However, the high cost to fully resolve the complex multiscale and multiphysics behavior makes direct simulation typically infeasible. The equivalent reactor network (ERN) approach attempts to improve computational efficiency by replacing a multidimensional turbulent simulation with a series of much cheaper 0-D and 1-D chemical reactors, providing a surrogate model that retains detailed chemistry at the cost of simplified flow physics. However, their development remains a challenge, often requiring either expert analysis, or automated approaches that sacrifice accuracy. In this work, we develop an automated machine-learning-assisted framework for constructing ERNs of the Sandia-D turbulent methane/air flame. Principal component analysis is first used to reduce high-dimensional thermochemical computational fluid dynamics (CFD) data to a low-dimensional latent space, where k-means clustering identifies physically interpretable flame regions used to initialize a reactor-network graph. This initialization is then refined using finite-difference gradient descent wrapped around non-differentiable Cantera reactor simulations. Across 30 RANS simulations spanning a range of pilot temperatures and inlet methane compositions, the optimized 7-reactor ERN achieves a maximum-temperature R2 score of 0.7945 while preserving a ∼6000× speedup over the CFD solver. Outlet CO prediction remains more challenging, with a final R2 score of −0.4183, but improves substantially from the unoptimized clustering initialization. These results show that unsupervised thermochemical feature extraction can provide effective physics-informed initializations for ERN construction, while gradient-based refinement can significantly improve predictive accuracy without manual reactor-network design.
Nicolas J. Tricard, Benjamin C. Koenig, Sili Deng
Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA