MCMC Informed Neural Emulators for Uncertainty Quantification in Dynamical Systems

Heikki Haario; Zhi-Song Liu; Martin Simon; Hendrik Weichel

cs.LGMar 11, 2026

MCMC Informed Neural Emulators for Uncertainty Quantification in Dynamical Systems

Authors: Heikki Haario, Zhi-Song Liu, Martin Simon, Hendrik Weichel

Organizations: Lappeenranta-Lahti University of Technology LUT, Lahti, Finland · Frankfurt University of Applied Sciences, Frankfurt am Main, Germany · University of Huddersfield, Huddersfield, United Kingdom

Abstract

Neural networks are a commonly used approach to replace physical models with computationally cheap surrogates. Parametric uncertainty quantification can be included in training, assuming that an accurate prior distribution of the model parameters is available. Here we study the common opposite situation, where direct screening or random sampling of model parameters leads to exhaustive training times and evaluations at unphysical parameter values. Our solution is to decouple uncertainty quantification from network architecture. Instead of sampling network weights, we introduce the model-parameter distribution as an input to network training via Markov chain Monte Carlo (MCMC). In this way, the surrogate achieves the same uncertainty quantification as the underlying physical model, but with substantially reduced computation time. The approach is fully agnostic with respect to the neural network choice. In our examples, we present a quantile emulator for prediction and a novel autoencoder-based ODE network emulator that can flexibly estimate different trajectory paths corresponding to different ODE model parameters. Moreover, we present a mathematical analysis that provides a transparent way to relate potential performance loss to measurable distribution mismatch.

Figures & tables

Figure 2: 2D scatter of Himmel parameters.

Figure 3: FaIR calibration results (until 2005) and sampling from posterior predictive distribution (from 2005). With historical temperature measurements in red, to the median, 90% credible interval and 99% credible interval in blue.

Figure 4: The structure of the proposed AEODE .

Panel of Figure 5 — Figure 5: Comparison of example $E_{0}$ distributions and resulting emission pathways.

Figure 6: Visualization of the fit of the empirical and trained 90% credible interval to the temperature projection.

Modules	Evaluation
Modules	MSE( $10^{-5}$ )	RMSE ( $10^{-3}$ )	MAE ( $10^{-3}$ )	MBE ( $10^{-4}$ )	Running time ( $10^{-5}$ s)
Torchdiffeq	3.231	5.426	3.552	-7.539	5.1
ChemiODE	2.582	5.085	3.356	-1.759	8.2
Ours	2.450	4.950	3.064	-1.316	9.5

Table 1: Testing data comparison among Torchdiffeq, ChemiODE, and ours. We report the results on the testing dataset using different methods. For all metrics, the smaller the values, the better the performance we get.

Figure 7: The proposed AEODE for chemical prediction. We show all 3000 testing data samples using different chemical reaction parameters in gray lines. The five lines with different colors represent the chemical compounds predicted by the proposed network.

Figure 8: The distribution comparison between the proposed model and the ground truth data. We show all 3000 testing data samples at the last time step (600) using different chemical reaction parameters in gray lines. The model predictions are shown in different colour, which indicate different chemical compounds.

Modules				Evaluation
NeuralODE	Time encoding	Attn	Physical loss	MSE( $10^{-5}$ )	RMSE ( $10^{-3}$ )	MAE ( $10^{-3}$ )	MBE ( $10^{-4}$ )
✓				3.101	5.569	3.565	-8.190
✓	✓			2.593	5.092	3.209	-0.951
✓	✓	✓		2.466	4.966	3.076	-1.273
✓	✓	✓	✓	2.450	4.950	3.064	-1.316

Table 2: Ablation studies of using proposed modules in the chemical kinetics example. We report the results on the testing dataset using different key modules and loss terms.

Figure 9: Visualization of the histogram of chemical concentration at time step 600. We use the histogram to statistically compute the distribution of all 3000 testing data samples at the last time step. The gray curve is the ground truth, and colored lines are model predictions.

Modules				Evaluation
NeuralODE	Time encoding	Attn	Physical loss	MSE( $10^{-5}$ )	RMSE ( $10^{-3}$ )	MAE ( $10^{-3}$ )	MBE ( $10^{-4}$ )
✓				3.200	1.859	1.596	-0.166
✓	✓			3.010	1.768	1.446	-0.016
✓	✓	✓		3.002	1.751	1.423	-0.015
✓	✓	✓	✓	2.949	1.718	1.389	-0.007

Table 3: Ablation studies of using proposed modules in the FaIR example. We report the results on the testing dataset using different key modules and loss terms.

Panel of Figure 10 — Figure 10: Comparison of forward operator simulations.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Chains of the MCMC experiments for the FaIR climate model.

Ordinary Differential Equations

See all 33 papers

A numerical study into neural network surrogate model performance for uncertainty propagation
May 15, 2026Noah Wade, Kirubel TeferraNeural Surrogate Modeling Uncertainty Quantification
ASEE Postdoctoral Associate, U.S. Naval Research Laboratory, Washington, DC, 20375, USA · U.S. Naval Research Laboratory, Washington, 20375, DC, USA
Predictive Uncertainty for Neural CAE Surrogates
Sep 21, 2026Kaustubh Tangsali, Mohammad Amin Nabian, Kelvin Lee +2Neural Surrogate Modeling Uncertainty Quantification
NVIDIA
Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates
Feb 24, 2026Haoze Song, Zhihao Li, Mengyi Deng +4Uncertainty Quantification Epistemic Uncertainty
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

May 15, 2026J/K move · Enter open · S save

A numerical study into neural network surrogate model performance for uncertainty propagation

Noah Wade, Kirubel Teferra

ASEE Postdoctoral Associate, U.S. Naval Research Laboratory, Washington, DC, 20375, USA · U.S. Naval Research Laboratory, Washington, 20375, DC, USA

Neural Surrogate Modeling Uncertainty Quantification PDE Surrogate Modeling OOD Generalization Deep Operator Networks Uncertainty Propagation

Open paper AskPDF arXiv

Neural network surrogate models have emerged as a promising approach to model solution fields for a wide variety of boundary value problems encountered in physical modeling. Stochastic problems represent an area of particularly high interest because of the potential to significantly reduce the repeated evaluation of expensive forward models via traditional numerical solvers when conducting parametric analysis. However, many studies found in the literature primarily focus on the ability of neural network surrogate models to represent deterministic samples or mean field solutions and largely overlook surrogate model performance at the tails of the distribution. The present study examines in detail the ability of neural network surrogate models to capture the full distribution of solution fields over the entire probability space, while emphasis is placed at the tails of the distribution. Serving as a canonical problem is the heat conduction equation with a highly stochastic source term, inducing extremely large variation in the thermal solution field. Comparisons are made between a classic feed-forward fully connected network and a Deep Operator Network architecture, using both data-driven and physics-informed loss functions. Results show that the worst-case prediction errors are an order of magnitude larger than the mean field error, highlighting the importance of the outlier samples. The large errors associated with extreme samples result from the networks having to extrapolate beyond the bounds of the training data. A method for identifying these samples is presented along with a discussion of potential approaches to account of their errors. Among the models considered, the fully connected neural network trained using a weak form residual loss performs best in handling these extrapolated inputs, achieving the highest prediction accuracy for the numerically produced datasets.

Abstract

Ordinary Differential Equations

Explore similar work

A numerical study into neural network surrogate model performance for uncertainty propagation

Predictive Uncertainty for Neural CAE Surrogates

Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates