Learning inadequacy of Bayesian neural operators for PDE-constrained inference and design
Authors: Pratyush Kumar Singh, Lianghao Cao, E. Bruce Pitman, Danial Faghihi
Organizations: Department of Mechanical and Aerospace Engineering, University at Buffalo, Buffalo, NY, USA · Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA · Department of Materials Design and Innovation, University at Buffalo, Buffalo, NY, USA
Partial differential equations (PDEs) underpin predictive modeling across science, engineering and medicine. Their use in digital twins, parameter inference and optimal design requires repeated solutions across parameters and scenarios. For nonlinear, transient and multiphysics systems, these calculations can make numerical simulation prohibitively expensive. Here we introduce the residual-informed Bayesian neural operator (RI-BNO), which learns operator inadequacy using a posteriori error estimates from the governing PDEs and incorporates its correction and uncertainty into Bayesian training and prediction. We establish local error bounds for the residual-informed estimates under stability and smoothness assumptions. We demonstrate RI-BNO on a non-linear reaction-diffusion benchmark with DeepONet and latent-subspace architectures, Markov chain Monte Carlo inference of glioma parameter fields from magnetic resonance imaging, and Monte Carlo risk-averse optimization of three-dimensional thermal breaks in building envelopes under manufacturing uncertainty. Across the evaluated examples, RI-BNO improves agreement with finite element (FE) reference predictions, parameter posteriors and optimized designs while using half as many full nonlinear or transient training simulations as the vanilla Bayesian neural operator (vBNO). For thermal break design, online optimization time decreases from 178 hours with the FE model to 18 minutes with RI-BNO. These results demonstrate the value of explicit inadequacy modeling in neural operators for reliable PDE-based inference and design.
Figures & tables
Figure 1: Residual-informed Bayesian neural operator (RI-BNO) workflow. A preliminary neural operator trained on Dpre supplies predictions at additional, independently sampled inputs. Solving ( 6 ) generate residual-informed observations E , from which the inadequacy model η is learned. This model is held fixed during final Bayesian training on the disjoint solution dataset Dfit . The learned inadequacy informs both the Bayesian likelihood and the predictive state.
Figure 2: Residual-informed Bayesian neural operators for learning solution operator of a nonlinear reaction–diffusion PDE. Panel (a) shows predictions for two unseen in-distribution parameter fields, m1 and m2 . For each input, the finite element (FE) state uh is compared with the state predictions of vBNO, uvBNO=FwMAP(m) , and RI-BNO, uRI=FwMAP(m)+μη(m) , using the latent-subspace and DeepONet architectures. The pointwise relative error εvBNO=∣uh−uvBNO∣/∣uh∣ and εRI=∣uh−uRI∣/∣uh∣ , and posterior predictive state variance σvBNO2=Vπpost(w∣D)[Fw(m)] and σRI2=Vπpost(w∣D,E)[Fw(m)]+Cη are also presented. Panel (b) illustrates the DeepONet architecture, comprising branch and trunk networks, and the latent-subspace architecture, comprising an active-subspace input encoder, a ResNet map in latent coordinates and a POD state decoder. Panel (c) shows the inadequacy mean μη and pointwise marginal variance Cη of the sparse Gaussian process for each architecture. The mean provides the residual-informed correction, whereas the marginal variance quantifies uncertainty in that correction. Panel (d) compares, for the latent-subspace architecture, kernel density estimates of the components of the MAP weight vectors wMAP between vBNO and RI-BNO, together with the posterior marginal variances (Γpost)ii ordered by their variance rank. Their differences illustrate that the inadequacy-aware likelihood modifies the posterior of the weights in RI-BNO compared to vBNO.
Figure 3: Predictive accuracy and uncertainty of vBNO and RI-BNO using latent-subspace and DeepONet architectures : (a) Relative accuracy versus the total number of data points Ntot (b) Coefficient of Variation (CV), measuring relative predictive spread, versus Ntot .
Figure 4: Forward prediction and inadequacy representation for synthetic glioma growth. Panel (a) shows the finite element evolution of the tumor volume fraction ϕT(x,t) from the prescribed initial condition to the terminal time tF=250 days under the mean parameter fields m=(logD,logG) , overlaid on an axial slice of a healthy brain atlas [ 35 ] . Panel (b) shows the terminal tumor fields for representative unseen realizations of m from the finite element reference uh with the vBNO and RI-BNO state prediction uvBNO and uRI . The accompanying distributions, computed over 100 unseen realizations of m , show the normalized tumor area NTA=(∫Ωu(x)dx)/∣Ω∣ , and the normalized diffuse-interface area NIA=(∫Ω\mathbbm1{0.1<u(x)<0.9}dx)/∣Ω∣ , where u=ϕT(x,tF) . Panel (c) shows the inadequacy mean μη and pointwise marginal variance Cη of the learned neural-operator inadequacy. Panel (d) shows terminal tumor boundaries for one fixed test realization of m . Red denotes the finite element tumor region, green the state prediction u , and the 50 blue curves posterior tumor boundary realizations. For vBNO, the boundary realizations are obtained by sampling w∼πpost(w∣D) . For RI-BNO, w∼πpost(w∣D,E) is combined with a realization of the learned inadequacy field η . The box plot summarizes the Dice similarity coefficients between the predicted and finite element tumor regions over the same 100 test inputs, ranging from 0 for no overlap to 1 for perfect overlap.
Figure 5: Patient-specific MRI inference using pretrained neural operators. Panel (a) shows an axial T1-CE MRI slice from the BraTS 2019 high-grade glioma case BraTS19_CBICA_AQP_1, together with the segmented tumor-core mask used for the two-dimensional inversion. Panel (b) summarizes offline operator construction, including the latent-subspace architecture, allocation of the matched total observation budget, offline neural operator construction cost, and representative synthetic realizations of training data of the spatially varying mobility and proliferation fields, m=(logD,logG) , and their terminal tumor states, uh=ϕT(x,tF) . Panel (c) compares the finite-element, vBNO and RI-BNO tumor reconstructions and their Dice similarity coefficients relative to the MRI-derived tumor core mask. Panel (d) shows the corresponding MAP estimates of the spatially varying fields logD and logG . Panel (e) shows bands of posterior infiltration exceedance probability, computed from 250 retained MCMC samples using ϕinv=0.075 . The bands 0.1<Pinv<0.5 , 0.5<Pinv<0.9 , and Pinv>0.9 indicate increasing posterior support for subvisual tumor extension outside the MRI-visible tumor core boundary, defined using the MRI-detection threshold ϕ=0.2 . Panel (f) shows the posterior distributions of the DICE similarity coefficient and brain-normalized tumor area computed from 500 retained MCMC samples.
Figure 6: Risk-averse design of a fiber-reinforced aerogel thermal break under spatial manufacturing uncertainty. Panel (a) shows the thermal break inserted between hot (red) and cold (blue) steel beams, along with geometry and bolt-hole displacements. Panel (b) illustrates the latent-subspace NO architecture. The NO learns the mapping from fiber volume fraction ϕf to the coupled PDE state u , while the heat flux q and von Mises stress TVM are recovered through post-processing operators applied to u . The two rows present training realizations of ϕf and the corresponding through thickness heat flux magnitude ∣q⋅eth∣ and von Mises stress fields. Panel (c) compares the optimal designs and physical fields obtained using FE-constrained, vBNO-accelerated and RI-BNO-accelerated optimization. The rows show, from top to bottom, the optimal design dopt(x) , the through thickness heat flux magnitude with arrows showing direction and relative magnitude, and the von Mises stress. Panel (d) compares risk indicators (rQ,rT) under realizations m∼νman , with each optimized design evaluated using its corresponding forward model: FE, vBNO or RI-BNO. Small markers denote individual realizations, while large markers denote the marginal CVaR0.95 values. Panel (e) compares the FE optimization cost with the total construction and optimization costs of vBNO and RI-BNO, including nonlinear PDE data generation, residual-informed observation generation, operator training and online risk-averse optimization. The right plot reports the online optimization times, with speedups of 639× and 600× for vBNO and RI-BNO relative to FE, respectively.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Comparison of state predictions across frameworks (vBNO, RI-BNO, and FE-error BNO) against FE solution for an unseen in distribution parameter field m using the latent subspace architecture.
Method
Test Error (%) (using GP mean)
Test Error (%) (Final)
Latent Subspace RI-BNO
2.65
1.02
Latent Subspace FE-error BNO
3.53
1.83
Appendix
Table S1: Comparison of test errors for the latent subspace architecture evaluated using the preliminary operator with GP mean correction versus the full inadequacy-aware Bayesian inference.
Method
Test Error (%) (using GP mean)
Test Error (%) (Final)
DeepONet RI-BNO
2.34
0.75
DeepONet FE-error BNO
3.17
1.52
Appendix
Table S2: Comparison of test errors for the DeepONet architecture evaluated using the preliminary operator with GP mean correction versus the full inadequacy-aware Bayesian inference.
Figure S2: Comparison of state predictions across frameworks (vBNO, RI-BNO, and FE-error BNO) against FE solution for an unseen in-distribution parameter field m using the DeepONet architecture.
Method
Test Error (%) (Final)
MPIW
PICP (%)
Latent Subspace vBNO
10.5
1.05 e-2
84.2
Latent Subspace RI-BNO
1.02
1.61 e-2
94.7
Latent Subspace FE-error BNO
1.83
1.57 e-2
94.4
Appendix
Table S3: Test accuracy and UQ metrics for the latent subspace architecture, comparing vBNO, RI-BNO, and FE-error BNO. Test Error denotes the relative L2 error at the final time, while MPIW and PICP denote the mean prediction interval width and prediction interval coverage probability of the 95% predictive intervals, respectively.
Method
Test Error (%) (Final)
MPIW
PICP (%)
DeepONet vBNO
7.73
0.93e-2
86.3
DeepONet RI-BNO
0.75
1.52e-2
94.2
DeepONet FE-error BNO
1.52
1.46e-2
93.8
Appendix
Table S4: Test accuracy and UQ metrics for the DeepONet architecture, comparing vBNO, RI-BNO, and FE-error BNO. Test Error denotes the relative L2 error at the final time, while MPIW and PICP denote the mean prediction interval width and prediction interval coverage probability of the 95% predictive intervals, respectively.
Method
FE solution generation
Correction Observation
Preliminary training
GP fitting
Final training
Total time
vBNO
1535.7
—
—
—
1214.6
2750.3
RI-BNO
751.3
119.1
303.6
512.9
371.2
2058.1
FE-error BNO
752.4
807.9
308.5
504.4
378.3
2751.6
Appendix
Table S5: Computational cost (seconds) per phase for latent subspace architecture: (a) FE data generation, (b) correction observation generation, (c) preliminary BNO training, (d) GP fitting, (e) final training using inadequacy-aware likelihood, and (f) total time.
Figure S3: Comparison of computational costs for latent subspace architecture for vBNO, RI-BNO and FE-error BNO. The total wall clock time is partitioned into finite element (FE) solution generation, correction-observation generation, and model construction time across the vBNO, RI-BNO, and FE-error BNO frameworks.
Figure S4: Comparison of predicted tumor volume fraction spatial profiles between FE solution uh , and state predictions through vBNO uvBNO , and RI-BNO uRI along with their pointwise relative error fields with respect to FE across four unseen test samples of the input parameter field m=(logD,logG) .
Parameter
Description
Value
Reference
K
phase separation energy of tumor and host tissues
0.025
[ 77 ]
H
interfacial energy among tumor cells and other species
0.01
[ 77 ]
R
coefficient of nutrient consumption
10
[ 78 ]
M
nutrient mobility
1
[ 79 , 80 ]
Appendix
Table S6: Summary of physical parameters used in the multiphase glioma growth model.
Figure S5: FE tumor trajectories for offline training and verification of MRI-inferred parameter fields. (a) Representative synthetic training realization, showing the sampled mobility and proliferation fields, m=(logD(x),logG(x)) , and the corresponding FE tumor evolution from the prescribed training initial condition. The terminal field at tF=240 days is used for operator learning. (b) FE tumor trajectories generated using the parameter MAP estimates obtained from vBNO inversion (upper row) and RI-BNO inversion (lower row). Both trajectories use the same patient-specific initial condition adopted in FE-based MRI inversion [ 44 ] , with the inferred parameter fields held fixed throughout each forward simulation.
Method
Forward Eval
Acceptance rate
τint
ESS % ( p^ess )
FE
2.1 min
58.3 %
25.3
3.953
vBNO
1.52 s
43.4 %
48.0
2.083
RI-BNO
1.67 s
57.2 %
27.1
3.691
Appendix
Table S7: Comparison of MCMC sampling performance metrics across FE, vBNO, and RI-BNO forward solvers, including wall clock time per iteration for forward evaluation, acceptance rate, integrated autocorrelation time ( τint ), and effective sample size percentage ( ESS% ).
Method
Effective Sampling Speed ( Seff )
Speedup vs. FE
FE
3.14×10−4 ESS/s
1 ×
vBNO
0.01370 ESS/s
43.6 ×
RI-BNO
0.02211 ESS/s
70.4 ×
Appendix
Table S8: Comparison of effective sampling speed Seff and relative online speedup for MCMC inference using FE, vBNO, and RI-BNO forward models.
Metric
FE
vBNO
RI-BNO
Wall time(MAP optimization)
134.2 min
21 s
12.9 s
Wall time(MCMC)
175.1 hrs
145.2 min
135.6 min
Appendix
Table S9: Wall clock time comparison for MAP optimization and MCMC inference across FE, vBNO, and RI-BNO.
Figure S6: Comparison of FE solutions (top row) and the preliminary neural operator predictions (middle row) for the state u , and auxiliary fields u− and n , alongside pointwise relative error fields (bottom row) for three representative test samples of m=(logD,logG) corresponding to panels (a)–(c). The rightmost column displays the error between FE and preliminary NO prediction eFE and the residual estimated error eest using the preliminary NO predictions. The preliminary neural operators were trained on Dpre .
Figure S7: MRI-informed tumor reconstruction using pretrained neural operators across four additional patients. Rows correspond to BraTS 2019 cases (a) BraTS19_TCIA10_413_1, (b) BraTS19_TCIA10_266_1, (c) BraTS19_TCIA10_351_1, and (d) BraTS19_TCIA10_276_1. Columns show the axial T1-CE MRI with its annotated tumor core and the tumor inversion obtained using FE, vBNO and RI-BNO, evaluated at the parameter MAP estimates from the corresponding inverse problems. The same pretrained neural operators are reused for all patients.
Method
Q
VaR0.95[Q]
CVaR0.95[Q]
rQ
VaR0.95[rQ]
CVaR0.95[rQ]
FE
1.971
2.015
2.031
-0.150
-0.131
-0.125
vBNO
2.379
2.437
2.459
0.025
0.051
0.061
RI-BNO
2.024
2.068
2.088
-0.127
-0.108
-0.101
Appendix
Table S10: Comparison of statistical risk measures (expected value, 95% VaR, and 95% CVaR) for heat leakage Q(W2/m) and insulation risk rQ evaluated across 5,000 Monte Carlo realizations. The optimal designs were generated using the FE, vBNO, and RI-BNO frameworks, while all performance metrics were evaluated using the FE model.
Method
g
VaR0.95[g]
CVaR0.95[g]
rT
VaR0.95[rT]
CVaR0.95[rT]
FE
-0.161
-0.128
-0.121
-0.041
-0.033
-0.031
vBNO
-0.433
-0.398
-0.391
-0.111
-0.102
-0.100
RI-BNO
-0.190
-0.162
-0.156
-0.049
-0.042
-0.040
Appendix
Table S11: Comparison of statistical risk measures (expected value, 95% VaR, and 95% CVaR) for stress violation g(MPa) and mechanical risk rT evaluated across 5,000 Monte Carlo realizations for full FE, vBNO, and RI-BNO models. The optimal designs were generated using the FE, vBNO, and RI-BNO frameworks, while all performance metrics were evaluated using the FE model.
Parameter
Description
Value
Reference
Kf
thermal conductivity of fiber phase
0.066 Wm−1K−1
[ 50 ]
Ks
thermal conductivity of solid phase
0.5 Wm−1K−1
[ 82 ]
Kg
thermal conductivity of gaseous phase
0.08 Wm−1K−1
[ 82 ]
λf
lamé first parameter for fiber phase
5.77 MPa
[ 50 ]
μf
lamé second parameter for fiber phase
3.84 MPa
[ 50 ]
λs
lamé first parameter for solid phase
0.7 MPa
[ 82 ]
Appendix
Table S12: Summary of physical parameters used in the three phase thermomechanical model of wheat straw aerogel composites.
Figure S8: Comparison of optimal designs and FE-evaluated fields : (a) Optimal design dopt(x) obtained via FE, vBNO, and RI-BNO ; (b) through thickness heat flux magnitude ∣q⋅eth∣ and von Mises stress TVM fields for manufacturing realization mA ; and (c) corresponding field predictions for manufacturing realization mB . Both fields shown in (b) and (c) are evaluated using FE.
Figure S9: Realizations of the training data for input ϕf and the six output states which are solid temperature θs , gas temperature θg , fiber temperature θf , pore gas pressure p , solid displacement us and fiber displacement uf shown for two different samples in (a) and (b). The displacement states are visualized using their magnitudes, ∣us∣ and ∣uf∣ .
Method
Data
Residual
Training
Optimization
End-to-end
vBNO
4795.7 min
—
54.4 min
16.7 min
4866.9 min
RI-BNO
2397.9 min
1558.2 min
47.6 min
17.8 min
4021.5 min
FE
—
—
—
10688.2 min
10688.2 min
Appendix
Table S13: Breakdown of wall clock computational time (in minutes) across offline data generation, residual evaluation, model training, and online risk-averse optimization stages for vBNO, RI-BNO, and FE baselines.
Method
Observable evaluation
Gradient
Total
vBNO
0.066 s
0.098 s
0.164 s
RI-BNO
0.071 s
0.105 s
0.176 s
Appendix
Table S14: Average wall-clock time per optimization iteration (in seconds) for risk-averse optimization with vBNO and RI-BNO models. Each iteration consists of a CVaR objective evaluation and its corresponding gradient computation using 5000 samples of m∼νman . The reported times are averaged over all optimization iterations.
Neural operators are widely used as surrogate solution maps for partial differential equations (PDEs), but full-size models can be costly to store, deploy, and evaluate in many-query scientific workflows. This work introduces Operator Boosting, a stagewise residual-learning framework for constructing compact neural-operator surrogates directly, rather than training a large model and compressing it afterward. Starting from the empirical mean predictor in normalized output coordinates, the method trains a sequence of tiny same-family neural operators on residual fields and incorporates each correction through validation-selected shrinkage. We instantiate the framework with Fourier neural operators (FNOs), DeepONets, and convolutional neural operators (CNOs), and compare boosted tiny stacks against full-size monolithic baselines across one-, two-, and three-dimensional PDE benchmarks from PDEBench, APEBench, and The Well. Across 30 dataset-architecture pairs, 21 show positive mean accuracy gains and 17 have positive confidence intervals, while all boosted stacks reduce trainable parameter count by approximately 72-95%. Best-model comparisons show empirical Pareto improvements on 7 of 10 completed PDE benchmarks, including two-dimensional Navier-Stokes, shallow-water dynamics, Darcy flow, one-dimensional transport and reaction systems, and three-dimensional compressible Navier-Stokes. These results show that Operator Boosting often improves the empirical accuracy-parameter Pareto frontier of neural PDE surrogates, while also exposing PDE- and architecture-dependent regimes where residual boosting fails to offset compression.
Lennon J. Shikhman
Georgia Institute of Technology College of Computing Atlanta, Georgia, USA
Physics-informed operator learning trains neural operators on the residual of a parametric partial differential equation (PDE) instead of labeled solution data, but its reliability depends on a variationally correct residual: one that is equivalent to the error in a norm dictated by the equation, the PDE-compliant norm. For nonlinear equations written as first-order systems, we prove that the first-order system least-squares residual is equivalent to the error in this norm near a solution branch if the linearized residual is stable and the residual derivative is Lipschitz continuous, possibly in a norm stronger than the error norm. We verify these conditions for monotone reaction, solution- and gradient-dependent diffusion, and compressible Neo-Hookean hyperelasticity. The stability mechanisms vary across these classes, as do the regions where the equivalence holds, from arbitrary bounded sets to a neighborhood of the branch. In a conforming finite element space, we show that the loss excess of a state near the finite element solution is equivalent to its squared distance from that solution. We use this loss to train a reduced basis neural operator with values in this space, with solution snapshots needed only for the basis and the loss and its symbolically derived gradient assembled by the finite element library. At inference, this loss estimates the error. Numerically, the square root of the loss excess tracks the error within factors comparable to those of the linear case, and the operator achieves higher accuracy in the PDE-compliant norm than grid-based baselines evaluated in the same finite element space.
Yuan Qiu, Wolfgang Dahmen, Peng Chen
School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA · Department of Mathematics, University of South Carolina, Columbia, SC 29208, USA
Neural operators evaluate parametric partial differential equations cheaply but degrade sharply outside their training distribution. Physics-informed neural networks avoid dependence on labeled data, yet their optimization can be basin-fragile: when the governing residual admits multiple solutions, a PINN trained from scratch may converge to a physically incorrect state despite achieving a small residual. We show that these failure modes can be addressed jointly: an imperfect NO provides the structural prior needed to place a PINN in the correct solution basin, while the PDE residual refines the solution beyond the operator's accuracy. We introduce a three-stage framework that freezes the spatial basis of a physics-informed NO, extrapolates its solution branch to an out-of-distribution parameter using a polynomial continuation prior, and distills the resulting field into a fresh PINN. The NO need not be accurate at the target; it transfers solution-branch information, while PDE residual minimization in the PINN governs convergence. We evaluate the framework on three nonlinear PDEs: 1D viscous Burgers, 2D steady Allen-Cahn near a pitchfork bifurcation, and 2D steady lid-driven cavity flow. For Allen-Cahn, where the trivial solution satisfies the PDE residual exactly, a standard PINN collapses to the trivial zero branch, whereas distillation from the crude extrapolated operator recovers the non-trivial branch that matches the finite-difference reference. For the lid-driven cavity, extrapolating to a Reynolds number of Re = 3200 accelerates convergence to the correct physical state, achieving competitive accuracy using fewer parameters and optimization steps than recent literature baselines. These results establish a simple principle: an NO need not accurately predict the solution to be useful; it only needs to identify the correct basin from which PINN optimization can recover it.
S. Mohammad Mousavi, Teeratorn Kadeethum, Nikolaos Bouklas +1
Sibley School of Mechanical and Aerospace Engineering, Cornell University · AI Lab, Siemens Energy · Department of Civil and Systems Engineering, Johns Hopkins University