Learning inadequacy of Bayesian neural operators for PDE-constrained inference and design
Authors: Pratyush Kumar Singh, Lianghao Cao, E. Bruce Pitman, Danial Faghihi
Organizations: Department of Mechanical and Aerospace Engineering, University at Buffalo, Buffalo, NY, USA · Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA · Department of Materials Design and Innovation, University at Buffalo, Buffalo, NY, USA
Partial differential equations (PDEs) underpin predictive modeling across science, engineering and medicine. Their use in digital twins, parameter inference and optimal design requires repeated solutions across parameters and scenarios. For nonlinear, transient and multiphysics systems, these calculations can make numerical simulation prohibitively expensive. Here we introduce the residual-informed Bayesian neural operator (RI-BNO), which learns operator inadequacy using a posteriori error estimates from the governing PDEs and incorporates its correction and uncertainty into Bayesian training and prediction. We establish local error bounds for the residual-informed estimates under stability and smoothness assumptions. We demonstrate RI-BNO on a non-linear reaction-diffusion benchmark with DeepONet and latent-subspace architectures, Markov chain Monte Carlo inference of glioma parameter fields from magnetic resonance imaging, and Monte Carlo risk-averse optimization of three-dimensional thermal breaks in building envelopes under manufacturing uncertainty. Across the evaluated examples, RI-BNO improves agreement with finite element (FE) reference predictions, parameter posteriors and optimized designs while using half as many full nonlinear or transient training simulations as the vanilla Bayesian neural operator (vBNO). For thermal break design, online optimization time decreases from 178 hours with the FE model to 18 minutes with RI-BNO. These results demonstrate the value of explicit inadequacy modeling in neural operators for reliable PDE-based inference and design.
Figures & tables
Figure 1: Residual-informed Bayesian neural operator (RI-BNO) workflow. A preliminary neural operator trained on Dpre supplies predictions at additional, independently sampled inputs. Solving ( 6 ) generate residual-informed observations E , from which the inadequacy model η is learned. This model is held fixed during final Bayesian training on the disjoint solution dataset Dfit . The learned inadequacy informs both the Bayesian likelihood and the predictive state.
Figure 2: Residual-informed Bayesian neural operators for learning solution operator of a nonlinear reaction–diffusion PDE. Panel (a) shows predictions for two unseen in-distribution parameter fields, m1 and m2 . For each input, the finite element (FE) state uh is compared with the state predictions of vBNO, uvBNO=FwMAP(m) , and RI-BNO, uRI=FwMAP(m)+μη(m) , using the latent-subspace and DeepONet architectures. The pointwise relative error εvBNO=∣uh−uvBNO∣/∣uh∣ and εRI=∣uh−uRI∣/∣uh∣ , and posterior predictive state variance σvBNO2=Vπpost(w∣D)[Fw(m)] and σRI2=Vπpost(w∣D,E)[Fw(m)]+Cη are also presented. Panel (b) illustrates the DeepONet architecture, comprising branch and trunk networks, and the latent-subspace architecture, comprising an active-subspace input encoder, a ResNet map in latent coordinates and a POD state decoder. Panel (c) shows the inadequacy mean μη and pointwise marginal variance Cη of the sparse Gaussian process for each architecture. The mean provides the residual-informed correction, whereas the marginal variance quantifies uncertainty in that correction. Panel (d) compares, for the latent-subspace architecture, kernel density estimates of the components of the MAP weight vectors wMAP between vBNO and RI-BNO, together with the posterior marginal variances (Γpost)ii ordered by their variance rank. Their differences illustrate that the inadequacy-aware likelihood modifies the posterior of the weights in RI-BNO compared to vBNO.
Figure 3: Predictive accuracy and uncertainty of vBNO and RI-BNO using latent-subspace and DeepONet architectures : (a) Relative accuracy versus the total number of data points Ntot (b) Coefficient of Variation (CV), measuring relative predictive spread, versus Ntot .
Figure 4: Forward prediction and inadequacy representation for synthetic glioma growth. Panel (a) shows the finite element evolution of the tumor volume fraction ϕT(x,t) from the prescribed initial condition to the terminal time tF=250 days under the mean parameter fields m=(logD,logG) , overlaid on an axial slice of a healthy brain atlas [ 35 ] . Panel (b) shows the terminal tumor fields for representative unseen realizations of m from the finite element reference uh with the vBNO and RI-BNO state prediction uvBNO and uRI . The accompanying distributions, computed over 100 unseen realizations of m , show the normalized tumor area NTA=(∫Ωu(x)dx)/∣Ω∣ , and the normalized diffuse-interface area NIA=(∫Ω\mathbbm1{0.1<u(x)<0.9}dx)/∣Ω∣ , where u=ϕT(x,tF) . Panel (c) shows the inadequacy mean μη and pointwise marginal variance Cη of the learned neural-operator inadequacy. Panel (d) shows terminal tumor boundaries for one fixed test realization of m . Red denotes the finite element tumor region, green the state prediction u , and the 50 blue curves posterior tumor boundary realizations. For vBNO, the boundary realizations are obtained by sampling w∼πpost(w∣D) . For RI-BNO, w∼πpost(w∣D,E) is combined with a realization of the learned inadequacy field η . The box plot summarizes the Dice similarity coefficients between the predicted and finite element tumor regions over the same 100 test inputs, ranging from 0 for no overlap to 1 for perfect overlap.
Figure 5: Patient-specific MRI inference using pretrained neural operators. Panel (a) shows an axial T1-CE MRI slice from the BraTS 2019 high-grade glioma case BraTS19_CBICA_AQP_1, together with the segmented tumor-core mask used for the two-dimensional inversion. Panel (b) summarizes offline operator construction, including the latent-subspace architecture, allocation of the matched total observation budget, offline neural operator construction cost, and representative synthetic realizations of training data of the spatially varying mobility and proliferation fields, m=(logD,logG) , and their terminal tumor states, uh=ϕT(x,tF) . Panel (c) compares the finite-element, vBNO and RI-BNO tumor reconstructions and their Dice similarity coefficients relative to the MRI-derived tumor core mask. Panel (d) shows the corresponding MAP estimates of the spatially varying fields logD and logG . Panel (e) shows bands of posterior infiltration exceedance probability, computed from 250 retained MCMC samples using ϕinv=0.075 . The bands 0.1<Pinv<0.5 , 0.5<Pinv<0.9 , and Pinv>0.9 indicate increasing posterior support for subvisual tumor extension outside the MRI-visible tumor core boundary, defined using the MRI-detection threshold ϕ=0.2 . Panel (f) shows the posterior distributions of the DICE similarity coefficient and brain-normalized tumor area computed from 500 retained MCMC samples.
Figure 6: Risk-averse design of a fiber-reinforced aerogel thermal break under spatial manufacturing uncertainty. Panel (a) shows the thermal break inserted between hot (red) and cold (blue) steel beams, along with geometry and bolt-hole displacements. Panel (b) illustrates the latent-subspace NO architecture. The NO learns the mapping from fiber volume fraction ϕf to the coupled PDE state u , while the heat flux q and von Mises stress TVM are recovered through post-processing operators applied to u . The two rows present training realizations of ϕf and the corresponding through thickness heat flux magnitude ∣q⋅eth∣ and von Mises stress fields. Panel (c) compares the optimal designs and physical fields obtained using FE-constrained, vBNO-accelerated and RI-BNO-accelerated optimization. The rows show, from top to bottom, the optimal design dopt(x) , the through thickness heat flux magnitude with arrows showing direction and relative magnitude, and the von Mises stress. Panel (d) compares risk indicators (rQ,rT) under realizations m∼νman , with each optimized design evaluated using its corresponding forward model: FE, vBNO or RI-BNO. Small markers denote individual realizations, while large markers denote the marginal CVaR0.95 values. Panel (e) compares the FE optimization cost with the total construction and optimization costs of vBNO and RI-BNO, including nonlinear PDE data generation, residual-informed observation generation, operator training and online risk-averse optimization. The right plot reports the online optimization times, with speedups of 639× and 600× for vBNO and RI-BNO relative to FE, respectively.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Comparison of state predictions across frameworks (vBNO, RI-BNO, and FE-error BNO) against FE solution for an unseen in distribution parameter field m using the latent subspace architecture.
Method
Test Error (%) (using GP mean)
Test Error (%) (Final)
Latent Subspace RI-BNO
2.65
1.02
Latent Subspace FE-error BNO
3.53
1.83
Appendix
Table S1: Comparison of test errors for the latent subspace architecture evaluated using the preliminary operator with GP mean correction versus the full inadequacy-aware Bayesian inference.
Method
Test Error (%) (using GP mean)
Test Error (%) (Final)
DeepONet RI-BNO
2.34
0.75
DeepONet FE-error BNO
3.17
1.52
Appendix
Table S2: Comparison of test errors for the DeepONet architecture evaluated using the preliminary operator with GP mean correction versus the full inadequacy-aware Bayesian inference.
Figure S2: Comparison of state predictions across frameworks (vBNO, RI-BNO, and FE-error BNO) against FE solution for an unseen in-distribution parameter field m using the DeepONet architecture.
Method
Test Error (%) (Final)
MPIW
PICP (%)
Latent Subspace vBNO
10.5
1.05 e-2
84.2
Latent Subspace RI-BNO
1.02
1.61 e-2
94.7
Latent Subspace FE-error BNO
1.83
1.57 e-2
94.4
Appendix
Table S3: Test accuracy and UQ metrics for the latent subspace architecture, comparing vBNO, RI-BNO, and FE-error BNO. Test Error denotes the relative L2 error at the final time, while MPIW and PICP denote the mean prediction interval width and prediction interval coverage probability of the 95% predictive intervals, respectively.
Method
Test Error (%) (Final)
MPIW
PICP (%)
DeepONet vBNO
7.73
0.93e-2
86.3
DeepONet RI-BNO
0.75
1.52e-2
94.2
DeepONet FE-error BNO
1.52
1.46e-2
93.8
Appendix
Table S4: Test accuracy and UQ metrics for the DeepONet architecture, comparing vBNO, RI-BNO, and FE-error BNO. Test Error denotes the relative L2 error at the final time, while MPIW and PICP denote the mean prediction interval width and prediction interval coverage probability of the 95% predictive intervals, respectively.
Method
FE solution generation
Correction Observation
Preliminary training
GP fitting
Final training
Total time
vBNO
1535.7
—
—
—
1214.6
2750.3
RI-BNO
751.3
119.1
303.6
512.9
371.2
2058.1
FE-error BNO
752.4
807.9
308.5
504.4
378.3
2751.6
Appendix
Table S5: Computational cost (seconds) per phase for latent subspace architecture: (a) FE data generation, (b) correction observation generation, (c) preliminary BNO training, (d) GP fitting, (e) final training using inadequacy-aware likelihood, and (f) total time.
Figure S3: Comparison of computational costs for latent subspace architecture for vBNO, RI-BNO and FE-error BNO. The total wall clock time is partitioned into finite element (FE) solution generation, correction-observation generation, and model construction time across the vBNO, RI-BNO, and FE-error BNO frameworks.
Figure S4: Comparison of predicted tumor volume fraction spatial profiles between FE solution uh , and state predictions through vBNO uvBNO , and RI-BNO uRI along with their pointwise relative error fields with respect to FE across four unseen test samples of the input parameter field m=(logD,logG) .
Parameter
Description
Value
Reference
K
phase separation energy of tumor and host tissues
0.025
[ 77 ]
H
interfacial energy among tumor cells and other species
0.01
[ 77 ]
R
coefficient of nutrient consumption
10
[ 78 ]
M
nutrient mobility
1
[ 79 , 80 ]
Appendix
Table S6: Summary of physical parameters used in the multiphase glioma growth model.
Figure S5: FE tumor trajectories for offline training and verification of MRI-inferred parameter fields. (a) Representative synthetic training realization, showing the sampled mobility and proliferation fields, m=(logD(x),logG(x)) , and the corresponding FE tumor evolution from the prescribed training initial condition. The terminal field at tF=240 days is used for operator learning. (b) FE tumor trajectories generated using the parameter MAP estimates obtained from vBNO inversion (upper row) and RI-BNO inversion (lower row). Both trajectories use the same patient-specific initial condition adopted in FE-based MRI inversion [ 44 ] , with the inferred parameter fields held fixed throughout each forward simulation.
Method
Forward Eval
Acceptance rate
τint
ESS % ( p^ess )
FE
2.1 min
58.3 %
25.3
3.953
vBNO
1.52 s
43.4 %
48.0
2.083
RI-BNO
1.67 s
57.2 %
27.1
3.691
Appendix
Table S7: Comparison of MCMC sampling performance metrics across FE, vBNO, and RI-BNO forward solvers, including wall clock time per iteration for forward evaluation, acceptance rate, integrated autocorrelation time ( τint ), and effective sample size percentage ( ESS% ).
Method
Effective Sampling Speed ( Seff )
Speedup vs. FE
FE
3.14×10−4 ESS/s
1 ×
vBNO
0.01370 ESS/s
43.6 ×
RI-BNO
0.02211 ESS/s
70.4 ×
Appendix
Table S8: Comparison of effective sampling speed Seff and relative online speedup for MCMC inference using FE, vBNO, and RI-BNO forward models.
Metric
FE
vBNO
RI-BNO
Wall time(MAP optimization)
134.2 min
21 s
12.9 s
Wall time(MCMC)
175.1 hrs
145.2 min
135.6 min
Appendix
Table S9: Wall clock time comparison for MAP optimization and MCMC inference across FE, vBNO, and RI-BNO.
Figure S6: Comparison of FE solutions (top row) and the preliminary neural operator predictions (middle row) for the state u , and auxiliary fields u− and n , alongside pointwise relative error fields (bottom row) for three representative test samples of m=(logD,logG) corresponding to panels (a)–(c). The rightmost column displays the error between FE and preliminary NO prediction eFE and the residual estimated error eest using the preliminary NO predictions. The preliminary neural operators were trained on Dpre .
Figure S7: MRI-informed tumor reconstruction using pretrained neural operators across four additional patients. Rows correspond to BraTS 2019 cases (a) BraTS19_TCIA10_413_1, (b) BraTS19_TCIA10_266_1, (c) BraTS19_TCIA10_351_1, and (d) BraTS19_TCIA10_276_1. Columns show the axial T1-CE MRI with its annotated tumor core and the tumor inversion obtained using FE, vBNO and RI-BNO, evaluated at the parameter MAP estimates from the corresponding inverse problems. The same pretrained neural operators are reused for all patients.
Method
Q
VaR0.95[Q]
CVaR0.95[Q]
rQ
VaR0.95[rQ]
CVaR0.95[rQ]
FE
1.971
2.015
2.031
-0.150
-0.131
-0.125
vBNO
2.379
2.437
2.459
0.025
0.051
0.061
RI-BNO
2.024
2.068
2.088
-0.127
-0.108
-0.101
Appendix
Table S10: Comparison of statistical risk measures (expected value, 95% VaR, and 95% CVaR) for heat leakage Q(W2/m) and insulation risk rQ evaluated across 5,000 Monte Carlo realizations. The optimal designs were generated using the FE, vBNO, and RI-BNO frameworks, while all performance metrics were evaluated using the FE model.
Method
g
VaR0.95[g]
CVaR0.95[g]
rT
VaR0.95[rT]
CVaR0.95[rT]
FE
-0.161
-0.128
-0.121
-0.041
-0.033
-0.031
vBNO
-0.433
-0.398
-0.391
-0.111
-0.102
-0.100
RI-BNO
-0.190
-0.162
-0.156
-0.049
-0.042
-0.040
Appendix
Table S11: Comparison of statistical risk measures (expected value, 95% VaR, and 95% CVaR) for stress violation g(MPa) and mechanical risk rT evaluated across 5,000 Monte Carlo realizations for full FE, vBNO, and RI-BNO models. The optimal designs were generated using the FE, vBNO, and RI-BNO frameworks, while all performance metrics were evaluated using the FE model.
Parameter
Description
Value
Reference
Kf
thermal conductivity of fiber phase
0.066 Wm−1K−1
[ 50 ]
Ks
thermal conductivity of solid phase
0.5 Wm−1K−1
[ 82 ]
Kg
thermal conductivity of gaseous phase
0.08 Wm−1K−1
[ 82 ]
λf
lamé first parameter for fiber phase
5.77 MPa
[ 50 ]
μf
lamé second parameter for fiber phase
3.84 MPa
[ 50 ]
λs
lamé first parameter for solid phase
0.7 MPa
[ 82 ]
Appendix
Table S12: Summary of physical parameters used in the three phase thermomechanical model of wheat straw aerogel composites.
Figure S8: Comparison of optimal designs and FE-evaluated fields : (a) Optimal design dopt(x) obtained via FE, vBNO, and RI-BNO ; (b) through thickness heat flux magnitude ∣q⋅eth∣ and von Mises stress TVM fields for manufacturing realization mA ; and (c) corresponding field predictions for manufacturing realization mB . Both fields shown in (b) and (c) are evaluated using FE.
Figure S9: Realizations of the training data for input ϕf and the six output states which are solid temperature θs , gas temperature θg , fiber temperature θf , pore gas pressure p , solid displacement us and fiber displacement uf shown for two different samples in (a) and (b). The displacement states are visualized using their magnitudes, ∣us∣ and ∣uf∣ .
Method
Data
Residual
Training
Optimization
End-to-end
vBNO
4795.7 min
—
54.4 min
16.7 min
4866.9 min
RI-BNO
2397.9 min
1558.2 min
47.6 min
17.8 min
4021.5 min
FE
—
—
—
10688.2 min
10688.2 min
Appendix
Table S13: Breakdown of wall clock computational time (in minutes) across offline data generation, residual evaluation, model training, and online risk-averse optimization stages for vBNO, RI-BNO, and FE baselines.
Method
Observable evaluation
Gradient
Total
vBNO
0.066 s
0.098 s
0.164 s
RI-BNO
0.071 s
0.105 s
0.176 s
Appendix
Table S14: Average wall-clock time per optimization iteration (in seconds) for risk-averse optimization with vBNO and RI-BNO models. Each iteration consists of a CVaR objective evaluation and its corresponding gradient computation using 5000 samples of m∼νman . The reported times are averaged over all optimization iterations.
School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA · Department of Mathematics, University of South Carolina, Columbia, SC 29208, USA
Sibley School of Mechanical and Aerospace Engineering, Cornell University · AI Lab, Siemens Energy · Department of Civil and Systems Engineering, Johns Hopkins University