Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Neural operators (NOs) provide fast, resolution-invariant surrogates for mapping input fields to PDE solution fields, but their predictions can exhibit significant epistemic uncertainty due to finite data, imperfect optimization, and distribution shift. For practical deployment in scientific computing, uncertainty quantification (UQ) must be both computationally efficient and spatially faithful, i.e., uncertainty bands should align with the localized residual structures that matter for downstream risk management. We propose a structure-aware epistemic UQ scheme that exploits the modular anatomy common to modern NOs (lifting-propagation-recovering). Instead of applying unstructured weight perturbations (e.g., naive dropout) across the entire network, we restrict Monte Carlo sampling to a module-aligned subspace by injecting stochasticity only into the lifting module, and treat the learned solver dynamics (propagation and recovery) as deterministic. We instantiate this principle with two lightweight lifting-level perturbations, including channel-wise multiplicative feature dropout and a Gaussian feature perturbation with matched variance, followed by standard calibration to construct uncertainty bands. Experiments on challenging PDE benchmarks (including discontinuous-coefficient Darcy flow and geometry-shifted 3D car CFD surrogates) demonstrate that the proposed structure-aware design yields more reliable coverage, tighter bands, and improved residual-uncertainty alignment compared with common baselines, while remaining practical in runtime.
Figures & tables
Method
Training Runs
Inference Runs
Deep Ensembles [ 9 ]
10
1
Laplace Approximation [ 13 ]
1
–
MC Dropout [ 2 ]
1
10 – 1000
Table 1: Comparison of common epistemic UQ methods in neural operators.
Figure 1: Overview of neural operator architectures and our structure-aware uncertainty quantification (UQ) scheme.
Figure 2: Sampling from other parameter groups can lead to uninformative and low-quality samples.
Figure 3: Visualizations of the residual and uncertainty band fields. Compared with other methods, our approach provides a more accurate characterization of the residual fields with reliable coverage.
Figure 4: Comprehensive comparisons with dropout-based baselines. Notably, for each choice of p and T , MC Dropout must be recalibrated since the uncertainty bands are altered by the newly introduced stochasticity, whereas our method A can share a single constant calibration scale across all p and T settings. Note that, to obtain the case-wise Avg. B.W., one should multiply the reported values by the grid size N , which typically ranges from a few hundred to tens of thousands.
Figure 5: Demonstrations of sampling on the Lifting parameters v.s. Sampling on/with other additional choices
Figure 6: Naive MCDropout uncertainty for different dropout probabilities p and sample counts T . Higher p causes more biased predictions and lower coverage of the calibrated uncertainty bands.
Figure 7: Naive MCDropout uncertainty for varying dropout probability p and sample counts T . Relative to the smaller- T results in Figure 6 , increasing T yields a smoother, re-distributed uncertainty-band field.
Figure 8: Naive MCDropout uncertainty for varying dropout probability p with a relatively large sample count T=10,000 . The near-identical predictive means, uncertainty bands, and coverage indicate that further increasing T has a negligible impact on the estimated mean and uncertainty band.
Figure 9: Quantified uncertainty visualizations of the A-implementation of our method for varying dropout probabilities p and sample counts T . With an appropriate dropout rate (i.e., neither extremely small nor large), our approach yields consistently far less biased predictions and higher coverage with calibrated uncertainty bands than the naive MCDropout results in Figure 6 . Notably, the calibration scale k is shared across all p values in our method A, whereas naive MCDropout typically requires case-by-case tuning of k .
Figure 10: Quantified uncertainty visualizations of the A-implementation of our method for varying dropout probabilities p and sample counts T . An appropriate dropout rate (i.e., neither extremely small nor large) provides reliable uncertainty quantification even with small T shown in Figure 9 . Increasing T to 100 or 1000 offers little benefit for normal p but helps reduce under-coverage when the dropout probability is very small ( p=0.01 ).
Figure 11: Quantified uncertainty visualizations of the A-implementation of our method for varying dropout probabilities p and sample counts T . Compared with the previously small T cases, further increasing T to 10000 has fewer benefits on coverage. However, compared with the uncertainty bands in Figure 8 , the A-implementation of our method provides a more faithful estimation of the residual field and achieves consistently much higher coverage.
Figure 12: UQ visualizations of our method B on the 3D ShapeNet Car with C.R.: 0.9162 ( All ) / 0.9248 ( Press ) / 0.9896 ( Velocity ) and Normalized Avg. B.W.: 17.530033 ( All ) / 0.970439 ( Press ) / 67.208816 ( Velocity ).
Neural operators such as the Fourier Neural Operator (FNO) have emerged as powerful surrogates for solving partial differential equations (PDEs), achieving speedups of several orders of magnitude over traditional numerical solvers. However, deploying these models in safety-critical engineering applications -- such as thermal management of electronic components and battery systems -- requires not only accurate point predictions but also rigorous uncertainty guarantees. Existing uncertainty quantification (UQ) methods for neural operators, including Monte Carlo Dropout and Deep Ensembles, provide only relative uncertainty estimates without formal coverage guarantees. In this work, we propose the first application of split conformal prediction to neural operator-based physics simulation, providing distribution-free prediction intervals with finite-sample coverage guarantees. We further introduce a normalized conformal prediction scheme that leverages MC Dropout uncertainty to produce adaptive-width intervals, yielding tighter intervals in regions of low uncertainty and wider intervals where the model is less certain. Full-scale experiments (33.7M parameters, 800 training samples, 5 ensemble members, NVIDIA V100) on steady-state heat conduction benchmarks demonstrate that our method achieves 89.1% empirical coverage at the target level of alpha=0.1, while producing spatially adaptive prediction intervals that reflect the underlying physical uncertainty structure. We also provide an uncertainty decomposition framework that separates epistemic uncertainty (68% of total) from aleatoric uncertainty (32% of total), offering actionable guidance for data collection and model improvement. Our method is implemented in an open-source platform with REST API endpoints and interactive 3D visualization.
Neural operators provide fast surrogates for PDEs but their deterministic predictions limit their use in tasks requiring uncertainty quantification (UQ), especially under geometric variability. Existing approaches primarily model uncertainty in network parameters, largely overlooking the geometry-aware representations learned by the operator itself. We propose REEF-GP (Residual on Embedded Features Gaussian Process), a post-hoc UQ framework that fits a GP to the residuals of a frozen neural operator whose internal embeddings define the kernel feature space. Rather than learning a separate feature map, REEF-GP adapts the operator's intrinsic coordinate-feature representations to construct geometry-aware uncertainties. To ensure stability and scalability on unstructured domains, REEF-GP incorporates spectral-normalized projections, heteroscedastic geometry-aware noise, and efficient subset-based training that avoids restrictive low-rank approximations. Across five PDE benchmarks with varying geometries, REEF-GP preserves predictive accuracy while achieving calibrated uncertainty estimates competitive with deep ensembles but at a fraction of their cost. Our approach remains robust under geometric distribution shift, with uncertainty concentrating in physically meaningful regions (e.g., shock fronts). Our results demonstrate that accurate and scalable post-hoc UQ for neural operators can be achieved directly in their learned feature space, offering a practical alternative to parameter-centric approaches.
Oriol Vendrell-Gallart, Nima Negarandeh, Ramin Bostanabad
Department of Mechanical and Aerospace Engineering University of California, Irvine Irvine, CA 92617
Neural operators are widely used as surrogate solution maps for partial differential equations (PDEs), but full-size models can be costly to store, deploy, and evaluate in many-query scientific workflows. This work introduces Operator Boosting, a stagewise residual-learning framework for constructing compact neural-operator surrogates directly, rather than training a large model and compressing it afterward. Starting from the empirical mean predictor in normalized output coordinates, the method trains a sequence of tiny same-family neural operators on residual fields and incorporates each correction through validation-selected shrinkage. We instantiate the framework with Fourier neural operators (FNOs), DeepONets, and convolutional neural operators (CNOs), and compare boosted tiny stacks against full-size monolithic baselines across one-, two-, and three-dimensional PDE benchmarks from PDEBench, APEBench, and The Well. Across 30 dataset-architecture pairs, 21 show positive mean accuracy gains and 17 have positive confidence intervals, while all boosted stacks reduce trainable parameter count by approximately 72-95%. Best-model comparisons show empirical Pareto improvements on 7 of 10 completed PDE benchmarks, including two-dimensional Navier-Stokes, shallow-water dynamics, Darcy flow, one-dimensional transport and reaction systems, and three-dimensional compressible Navier-Stokes. These results show that Operator Boosting often improves the empirical accuracy-parameter Pareto frontier of neural PDE surrogates, while also exposing PDE- and architecture-dependent regimes where residual boosting fails to offset compression.
Lennon J. Shikhman
Georgia Institute of Technology College of Computing Atlanta, Georgia, USA