Operator Surrogates

Christoph Schwab; Josephine Westermann; Jakob Zech

cs.LGOct 13, 2026

Operator Surrogates

Authors: Christoph Schwab, Josephine Westermann, Jakob Zech

Organizations: SAM, ETH Zürich, Rämistrasse 101, 8092 Zürich, Switzerland · IWR, Uni Heidelberg, INF 205, 69120 Heidelberg, Germany

Abstract

This chapter provides a mathematical introduction to operator surrogates, which are finite-parametric maps approximating complex (typically nonlinear) operators between infinite-dimensional (function) spaces. A typical approach to constructing operator surrogates is the encoder-approximator-decoder paradigm, whereby infinite-dimensional inputs are mapped to a finite-dimensional latent space, processed, and then mapped back to the infinite-dimensional output space. We discuss various methods for creating efficient representation systems, including stable linear frameworks like Banach frames and data-driven techniques such as Principal Component Analysis (PCA), Active Subspaces, and nonlinear autoencoders. Subsequently, we survey key surrogate architectures, contrasting spectral methods, based on polynomial chaos expansions and related technologies, with more recent neural network approaches. We further discuss architectures that do not rely on representation systems, such as Fourier Neural Operators and transformers. We elaborate on their relationship and how they relate to other neural network operator surrogates, particularly those obtained by so-called "algorithm-unrolling". Additionally, we provide an introduction to the theoretical analysis of approximation and generalization errors. We explain how underlying low-dimensional structures, provided for example by holomorphy, are essential to overcome the so-called "curse of dimensionality": this refers to the fact that the complexity of numerical algorithms tends to increase exponentially in the problem dimensionality. We present convergence rate bounds for holomorphic operators that depend on the number of trainable parameters and the regularity of the input and output data. Throughout, we prioritize intuition over detailed technical proofs and provide precise pointers to the literature for detailed arguments.

Figures & tables

Figure 1: Continuous, piecewise linear wavelet basis of

{\mathcal{X}}=H^{1}_{\{0\}}(0,1)

on subdivision level

j=2

I=(0,1)

with two vanishing moments and a homogeneous Dirichlet boundary condition in

t=0

(imposed only on the left boundary wavelet). See the appendix of [ 65 ] for details and [ 146 , 33 , 113 ] for other, spline- and Finite-Element like constructions.

Explore similar work

Apr 1, 2026

Performance of Neural and Polynomial Operator Surrogates

We consider the problem of constructing surrogate operators for parameter-to-solution maps arising from parametric partial differential equations, where repeated forward model evaluations are computationally expensive. We present a systematic empirical comparison of neural operator surrogates, including a reduced-basis neural operator trained with

L^2_μ

and

H^1_μ

objectives and the Fourier neural operator, against polynomial surrogate methods, specifically a reduced-basis sparse-grid surrogate and a reduced-basis tensor-train surrogate. All methods are evaluated on a linear parametric diffusion problem and a nonlinear parametric hyperelasticity problem, using input fields with algebraically decaying spectral coefficients at varying rates of decay

s

. To enable fair comparisons, we analyze ensembles of surrogate models generated by varying hyperparameters and compare the resulting Pareto frontiers of cost versus approximation accuracy, decomposing cost into contributions from data generation, setup, and evaluation. Our results show that no single method is universally superior. Polynomial surrogates achieve substantially better data efficiency for smooth input fields (

s \geq 2

), with convergence rates for the sparse-grid surrogate in agreement with theoretical predictions. For rough inputs (

s \leq 1

), the Fourier neural operator displays the fastest convergence rates. Derivative-informed training consistently improves data efficiency over standard

L^2_μ

training, providing a competitive alternative for rough inputs in the low-data regime when Jacobian information is available at reasonable cost. These findings highlight the importance of matching the surrogate methodology to the regularity of the problem as well as accuracy demands and computational constraints of the application.

Josephine Westermann, Benno Huber, Thomas O'Leary-Roseberry +1

Heidelberg University · The Ohio State University

Surrogate Modeling PDE Surrogate Modeling Neural Operators Fourier Neural Operator

Ask PDF

Jun 16, 2026

Operator Boosting Produces Pareto-Efficient PDE Surrogates

Neural operators are widely used as surrogate solution maps for partial differential equations (PDEs), but full-size models can be costly to store, deploy, and evaluate in many-query scientific workflows. This work introduces Operator Boosting, a stagewise residual-learning framework for constructing compact neural-operator surrogates directly, rather than training a large model and compressing it afterward. Starting from the empirical mean predictor in normalized output coordinates, the method trains a sequence of tiny same-family neural operators on residual fields and incorporates each correction through validation-selected shrinkage. We instantiate the framework with Fourier neural operators (FNOs), DeepONets, and convolutional neural operators (CNOs), and compare boosted tiny stacks against full-size monolithic baselines across one-, two-, and three-dimensional PDE benchmarks from PDEBench, APEBench, and The Well. Across 30 dataset-architecture pairs, 21 show positive mean accuracy gains and 17 have positive confidence intervals, while all boosted stacks reduce trainable parameter count by approximately 72-95%. Best-model comparisons show empirical Pareto improvements on 7 of 10 completed PDE benchmarks, including two-dimensional Navier-Stokes, shallow-water dynamics, Darcy flow, one-dimensional transport and reaction systems, and three-dimensional compressible Navier-Stokes. These results show that Operator Boosting often improves the empirical accuracy-parameter Pareto frontier of neural PDE surrogates, while also exposing PDE- and architecture-dependent regimes where residual boosting fails to offset compression.

Lennon J. Shikhman

Georgia Institute of Technology College of Computing Atlanta, Georgia, USA

PDE Surrogate Modeling Deep Operator Networks Neural Network Compression Neural Operators

Ask PDF

Apr 20, 2026

Neural Shape Operator Surrogates -- Expression Rate Bounds

We prove error bounds for operator surrogates of solution operators for partial differential and boundary integral equations on families of domains which are diffeomorphic to one common reference (or latent) domain

D_{ref}

. The pullback of the PDE to

D_{ref}

via affine-parametric shape encoding produces a collection of holomorphic parametric PDEs on

D_{ref}

. Sufficient conditions for (uniformly with respect to the parameter) well-posedness are given, implying existence, uniqueness and stability of parametric solution families on

D_{ref}

. We illustrate the abstract hypotheses by reviewing recent holomorphy results for a suite of elliptic and parabolic PDEs. Quantified parametric holomorphy implies existence of finite-parametric, discrete approximations of the parametric solution families with convergence rates in terms of the number

N

of parameters. We obtain constructive proofs of existence of Neural and Spectral Operator surrogates for the shape-to-solution maps with error bounds and convergence rate guarantees uniform on the collection of admissible shapes. We admit principal-component shape encoders and frame decoders. Our results support in particular the (empirically reported) ability of neural operators to realize data-to-solution maps for elliptic and parabolic PDEs and BIEs that generalize across parametric families of shapes.

Helmut Harbrecht, Christoph Schwab

PDE Operator Learning Neural Network Approximation Theory Operator Learning Neural Operators

Ask PDF