The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

Alexander Heinlein; Johannes Taraz

cs.LGFeb 25, 2026

The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

Authors: Alexander Heinlein, Johannes Taraz

Organizations: Delft Institute of Applied Mathematics, Delft University of Technology, Delft, 2628CD, The Netherlands

Abstract

Operator learning has the potential to strongly impact scientific computing by learning solution operators for differential equations, potentially accelerating multi-query tasks such as design optimization and uncertainty quantification by orders of magnitude. Despite proven universal approximation properties, deep operator networks (DeepONets) often exhibit limited accuracy and generalization in practice, which hinders their adoption. Understanding these limitations is therefore crucial for further advancing the approach. This work analyzes performance limitations of the classical DeepONet architecture. It is shown that the approximation error is dominated by the branch network when the internal dimension is sufficiently large, and that the learned trunk basis can often be replaced by classical basis functions without a significant impact on performance. To investigate this further, a modified DeepONet is constructed in which the trunk network is replaced by the left singular vectors of the training solution matrix. This modification yields several key insights. First, for examples involving the KdV and Burgers equations, a spectral bias in the branch network is observed, with coefficients of dominant, low-frequency modes learned more effectively. Second, through the interplay of the singular-value weighting and the optimizer's neglect of small modes, the branch error is dominated by modes with large and intermediate singular values. Third, using a shared branch network for all mode coefficients, as in the standard architecture, improves generalization of small modes compared to a stacked architecture in which coefficients are computed separately. Finally, detrimental coupling between modes in parameter space is identified, which weakens with increasing network width.

Figures & tables

Figure 1: Visualizations of the standard DeepONet (top left) and this work’s main contributions: the decomposition into trunk and branch error (top right) and the modified DeepONet’s architecture (bottom left) which allows us to further investigate the dominant branch error via the mode loss decomposition (bottom right). The base loss shown there is the loss of the trivial prediction

b_{i}=0

. For visual clarity, the bottom-right plot contains no shaded bands. The visualizations of the architectures are adapted and reproduced from [ 57 ] .

symbol	name	note
$Z^{+}$	Moore–Penrose inverse of matrix $Z$
$\text{ran}(Z)$	range / column space of matrix $Z$
$\|\|Z\|\|_{F}$	Frobenius norm	$\|\|Z\|\|_{F}\coloneqq\sqrt{\sum_{ij}Z_{ij}^{2}}$
$N$	number of output neurons of branch and trunk net (“inner dimension”)	$N\in\mathbb{N}$	eq. 2
$\mathcal{X}$	spatial domain of interest	$\mathcal{X}\subset\mathbb{R}^{d}$	section 3.2
$\mathcal{C}(\mathcal{X})$	set of continuous functions on $\mathcal{X}$

Table 1: Notation frequently used in this work.

Figure 2: Relative total, trunk and branch test errors of DeepONets with various bases plotted over

N

. Colors: Purple lines: learned trunk (standard DeepONet), orange: SVD basis, green: Legendre polynomials, pink: Chebyshev polynomials, brown: cosine. Line-symbols: Total error

\delta

(circle), trunk error

\delta_{T}

(dashed), branch error

\delta_{B}

(solid). The relative error is defined in the glossary, table 1 . Since the Legendre and the Chebyshev polynomials span the same space, they have identical trunk errors; thus only one curve is visible.

	matrix formulation		individual evaluation
true solution (train)	$\left(\Phi_{1}\Sigma_{1}V_{1}^{T}+\Phi_{2}\Sigma_{2}V_{2}^{T}\right)_{ij}$	$=$	$\sum_{k=1}^{r}\sigma_{k}(\phi_{k})_{i}(v_{k})_{j}$
true solution (test)	$\left(\Phi_{1}\Sigma_{1}W_{1}^{T}+\Phi_{2}\Sigma_{2}W_{2}^{T}\right)_{ij}$	$=$	$\sum_{k=1}^{r}\sigma_{k}(\phi_{k})_{i}(w_{k})_{j}$
stand. DeepONet	$\left(TB^{T}\right)_{ij}$	$=$	$\sum_{k=1}^{N}t_{k}(x_{i})b_{k}(p_{j})$
mod. DeepONet	$\left(\Phi_{1}\Sigma_{1}B^{T}\right)_{ij}$	$=$	$\sum_{k=1}^{N}\sigma_{k}(\phi_{k})_{i}b_{k}(p_{j})$

Table 2: Comparison of true solution (train/test), modified DeepONet and standard DeepONet in matrix formulation and for individual evaluation at coordinate

x_{i}

and initial condition

p_{j}

. The rank of the data matrix is denoted as

r

. For better comparison to the modified DeepONet, the SVD of the true solution is displayed. Here,

W:=(\Sigma^{+}\Phi^{T}A_{te})^{T}

contains the optimal coefficients for the test data; see appendix H for details.

Figure 3: Weighted mode losses for DeepONet trained with GD. Unweighted (A,B) and weighted (C,D) training (A,C) and test (B,D) mode losses at different training stages, colored from gray (initial) to red/blue (final) shown as dashed lines. The respective base losses are shown as black dot-solid lines. The terms large, intermediate, and small modes refer to the singular-value regimes defined in section 6.3 . For visual clarity, this figure contains no shaded bands.

Figure 4: Model performance across different exponents

e

and training epochs for DeepONets trained using GD. Top panel: Relative error

\delta=||A-\tilde{A}||_{F}/||A||_{F}

for both training (dashed lines) and test (circles) data across different exponents (

e=-1.0,-0.5,0.0,0.5,1.0

) over

4000

epochs. Center and bottom row: Weighted training (center row) and test (bottom row) mode losses at different training stages, colored from gray (initial) to red/blue (final). Each column corresponds to a different exponent

e

. The third column shows the DeepONet trained using the standard loss (

e=0

). The plots in the center and bottom row contain the respective base losses in black, and a pink dashed horizontal line marking the maximum mode loss in the last training epoch of

e=0

, facilitating comparison between different exponents

e

. For visual clarity, the training-loss curve in the top panel and the plots in the center and bottom rows have no shaded bands.

Figure 5: Weighted mode losses for DeepONet trained with Adam. Unweighted (A,B) and weighted (C,D) training (A,C) and test (B,D) mode losses at different training stages, colored from gray (initial) to red/blue (final) shown as dashed lines. The respective base losses are shown as black dot-solid lines. For visual clarity, this figure contains no shaded bands.

Figure 6: Spectral bias in modified DeepONet for synthetic data. Left: Relative error over epochs (training error as dashed line and test error as circles; nearly identical). Center/right: Unweighted mode losses (left scale) at different training stages (colored from black (initial) to red (final)) with frequencies of right singular functions (right scale). The center column shows the case of increasing frequencies (

\alpha=0.2

), and the right column shows the case of decreasing frequencies (

\alpha=-0.2

); the corresponding relative errors are shown in purple and orange respectively in the left column. Top/bottom: The top row shows synthetic data with singular values

\sigma_{j}=\exp(-0.01j)

, while the bottom row has

\sigma_{j}=\exp(-0.5j)

. Each curve is averaged over 20 randomly initialized DeepONets. For visual clarity, the center and right plots contain no shaded bands.

Figure 7: Spectral bias in modified DeepONet for different PDEs. Unweighted training (red) and test (blue) mode losses (left scale), the frequencies (right scale) of the right singular functions

\rho_{i}

estimated via the TV norm with

k=3

(green) and LE

k=50

(pink) and the scaled singular values

\sigma_{i}^{-0.2}

(black, right scale) are shown. Example problems (by column): advection-diffusion (AD) equation with

\tau=0.5

, KdV equation with

\tau=0.2

, KdV equation with

\tau=0.6

and Burgers’ equation with

\tau=0.1

Panel of Figure 8 — Figure 8: Stacked and unstacked DeepONets. (a) Architecture of the stacked modified DeepONet. Figure 1 (bottom left) shows the unstacked DeepONets for reference. Adapted and reproduced from [ 57 ] . (b) Relative errors for stacked and unstacked DeepONets trained with Adam. (c) Weighted mode losses for stacked and unstacked DeepONets trained with Adam. Weighted training (top row) and test (bottom row) mode losses at different training steps, colored from gray (initial) to red/blue (final). Columns correspond to different architectures and hidden layer widths (left: stacked with $w_{sta}=42$ , right: unstacked with $w_{unst}=495$ ), as indicated by the labels. The top and bottom row plots also contain the respective base losses in black, and a pink dashed horizontal line marking the maximum mode loss of the fully-trained unstacked DeepONet. For visual clarity, figure (c) contains no shaded bands.

Panel of Figure 9 — Figure 9: Update-based coupling for DeepONets of different hidden-layer widths.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

figure	example prob.	optimizer	$\mathbf{\alpha_{1}}$	depth	width	$\mathbf{N}$	number of epochs
1	KdV $\tau=0.2$	Adam	$2\times 10^{-3}$	$5$	$100$		$5000$
2	KdV $\tau=0.2$	Adam	$2\times 10^{-3}$	$5$	$100$		$5000$
3	KdV $\tau=0.2$	GD	$10^{-4}$	$5$	$335$	$50$	$4000$
4	KdV $\tau=0.2$	GD	var1	$5$	$335$	$50$	$4000$
5	KdV $\tau=0.2$	Adam	$10^{-4}$	$5$	$335$	$50$	$4000$
6	synth. data	Adam	$2\times 10^{-3}$	$5$	$50$	$5$	$2000$

Appendix

Table 3: Hyperparameters of DeepONets and SVDONets used in each figure. A cell is left empty if a figure shows multiple DeepONets and the corresponding hyperparameter values are apparent from the figure. The term var x denotes a varying value for the different DeepONets which is not apparent from the figure. Each var x is explained in the text of this section.

DeepONet	on-grid	off-grid
standard	$2.38\times 10^{-1}$	$2.40\times 10^{-1}$
modified	$6.84\times 10^{-2}$	$6.84\times 10^{-2}$

Appendix

Table 4: Relative test errors

\delta_{te}

on- and off-training-grid. The modified DeepONet’s basis vectors are interpolated linearly. Both DeepONets have

N=100

, depth

5

, width

500

, and approximate the KdV equation with

\tau=0.2

problem	$L$	$\omega_{i}$	$\tau$	solver	$n$	$M$	$\text{rank}(A_{tr})$
AD	20	$2\pi i$	0.5, 1.0	FD + RK [ 93 ]	200	200	20
KdV	5	$2\pi i$	0.2, 0.6, 1.0	FD + RK [ 93 ]	400	400	400
Burgers	5	$\pi i$	0.1, 1.0	spectral (100 basis) + Euler	200	50	100

Appendix

Table 5: Data generation parameters for the three example problems. Here,

L

is the number of input function modes,

\omega_{i}

the corresponding frequencies, and FD + RK denotes the finite-difference method combined with a Runge–Kutta method.

$N$	$\log_{10}\varepsilon_{te}$	$\log_{10}\varepsilon_{T,te}$	$\log_{10}\varepsilon_{B,te}$	$\log_{10}\varepsilon_{D,te}$	$\log_{10}(\varepsilon_{D,te}/\varepsilon_{B,te})$
2	4.31	4.31	0.43	0.61	0.18
10	3.26	3.24	2.05	3.34	1.30
18	2.86	2.61	2.5	4.29	1.79
50	2.59	1.62	2.54	8.29	5.75
90	2.63	-0.31	2.63	13.48	10.86

Appendix

Table 6: Total error, trunk error, branch error, upper bound on branch error

\varepsilon_{D}=||T||_{2}^{2}\varepsilon_{C}

, and the ratio

\varepsilon_{D}/\varepsilon_{B}

problem		$\log_{10}\delta_{T,tr}$	$\log_{10}\delta_{T,te}$	$\log_{10}\delta_{T,tr}$	$\log_{10}\delta_{T,te}$	$\log_{10}\delta_{T,tr}$	$\log_{10}\delta_{T,te}$
		$N=10$	$N=10$	$N=15$	$N=15$	$N=20$	$N=20$
AD	$\tau=0.5$	-0.41	-0.41	-0.78	-0.78	-12.6	-12.6
AD	$\tau=1.0$	-0.69	-0.69	-1.32	-1.32	-13.1	-13.1
		$N=30$	$N=30$	$N=50$	$N=50$	$N=70$	$N=70$
KdV	$\tau=0.2$	-1.48	-1.46	-2.19	-2.10	-2.94	-2.84
KdV	$\tau=0.6$	-1.04	-1.03	-1.90	-1.88	-2.71	-2.59

Appendix

Table 7: Relative training and test trunk errors

\delta_{T}

for all example problems (advection-diffusion (AD), Korteweg-de Vries (KdV) and Burgers) and various

N

Figure 10: Comparison of modified DeepONets with

T=\Phi_{1}

and

T=\Phi_{1}\Sigma_{1}

, trained with GD and Adam. Top / bottom block: DeepONets trained with GD / Adam. Left panel of each block: Relative error

\delta=||A-\tilde{A}||_{F}/||A||_{F}

for training (dashed lines) and test (circles) data over

10\,000

epochs, for

T=\Phi_{1}\Sigma_{1}

(purple) and

T=\Phi_{1}

(orange). Remaining panels: Training (upper row) and test (lower row) mode losses at different training stages, colored from gray (initial) to red/blue (final), with the respective base losses in black. Panels marked with a purple square show the DeepONet with

T=\Phi_{1}\Sigma_{1}

, those marked with an orange square the one with

T=\Phi_{1}

. The two center columns show the mode losses relative to the

\sigma_{j}^{2}

, while the two right columns show the absolute contributions of mode

j

to the error; the panels within each pair are thus directly comparable. The curves correspond to only one DeepONet; no averaging over multiple seeds was done.

dataset	family	parameters	$\tau$	$n=M$	$m_{tr}$ / $m_{te}$	$\text{rank}(A_{tr})$	$\text{rank}(P)$
sine	(a)	$K=8$	0.004	4096	4500 / 500	64	64
GRF	(b)	$\ell=0.05$	0.004	4096	4500 / 500	$\approx 300$	$\approx 2300$

Appendix

Table 8: Data generation parameters for the two-dimensional heat equation datasets. All datasets share the exact spectral solver. We report the rank for both the training data matrix

A_{tr}

and the matrix containing the input vectors

P\in\mathbb{R}^{M\times m_{tr}}

. For the sine datasets, these ranks are exact; for the GRF datasets, we report the numerical rank (singular values above

10^{-8}\sigma_{1}

Figure 11: Left: Singular values of 1D and 2D datasets. Center/right: Relative total, trunk and branch errors for the 2D heat equation plotted over

N

. The two panels show the errors for sine input functions (

K=8

, center) and GRFs (

\ell=0.05

, right). Colors: Purple: learned trunk (standard DeepONet), orange: SVD basis. Line-symbols: Total error

\delta

(circle), trunk error

\delta_{T}

(dashed), and branch error

\delta_{B}

(solid); see fig. 2 . For the sine dataset, the SVD trunk error vanishes for

N\geq\text{rank}(A_{tr})=64

. Each data point corresponds to one DeepONet; no averaging over multiple seeds was done.

Figure 12: Mode losses for the 2D heat equation. Unweighted (left) and weighted (right) training (red) and test (blue) mode losses at different training stages, colored from gray (initial) to red/blue (final), shown as dashed lines. The respective base losses are shown as black dot-solid lines. Rows: sine input functions (

K=8

) and GRFs with

\ell=0.05

. The curves correspond to only one DeepONet; no averaging over multiple seeds was done.

Figure 13: Model performance across different exponents

e

and training epochs for modified DeepONets trained using Adam. Layout and parameters identical to fig. 4 , with Adam replacing GD. Top panel: Relative error

\delta=||A-\tilde{A}||_{F}/||A||_{F}

for both training (dashed lines) and test (circles) data across different exponents (

e=-1.0,-0.5,0.0,0.5,1.0

) over

4000

epochs. Center and bottom row: Weighted training (center row) and test (bottom row) mode losses at different training steps, colored from gray (initial) to red/blue (final). Each column corresponds to a different exponent

e

. The third column shows the DeepONet trained using the standard loss (

e=0

). The center and bottom row plots also contain the respective base losses in black, and a pink dashed horizontal line marking the maximum mode loss in the last training epoch of

e=0

, facilitating comparison between different exponents

e

. For visual clarity, the training-loss curve in the top panel and the plots in the center and bottom rows have no shaded bands.

Figure 14: Model performance for DeepONets trained with different optimizers. Top panel: Relative error

\delta=||A-\tilde{A}||_{F}/||A||_{F}

for both training (dashed lines) and test (circles) data for different optimizers (purple: Adam, orange: AdaGrad, green: GD) over

4000

epochs. Center and bottom row: Weighted training (center row) and test (bottom row) mode losses at different training steps, colored from gray (initial) to red/blue (final). The center and bottom row plots also contain the respective base losses in black, and a pink dashed horizontal line marking the maximum mode loss in the last training epoch of the DeepONet trained with GD, facilitating comparison between different algorithms. For visual clarity, the training-loss curve in the top panel and the plots in the center and bottom rows have no shaded bands.

Figure 15: Estimated frequency of synthetic data. Columns: Different frequency families (synthetic datasets). Colors and markers: Pink: Dictated frequency

f_{j}=F_{0}\exp(\alpha j)

, Orange: TV norm (dot-solid:

k=3

, dashed:

k=20

, dash-dotted:

k=50

), Green: LE (dot-solid:

k=3

, dashed:

k=50

), Purple: projection (left-singular vectors of

X

, dot-solid:

k=1

, dashed:

k=5

Deep Operator Networks

Deep Operator Networks learn mappings between functions, enabling fast surrogates for parameterized differential equations and other complex systems.

See all 50 papers

FAST-DeepONet: Factor-Augmented Branch Representations for High-Dimensional PDE Inputs in the Small-Sample Regime
Aug 15, 2026Jiyong Kwon, Bongseok Kim, Guang LinPDE Operator Learning Deep Operator Networks
School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA · Department of Mathematics, Purdue University, 150 N. University Street, West Lafayette, 47907, IN, USA
Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximation
Dec 9, 2025Muhammad Abid, Omer SanPDE Surrogate Modeling PDE Operator Learning
Department of Mechanical and Aerospace Engineering, University of Tennessee, Knoxville, TN 37996, USA.
In-Context Multi-Operator Learning with DeepOSets
Dec 18, 2025Shao-Ting Chiu, Aditya Nambiar, Ali Syed +2Operator Learning In-Context Learning
Department of Electrical and Computer Engineering, Texas A&M University · Department of Mathematics, Texas A&M University

Aug 15, 2026J/K move · Enter open · S save

FAST-DeepONet: Factor-Augmented Branch Representations for High-Dimensional PDE Inputs in the Small-Sample Regime

Jiyong Kwon, Bongseok Kim, Guang Lin

School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA · Department of Mathematics, Purdue University, 150 N. University Street, West Lafayette, 47907, IN, USA

PDE Operator Learning Deep Operator Networks PDE Surrogate Modeling Operator Learning

Open paper AskPDF arXiv

Deep operator networks can become statistically unstable when partial differential equation inputs are observed at thousands of strongly correlated sensors but only a small number of operator samples is available. We introduce FAST-DeepONet, a branch representation combining a fixed spectral path with a regularized projection of the orthogonal residual, in which the directional penalty acts on the effective residual map after each of its rows is normalized. On Navier--Stokes flow a plain DeepONet degrades from

0.0394

0.1556

mean relative

L_2

error as the branch grows from

129

8193

coordinates, while FAST-DeepONet stays near

0.04

, so the sensor grid can be refined without a statistical penalty. Across independent test sets for Navier--Stokes flow, Darcy flow, and signed terminal wavefield prediction it lowers mean relative

L_2

error by

4.7\%

37.0\%

with three to seven times fewer trainable parameters. A spectral-only branch sharing the same basis separates the two paths: the fixed spectral path carries the improvement on Navier--Stokes and Darcy, while terminal wave prediction requires the residual path together with its directional penalty. FAST-DeepONet targets coordinate-query architectures and trains on solution values alone.

Abstract

Deep Operator Networks

Explore similar work

FAST-DeepONet: Factor-Augmented Branch Representations for High-Dimensional PDE Inputs in the Small-Sample Regime

Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximation

In-Context Multi-Operator Learning with DeepOSets