FAST-DeepONet: Factor-Augmented Branch Representations for High-Dimensional PDE Inputs in the Small-Sample Regime

Jiyong Kwon; Bongseok Kim; Guang Lin

cs.LGAug 15, 2026

FAST-DeepONet: Factor-Augmented Branch Representations for High-Dimensional PDE Inputs in the Small-Sample Regime

Authors: Jiyong Kwon, Bongseok Kim, Guang Lin

Organizations: School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA · Department of Mathematics, Purdue University, 150 N. University Street, West Lafayette, 47907, IN, USA

Abstract

Deep operator networks can become statistically unstable when partial differential equation inputs are observed at thousands of strongly correlated sensors but only a small number of operator samples is available. We introduce FAST-DeepONet, a branch representation combining a fixed spectral path with a regularized projection of the orthogonal residual, in which the directional penalty acts on the effective residual map after each of its rows is normalized. On Navier--Stokes flow a plain DeepONet degrades from $0.0394$ to $0.1556$ mean relative $L_2$ error as the branch grows from $129$ to $8193$ coordinates, while FAST-DeepONet stays near $0.04$ , so the sensor grid can be refined without a statistical penalty. Across independent test sets for Navier--Stokes flow, Darcy flow, and signed terminal wavefield prediction it lowers mean relative $L_2$ error by $4.7\%$ to $37.0\%$ with three to seven times fewer trainable parameters. A spectral-only branch sharing the same basis separates the two paths: the fixed spectral path carries the improvement on Navier--Stokes and Darcy, while terminal wave prediction requires the residual path together with its directional penalty. FAST-DeepONet targets coordinate-query architectures and trains on solution values alone.

Figures & tables

Figure 1: FAST-DeepONet architecture. A basis estimated from the training fields defines a fixed spectral path with scale $c_{f}$ and an orthogonal residual path. The fixed and learned features are concatenated and mapped to the branch output $b$ , while the trunk network maps an output query $\xi$ to $t$ . Their inner product $b^{\top}t+c$ gives the predicted field value at that query. The diagram uses $x$ , $Q$ , and $p$ for the field block. Standardized auxiliary parameters are appended to the fixed representation and excluded from the residual path. The dashed bracket denotes the directional penalty used only during training.

Figure 2: Representative inputs and outputs, selected without using model predictions or errors. Black contours mark the active disks in the two designed input distributions and are not supplied to the models. Each colorbar applies only to its panel.

Benchmark	$n_{\mathrm{tr}}$	$p$	Output grid	Rank	$s$	$(c_{f},\lambda)$
Navier–Stokes	60	8193	$128^{2}$	$8/6$	8	$(1,0.01)$
Darcy	60	4096	$64^{2}$	$16/16$	16	$(1,1)$
Terminal wave	40	4096	$64^{2}$	$6/6$	16	$(8,1)$

Table 1: Final benchmark and FAST representation settings. The rank column gives the requested and effective factor ranks. For Navier–Stokes, the effective representation contains five field directions and one separately standardized viscosity scalar. The wave training set is a fixed subset of a fresh

60

field pool.

Branch representation	Navier–Stokes	Darcy	Terminal wave
Plain DeepONet	$0.1776\pm 0.0057$	$0.1029\pm 0.0022$	$0.2165\pm 0.0030$
FAST-DeepONet	$\mathbf{0.1119\pm 0.0021}$	$\mathbf{0.0925\pm 0.0019}$	$\mathbf{0.2063\pm 0.0026}$

Table 2: Mean fieldwise relative

L_{2}

error on

100

independent test fields.

Figure 3: Plain and FAST test errors for the three PDE problems, where lower is better. Bars and whiskers show the mean and sample standard deviation across paired fits. Exact values are given in Table 2 .

Benchmark	Mean difference	Paired fit $95\%$ CI	Two-way bootstrap $95\%$ CI
Navier–Stokes	$-0.0657$	$[-0.0737,-0.0578]$	$[-0.0816,-0.0511]$
Darcy	$-0.0104$	$[-0.0143,-0.0064]$	$[-0.0159,-0.0048]$
Terminal wave	$-0.0103$	$[-0.0160,-0.0046]$	$[-0.0164,-0.0042]$

Table 3: Paired FAST minus Plain differences in mean relative

L_{2}

error. Negative values favor FAST. The bootstrap resamples fit and field indices while retaining model pairing.

Figure 4: Representative Plain and FAST reconstructions under the main training protocol. Each block shows the ground truth, both predictions, and absolute error maps, in which smaller values are better.

Figure 5: Sensor resolution and sample size on the three benchmarks. Top: mean test error, lower is better. Bottom: the FAST reduction against a plain branch in the same cell, higher is better, so a rising curve means the advantage grows with the branch dimension. Corner cells carry five paired seeds and a one standard deviation whisker; interior cells carry one seed. The designed benchmarks start at

16\times 16

; Appendix K explains why and Table K.1 keeps the

8\times 8

row.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Navier–Stokes	Darcy	Terminal wave
Data
Training fields	60	60	40 of 60
Tuning fields	10
Accuracy test fields	100
UQ calibration and test fields	20 and 100
Targets and queries

Appendix

Table A.1: Run settings for the final study. Entries spanning the three columns are shared by all benchmarks. Wave uses a fixed subset of a fresh training pool. Conformal calibration and UQ streams are independent of the accuracy test streams.

Benchmark	Numerical rank	Effective rank	$k_{95}$
Navier–Stokes	6	4.05	4
Darcy	31	5.05	9
Terminal wave	16	10.76	12

Appendix

Table A.2: Training input spectral diagnostics for the final datasets. These values describe the data and are distinct from the selected factor ranks in Table 1 .

Benchmark	Spectral only	FAST	Difference	Paired fit $95\%$ CI
Navier–Stokes	$0.1108\pm 0.0055$	$0.1119\pm 0.0021$	$0.0011$	$[-0.0033,0.0055]$
Darcy	$0.0876\pm 0.0026$	$0.0925\pm 0.0019$	$0.0048$	$[0.0005,0.0092]$
Terminal wave	$0.2557\pm 0.0016$	$0.2063\pm 0.0026$	$-0.0494$	$[-0.0535,-0.0453]$

Appendix

Table B.1: Spectral-only reference on the independent test sets. Errors are mean

\pm

sample standard deviation across paired fits. The difference is FAST minus the spectral-only reference, so negative values favor residual augmentation.

Benchmark	Plain	Plain, matched	Random basis	Spectral only	FAST
Navier–Stokes	$0.1776\pm 0.0057$	$0.2504\pm 0.0224$	$0.2731\pm 0.0314$	$0.1108\pm 0.0055$	$\mathbf{0.1119\pm 0.0021}$
Darcy	$0.1029\pm 0.0022$	$0.1234\pm 0.0038$	$0.1524\pm 0.0142$	$0.0876\pm 0.0026$	$\mathbf{0.0925\pm 0.0019}$
Terminal wave	$0.2165\pm 0.0030$	$0.2462\pm 0.0063$	$0.3748\pm 0.0164$	$0.2557\pm 0.0016$	$\mathbf{0.2063\pm 0.0026}$

Appendix

Table C.1: The two controls beside the reported representations, on the same evaluation streams and paired seeds. Entries are mean

\pm

sample standard deviation across fits. The matched plain widths and trainable counts are

20

(

166{,}741

) for Navier–Stokes,

40

(

174{,}481

) for Darcy, and

40

(

174{,}401

) for terminal wave, against FAST counts of

168{,}713

171{,}265

, and

169{,}729

Benchmark	$\lambda$	$\lambda=0$	Selected $\lambda$	Paired fit $95\%$ CI	Wins
Navier–Stokes	$0.01$	$0.1127\pm 0.0015$	$0.1119\pm 0.0021$	$[-0.0032,0.0016]$	4/5
Darcy	$1$	$0.1385\pm 0.0110$	$0.0925\pm 0.0019$	$[-0.0578,-0.0342]$	5/5
Terminal wave	$1$	$0.2572\pm 0.0079$	$0.2063\pm 0.0026$	$[-0.0627,-0.0391]$	5/5

Appendix

Table D.1: Effect of the directional penalty. Errors are mean

\pm

sample standard deviation across paired fits. The interval is for the selected

\lambda

minus

\lambda=0

, so negative values favor the penalty. The last column counts the fits in which the selected

\lambda

has the lower mean error.

	Darcy		Terminal wave
$c_{f}$	Tuning MSE	Test error	Tuning MSE	Test error
1	$\mathbf{0.0623}$	$0.0925$	$0.7331$	$0.3980$
2	$0.0689$	$0.0900$	$0.2784$	$0.2385$
4	$0.0693$	$0.0904$	$0.2325$	$0.2140$
8	$0.0708$	$0.0923$	$\mathbf{0.2168}$	$0.2063$

Appendix

Table E.1: Factor-scale sensitivity, averaged over paired fits. Tuning MSE is the criterion by which

c_{f}

was selected; its minimum in each benchmark is shown in bold. Test error is the mean fieldwise relative

L_{2}

on the evaluation stream.

Benchmark	$p$	Energy in top $1\%$ of columns	Participation ratio
Navier–Stokes	8193	$0.741\;[0.668,0.862]$	$161.0\;[125.2,202.0]$
Darcy	4096	$0.892\;[0.857,0.931]$	$56.8\;[50.5,60.9]$
Terminal wave	4096	$0.857\;[0.835,0.890]$	$74.3\;[72.1,78.5]$

Appendix

Table F.1: Directional concentration of the row-normalized effective residual map (mean and range over the evaluation fits).

Benchmark	Chance	True support	Transposed	Random
Darcy	$6.4$	$34.2\;[29,38]$	$12.0$	$8.6$
Terminal wave	$7.8$	$18.4\;[15,23]$	$3.6$	$6.0$

Appendix

Table F.2: Number of the

100

largest columns of

\widetilde{G}

falling inside a mask, averaged over the evaluation fits. The chance column is the count expected if the largest columns were placed uniformly at random.

Benchmark	Retained on support	Retained elsewhere	Residual variance ratio
Darcy	$3.8\%$	$0.11\%$	$69.2$
Terminal wave	$61.5\%$	$9.45\%$	$43.9$

Appendix

Table F.3: Fraction of the per-coordinate training input variance retained by the orthogonal residual, averaged over the evaluation fits. The effective field rank is

16

for Darcy and

6

for wave. The last column is the ratio of absolute residual variance on the support to absolute residual variance elsewhere. It is not the quotient of the two preceding columns, because those are fractions of an input variance that is itself larger on the support, by

1.99

for Darcy and

6.74

for wave; the quotient of the fractions is

34.8

and

6.5

respectively.

Benchmark	Plain	FAST	Reduction	Plain epoch	FAST epoch
Navier–Stokes	$0.1737$	$0.1176$	$32.3\%$	71	200
Darcy	$0.1063$	$0.0947$	$10.9\%$	189	200
Terminal wave	$0.2153$	$0.2140$	$0.6\%$	139	198

Appendix

Table H.1: Fixed-budget diagnostic using the first paired seed of each benchmark. Each entry is the mean fieldwise relative

L_{2}

error; the retained epoch minimizes tuning MSE within the common budget.

Figure I.1: Tuning MSE against the epoch for the five paired model seeds of each representation, where lower is better. The vertical range is set separately in each panel, so the levels are comparable within a benchmark and not across benchmarks.

Benchmark	Model	Branch input	Trainable	Fixed buffer
Navier–Stokes	Plain	8193	$1{,}150{,}081$	$0$
Navier–Stokes	FAST	14	$\mathbf{168{,}713}$	$98{,}316$
Darcy	Plain	4096	$625{,}921$	$0$
Darcy	FAST	32	$\mathbf{171{,}265}$	$131{,}072$
Terminal wave	Plain	4096	$625{,}665$	$0$
Terminal wave	FAST	22	$\mathbf{169{,}729}$	$49{,}152$

Appendix

Table J.1: Model size. The branch input dimension is

r_{\mathrm{eff}}+s

for FAST and

p

for Plain. Counts are taken from the retained checkpoints; the fixed buffer stores

Q_{f}

and its reconstruction and carries no gradient.

Figure K.1: Navier–Stokes results at the four corners of the sensor resolution and sample size study, where lower is better. Markers and lines show means across paired fits; whiskers show one sample standard deviation. The target grid remains

128\times 128

in every case.

	Navier–Stokes			Darcy			Terminal wave
Sensor grid	$n=20$	$60$	$240$	$n=20$	$60$	$240$	$n=20$	$60$	$240$
$8\times 8$	$+8.8$	$+14.7$	$-7.1$	$+5.5$	$-2.6$	$+12.3$	$+1.0$	$-22.0$	$-1.1$
$16\times 16$	$+17.7$	$+42.4$	$+8.6$	$+1.8$	$+0.9$	$+16.0$	$-3.2$	$-9.3$	$-34.3$
$32\times 32$	$+4.8$	$+53.1$	$+54.6$	$+1.9$	$+4.0$	$+21.4$	$-5.0$	$-4.1$	$-16.6$
$64\times 64$	$+11.0$	$+56.0$	$+73.8$	$+9.2$	$+10.5$	$+31.1$	$-1.0$	$-1.2$	$+5.6$

Appendix

Table K.1: FAST reduction in mean relative

L_{2}

error against a plain branch at the same sensor grid and training set size. Positive values favor FAST. Reading down a column gives the dependence on the branch dimension at a fixed sample size.

Figure L.1: Signed wavefield snapshots from $t=0.08$ to the terminal time $T=0.4$ . The centered pulse propagates through the heterogeneous medium and develops distorted wave fronts. All panels use one symmetric color scale, and the final panel is the learning target.

Benchmark	Model	MC error	Width	Mean cov.	P10 cov.	Success range	Flags
Navier–Stokes	Plain	$0.1675\pm 0.0029$	$0.5115\pm 0.0152$	$0.9646\pm 0.0050$	$0.8843\pm 0.0226$	$0.85$ – $0.90$	0/5
Navier–Stokes	FAST	$\mathbf{0.1258\pm 0.0036}$	$\mathbf{0.3537\pm 0.0199}$	$0.9438\pm 0.0071$	$0.8091\pm 0.0325$	$0.78$ – $0.82$	5/5
Darcy	Plain	$0.1138\pm 0.0024$	$0.01868\pm 0.00138$	$0.9333\pm 0.0087$	$0.8367\pm 0.0182$	$0.68$ – $0.77$	5/5
Darcy	FAST	$\mathbf{0.1071\pm 0.0025}$	$\mathbf{0.01632\pm 0.00087}$	$0.9240\pm 0.0080$	$0.8208\pm 0.0172$	$0.63$ – $0.73$	5/5
Terminal wave	Plain	$0.2331\pm 0.0029$	$\mathbf{0.05853\pm 0.00224}$	$0.9442\pm 0.0049$	$0.9089\pm 0.0069$	$0.89$ – $0.96$	0/5
Terminal wave	FAST	$\mathbf{0.2250\pm 0.0016}$	$0.06079\pm 0.00288$	$0.9560\pm 0.0055$	$0.9226\pm 0.0076$	$0.93$ – $0.98$	0/5

Appendix

Table M.1: Operator-level conformal evaluation of the dropout variants. MC error is measured on the aligned accuracy stream; width and coverage metrics use the independent fresh UQ stream. Entries are mean

\pm

sample standard deviation across fitted seeds. Field success is the fraction of UQ fields attaining the spatial target in ( M.2 ); brackets give the range across seeds. A flag is a conditional empirical diagnostic: the upper endpoint of that seed’s

95\%

Wilson interval, computed on one realized calibration set, lies below the nominal

0.9

field-success target. Width is in original target units.

Benchmark	Target replay	Solver check	Refinement relative $L_{2}$
Navier–Stokes	$2.43\times 10^{-8}$	div. RMS $2.90\times 10^{-14}$	$6.23\times 10^{-4}$ ( $7.24\times 10^{-4}$ )
Darcy	$2.54\times 10^{-8}$	linear residual $1.61\times 10^{-13}$	$1.60\times 10^{-2}$ ( $2.91\times 10^{-2}$ )
Terminal wave	$2.76\times 10^{-8}$	max CFL $0.323<0.707$	$4.25\times 10^{-2}$ ( $4.84\times 10^{-2}$ )

Appendix

Table N.1: Solver verification on five fixed inputs. Replay and solver-check entries are means unless marked as maxima; refinement entries are mean (maximum). Navier–Stokes jointly refines

128^{2}

\Delta t=0.001

256^{2}

\Delta t=0.0005

; Darcy refines

64^{2}

128^{2}

; wave refines

64^{2}

\Delta t=0.004

128^{2}

\Delta t=0.002

PDE Operator Learning

Partial differential equation (PDE) operator learning builds models that map inputs such as coefficients and initial conditions to solution fields.

See all 228 papers

Feb 25, 2026

The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

Operator learning has the potential to strongly impact scientific computing by learning solution operators for differential equations, potentially accelerating multi-query tasks such as design optimization and uncertainty quantification by orders of magnitude. Despite proven universal approximation properties, deep operator networks (DeepONets) often exhibit limited accuracy and generalization in practice, which hinders their adoption. Understanding these limitations is therefore crucial for further advancing the approach. This work analyzes performance limitations of the classical DeepONet architecture. It is shown that the approximation error is dominated by the branch network when the internal dimension is sufficiently large, and that the learned trunk basis can often be replaced by classical basis functions without a significant impact on performance. To investigate this further, a modified DeepONet is constructed in which the trunk network is replaced by the left singular vectors of the training solution matrix. This modification yields several key insights. First, for examples involving the KdV and Burgers equations, a spectral bias in the branch network is observed, with coefficients of dominant, low-frequency modes learned more effectively. Second, through the interplay of the singular-value weighting and the optimizer's neglect of small modes, the branch error is dominated by modes with large and intermediate singular values. Third, using a shared branch network for all mode coefficients, as in the standard architecture, improves generalization of small modes compared to a stacked architecture in which coefficients are computed separately. Finally, detrimental coupling between modes in parameter space is identified, which weakens with increasing network width.

Alexander Heinlein, Johannes Taraz

Delft Institute of Applied Mathematics, Delft University of Technology, Delft, 2628CD, The Netherlands

Deep Operator Networks Neural Operators Neural Network Generalization Neural Network Training Dynamics

Ask PDF

Dec 9, 2025

Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximation

Deep Operator Networks (DeepONets) have emerged as a powerful framework for data-driven operator learning, providing flexible surrogates for nonlinear mappings arising in partial differential equations (PDEs). However, the standard trunk network, which operates directly on raw spatial or spatiotemporal coordinates through fully connected layers, often struggles to represent sharp gradients, boundary layers, and other non-periodic solution structures on bounded domains. To address these limitations, we introduce the Spectral-Embedded Deep Operator Network (SEDONet), a novel DeepONet architecture in which the trunk is driven by a fixed Chebyshev spectral dictionary instead of coordinate inputs. This non-periodic spectral embedding provides a principled inductive bias for bounded domains, enabling the learned operator to capture fine-scale features that are difficult for Fourier-based or MLP-only trunks to represent. SEDONet is evaluated on the 2-D Poisson equation, 1-D Burgers' equation, 1-D advection-diffusion equation, Allen-Cahn equation, Lorenz-96 chaotic system, and Darcy flow, covering elliptic, hyperbolic, parabolic, chaotic, and multiscale problems. Across all benchmarks, SEDONet consistently achieves the lowest or statistically comparable relative

L^2

errors among DeepONet, FEDONet, and SEDONet, with improvements of up to 54% over the baseline DeepONet and consistent gains over Fourier-embedded variants on bounded, non-periodic problems. Energy spectrum analyses further demonstrate that SEDONet more accurately preserves intermediate- and high-frequency solution structures. The proposed framework provides a simple, parameter-neutral modification to DeepONets, offering a robust and computationally efficient spectral approach for surrogate modeling of nonlinear operators in scientific computing.

Muhammad Abid, Omer San

Department of Mechanical and Aerospace Engineering, University of Tennessee, Knoxville, TN 37996, USA.

PDE Surrogate Modeling PDE Operator Learning Spectral Methods Deep Operator Networks

Ask PDF

May 9, 2026

Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates

Neural operators for time-dependent PDEs face a structural tension: spectral architectures (FNO and descendants) inherit exponential rollout-error growth from their one-step Lipschitz constant, while hierarchical U-Net operators trade resolution invariance for multi-scale detail. We introduce SpectraNet, an autoregressive neural operator that composes truncated spectral convolutions inside a U-Net hierarchy with a Residual-Target Spectral Block trained under a Semigroup-Consistency Loss. The residual-target parametrization replaces L^T stability blow-up with linear T*delta drift, and the spectral path's parameter count is Theta(L w^2 M^2), independent of grid N. Under a single unified protocol against 16 published neural-operator baselines on Navier-Stokes nu=1e-5 at 64x64, SpectraNet reaches test relative L2 = 0.0822 at 2.04M parameters -- 2.33x fewer than canonical FNO at ~20% lower error -- and wins five of six rows in a cross-PDE comparison against FNO (NS at nu in {1e-4, 1e-3}, PDEBench Shallow-Water 2D and Diffusion-Reaction, with the Active-Matter row going to FNO inside its seed spread). Trained from scratch at native 128^2 under the same protocol, SpectraNet improves to 0.0724 while FNO regresses to 0.3080. Free rollout stays bounded for T=100 where FNO diverges across all 200 test trajectories. On consumer CPU at B=1, SpectraNet runs sub-200ms while the full-attention Transformer that wins raw L2 pays ~60x latency; we do not claim to beat that Transformer on raw L2, only to dominate the lightweight (<=5M parameter, sub-200ms CPU) Pareto frontier. Source code: https://github.com/Enrikkk/spectranet

Enrique Hernández Noguera, Md Meftahul Ferdaus, Elias Ioup +2

University of New Orleans, New Orleans, LA, USA · Naval Research Laboratory, Stennis Space Center, MS, USA

U-Net PDE Surrogate Modeling Convolutional Neural Networks Spectral Methods Neural Operators

Ask PDF

Abstract

PDE Operator Learning

Explore similar work

The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximation

Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates