The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

Deep operator networks can become statistically unstable when partial differential equation inputs are observed at thousands of strongly correlated sensors but only a small number of operator samples is available. We introduce FAST-DeepONet, a branch representation combining a fixed spectral path with a regularized projection of the orthogonal residual, in which the directional penalty acts on the effective residual map after each of its rows is normalized. On Navier--Stokes flow a plain DeepONet degrades from $0.0394$ to $0.1556$ mean relative $L_2$ error as the branch grows from $129$ to $8193$ coordinates, while FAST-DeepONet stays near $0.04$, so the sensor grid can be refined without a statistical penalty. Across independent test sets for Navier--Stokes flow, Darcy flow, and signed terminal wavefield prediction it lowers mean relative $L_2$ error by $4.7\%$ to $37.0\%$ with three to seven times fewer trainable parameters. A spectral-only branch sharing the same basis separates the two paths: the fixed spectral path carries the improvement on Navier--Stokes and Darcy, while terminal wave prediction requires the residual path together with its directional penalty. FAST-DeepONet targets coordinate-query architectures and trains on solution values alone.

Deep Operator Networks (DeepONets) have emerged as a powerful framework for data-driven operator learning, providing flexible surrogates for nonlinear mappings arising in partial differential equations (PDEs). However, the standard trunk network, which operates directly on raw spatial or spatiotemporal coordinates through fully connected layers, often struggles to represent sharp gradients, boundary layers, and other non-periodic solution structures on bounded domains. To address these limitations, we introduce the Spectral-Embedded Deep Operator Network (SEDONet), a novel DeepONet architecture in which the trunk is driven by a fixed Chebyshev spectral dictionary instead of coordinate inputs. This non-periodic spectral embedding provides a principled inductive bias for bounded domains, enabling the learned operator to capture fine-scale features that are difficult for Fourier-based or MLP-only trunks to represent. SEDONet is evaluated on the 2-D Poisson equation, 1-D Burgers' equation, 1-D advection-diffusion equation, Allen-Cahn equation, Lorenz-96 chaotic system, and Darcy flow, covering elliptic, hyperbolic, parabolic, chaotic, and multiscale problems. Across all benchmarks, SEDONet consistently achieves the lowest or statistically comparable relative $L^2$ errors among DeepONet, FEDONet, and SEDONet, with improvements of up to 54% over the baseline DeepONet and consistent gains over Fourier-embedded variants on bounded, non-periodic problems. Energy spectrum analyses further demonstrate that SEDONet more accurately preserves intermediate- and high-frequency solution structures. The proposed framework provides a simple, parameter-neutral modification to DeepONets, offering a robust and computationally efficient spectral approach for surrogate modeling of nonlinear operators in scientific computing.

An important application of neural networks in scientific computing is non-linear operator learning. In this framework, a neural network is trained to fit a non-linear map between infinite dimensional spaces, for example, the solution operator of ordinary and partial differential equations. Recently, inspired by the discovery of in-context learning for large language models, an even more ambitious paradigm has been explored, called multi-operator learning. In this approach, a neural network is trained to learn many different operators at the same time. At inference time, the network is passed a small number of example input and output pairs to disambiguate the desired operator. In this work, we provide a precise mathematical formulation of the multi-operator learning problem, propose a parameter-efficient, attention-free architecture, called DeepOSets, and prove its universality in the in-context multi-operator learning task. We perform experiments demonstrating the efficacy of DeepOSets on several benchmarks, including Burgers' equation, 1D and 2D Darcy flow, and a variety of initial-value and boundary-value differential equations. In a comparison with the transformer-based ICON model on the 1D Darcy problem, under a small training budget (10k steps), DeepOSets attained lower error at every context size with 19% of the parameters, while with an extended budget (100k steps), ICON attains 26-41% lower error, but at the cost of roughly 218$\times$ more training time, 12$\times$ higher inference latency, and 5.3$\times$ more parameters.