cs.LGApr 2, 2026

Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networks

Authors: Adrien Weihs, Hayden Schaeffer

Organizations: Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Multiple operator learning concerns learning operator families {G[α]:U→V}α∈W\{G[α]:U\to V\}_{α\in W} indexed by an operator descriptor αα. Training data are collected hierarchically by sampling operator instances αα, then input functions uu per instance, and finally evaluation points xx per input, yielding noisy observations of G[α][u](x)G[α][u](x). While recent work has developed expressive multi-task and multiple operator learning architectures and approximation-theoretic scaling laws, quantitative statistical generalization guarantees remain limited. We provide a covering-number-based generalization analysis for separable models, focusing on the Multiple Neural Operator (MNO) architecture: we first derive explicit metric-entropy bounds for hypothesis classes given by linear combinations of products of deep ReLU subnetworks, and then combine these complexity bounds with approximation guarantees for MNO to obtain an explicit approximation-estimation tradeoff for the expected test error on new (unseen) triples (α,u,x)(α,u,x). The resulting bound makes the dependence on the hierarchical sampling budgets (nα,nu,nx)(n_α,n_u,n_x) transparent and yields an explicit learning-rate statement in the operator-sampling budget nαn_α, providing a sample-complexity characterization for generalization across operator instances. The structure and architecture can also be viewed as a general purpose solver or an example of a "small'' PDE foundation model, where the triples are one form of multi-modality.

Figures & tables

 # networkswidthdepthsparsityparameter magnitude
lpl_{p}P≲ε−ε−dWP\lesssim\varepsilon^{-\varepsilon^{-d_{W}}}O(1)\mathcal{O}(1)≲ε−dW\lesssim\varepsilon^{-d_{W}}≲ε−dW\lesssim\varepsilon^{-d_{W}}≲ε−ε−dW\lesssim\varepsilon^{-\varepsilon^{-d_{W}}}
bkb_{k}H≲ε−ε−ε−dWH\lesssim\varepsilon^{-\varepsilon^{-\varepsilon^{-d_{W}}}}O(1)\mathcal{O}(1)≲ε−ε−dW\lesssim\varepsilon^{-\varepsilon^{-d_{W}}}≲ε−ε−dW\lesssim\varepsilon^{-\varepsilon^{-d_{W}}}≲ε−ε−ε−dW\lesssim\varepsilon^{-\varepsilon^{-\varepsilon^{-d_{W}}}}
τℓ\tau_{\ell}N≲ε−ε−dWN\lesssim\varepsilon^{-\varepsilon^{-d_{W}}}O(1)\mathcal{O}(1)≲ε−dW\lesssim\varepsilon^{-d_{W}}≲ε−dW\lesssim\varepsilon^{-d_{W}}≲ε−ε−dW\lesssim\varepsilon^{-\varepsilon^{-d_{W}}}
Table 1
Figure 1: Schematic structure of the training dataset SG,{ys},{cs}S_{G,\{y_{s}\},\{c_{s}\}} defined in Definition 3.2 for multiple operator learning. supG( σ2\sigma^{2} ) denotes a sub-Gaussian distribution with variance proxy σ2\sigma^{2} . Blue arrows indicate discretization steps; black arrows represent the flow of data.
More in this topic

Operator Learning

Operator learning studies models that map functions or fields to functions, enabling fast prediction across scientific and dynamical systems.

See all 99 papers

Explore similar work

CardsList
May 21, 2026

Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning

We study the approximation and statistical complexity of learning collections of operators in a shared multi-task setting, with a focus on the Multiple Neural Operators (MNO) architecture. For broad classes of Lipschitz multiple operator maps, we derive near-optimal upper bounds for approximation and statistical generalization. On the lower-bound side, we establish a curse of parametric complexity and prove corresponding minimax rates. Together, these results show that shared representations across tasks do not increase the overall cost: multi-task operator learning follows the same scaling laws as single operator learning. We also compare MNO with a multi-task extension of DeepONet based on concatenated task inputs and show that, from a worst-case approximation-complexity perspective, both architectures satisfy essentially the same asymptotic rates.

Adrien Weihs, Hayden Schaeffer

Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA.

Neural Network Approximation TheoryOperator LearningStatistical Learning Theory
AskPDF
Aug 17, 2026

Operator-Theoretic Generalization Bounds for Multitask Deep Learning

We develop operator-theoretic generalization bounds for deep multi-output function classes by representing network layers as Koopman composition operators on vector-valued reproducing kernel Hilbert spaces. In vector-valued Sobolev RKHSs, we derive Rademacher complexity bounds for invertible and width-expanding injective architectures. The estimates separate the output-coupling contribution, represented by the trace of the task matrix, from the layerwise operator norms, Sobolev symbol ratios, determinant factors, and restriction constants generated by the linear maps. We then analyze a distinct one-dimensional Brownian/Cameron--Martin regime. Using the exact anchored derivative-norm characterization of the vector-valued Brownian RKHS, we obtain layerwise bounds for domain-preserving scalar linear maps and anchored diffeomorphic activations; the corresponding factors scale as ∣Wl∣1/2|W_l|^{1/2} and ∥σl′∥∞1/2\|σ_l'\|_\infty^{1/2}, respectively, and do not involve Sobolev smoothness exponents. Because the Sobolev and Brownian results concern different hypothesis spaces, neither is asserted to dominate the other uniformly. We additionally formulate shared operator learning across tasks, prove a finite-rank representer theorem, derive the exact finite-dimensional problem for squared loss, and establish a target-transfer bound when the learned operator is obtained independently of the target sample. Synthetic and MNIST studies examine stabilized Sobolev-inspired and Brownian-inspired complexity proxies; these empirical proxies are not evaluations of the proved bounds for rank-deficient architectures.

Mahdi Mohammadigohari, Thomas Borsani, Giuseppe Di Fatta

Free University of Bozen–Bolzano, Italy

Generalization BoundsMulti-Task LearningReproducing Kernel Hilbert SpacesOperator Learning
AskPDF
Jun 16, 2026

Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces

We develop approximation and generalization error estimates for multi-input neural operators, with the output error measured in Sobolev norms. In contrast to standard operator-learning settings with a single input function, our framework allows multiple input functions defined on possibly different domains, with different dimensions and Sobolev regularities. The derived rates explicitly quantify the contribution of each input space to the final error bound. In particular, in the balanced regime, the approximation and generalization rates are governed by the interaction between the input dimensions, regularities, and Sobolev orders, while the dependence on the model complexity retains a log⁡log⁡/log⁡\log\log/\log-type structure. Our analysis provides a general theoretical framework for multi-input operator learning, including Sobolev training, and is applicable to operator learning problems arising from partial differential equations and scientific computing.

Yahong Yang, Zecheng Zhang, Wei Zhu +2

School of Mathematics, Georgia Institute of Technology, 686 Cherry Street, Atlanta, GA 30332-0160, USA. · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA. · Department of Mathematics, Hong Kong Baptist University, FSC1202, Fong Shu Chuen Building, Hong Kong Baptist University, Kowloon Tong, Hong Kong.

Neural Network Approximation TheoryOperator LearningNeural OperatorsGeneralization Bounds
AskPDF