Operator-Theoretic Generalization Bounds for Multitask Deep Learning

Mahdi Mohammadigohari; Thomas Borsani; Giuseppe Di Fatta

cs.LGAug 17, 2026

Operator-Theoretic Generalization Bounds for Multitask Deep Learning

Authors: Mahdi Mohammadigohari, Thomas Borsani, Giuseppe Di Fatta

Organizations: Free University of Bozen–Bolzano, Italy

Abstract

We develop operator-theoretic generalization bounds for deep multi-output function classes by representing network layers as Koopman composition operators on vector-valued reproducing kernel Hilbert spaces. In vector-valued Sobolev RKHSs, we derive Rademacher complexity bounds for invertible and width-expanding injective architectures. The estimates separate the output-coupling contribution, represented by the trace of the task matrix, from the layerwise operator norms, Sobolev symbol ratios, determinant factors, and restriction constants generated by the linear maps. We then analyze a distinct one-dimensional Brownian/Cameron--Martin regime. Using the exact anchored derivative-norm characterization of the vector-valued Brownian RKHS, we obtain layerwise bounds for domain-preserving scalar linear maps and anchored diffeomorphic activations; the corresponding factors scale as $|W_l|^{1/2}$ and $\|σ_l'\|_\infty^{1/2}$ , respectively, and do not involve Sobolev smoothness exponents. Because the Sobolev and Brownian results concern different hypothesis spaces, neither is asserted to dominate the other uniformly. We additionally formulate shared operator learning across tasks, prove a finite-rank representer theorem, derive the exact finite-dimensional problem for squared loss, and establish a target-transfer bound when the learned operator is obtained independently of the target sample. Synthetic and MNIST studies examine stabilized Sobolev-inspired and Brownian-inspired complexity proxies; these empirical proxies are not evaluations of the proved bounds for rank-deficient architectures.

Figures & tables

Panel of Figure 1 — Figure 1: Empirical behavior of stabilized complexity proxies. (a) Sobolev-inspired and Brownian-inspired proxy values during training on the synthetic regression problem. (b) MNIST test performance for the unregularized baseline and for models trained with the two proxy penalties. The labels inside the original plots use the shorthand “bound,” but the quantities are empirical surrogates rather than evaluations of the proved theorems.

Explore similar work

Jun 16, 2026cs.LG

Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces

We develop approximation and generalization error estimates for multi-input neural operators, with the output error measured in Sobolev norms. In contrast to standard operator-learning settings with a single input function, our framework allows multiple input functions defined on possibly different domains, with different dimensions and Sobolev regularities. The derived rates explicitly quantify the contribution of each input space to the final error bound. In particular, in the balanced regime, the approximation and generalization rates are governed by the interaction between the input dimensions, regularities, and Sobolev orders, while the dependence on the model complexity retains a

\log\log/\log

-type structure. Our analysis provides a general theoretical framework for multi-input operator learning, including Sobolev training, and is applicable to operator learning problems arising from partial differential equations and scientific computing.

Yahong Yang, Zecheng Zhang, Wei Zhu +2

School of Mathematics, Georgia Institute of Technology, 686 Cherry Street, Atlanta, GA 30332-0160, USA. · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA. · Department of Mathematics, Hong Kong Baptist University, FSC1202, Fong Shu Chuen Building, Hong Kong Baptist University, Kowloon Tong, Hong Kong.

Neural Network Approximation Theory Operator Learning Neural Operators Generalization Bounds

Ask PDF

May 21, 2026cs.LG

Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning

We study the approximation and statistical complexity of learning collections of operators in a shared multi-task setting, with a focus on the Multiple Neural Operators (MNO) architecture. For broad classes of Lipschitz multiple operator maps, we derive near-optimal upper bounds for approximation and statistical generalization. On the lower-bound side, we establish a curse of parametric complexity and prove corresponding minimax rates. Together, these results show that shared representations across tasks do not increase the overall cost: multi-task operator learning follows the same scaling laws as single operator learning. We also compare MNO with a multi-task extension of DeepONet based on concatenated task inputs and show that, from a worst-case approximation-complexity perspective, both architectures satisfy essentially the same asymptotic rates.

Adrien Weihs, Hayden Schaeffer

Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA.

Neural Network Approximation Theory Operator Learning Statistical Learning Theory

Ask PDF

Aug 12, 2026cs.LG

Kernel Methods for Learning Operators with Multiple Inputs and Outputs

Learning mappings between infinite-dimensional objects is a central challenge in scientific machine learning. We introduce a general kernel-based encoder-decoder framework for operator learning that separates observation, representation, learning, and reconstruction. We develop this framework for multi-input, multi-output operator learning, where operators map between products of potentially distinct function spaces. Our approximation theory shows that, although the number of inputs and outputs can increase, the convergence rate is governed by the most challenging constituent approximation problem rather than the overall problem dimension. The framework leads to practical kernel methods with closed-form training and inference, combining mathematical tractability with computational efficiency. We further specialize the approach to multiple operator learning by introducing KernelMO, a family of kernel methods with complementary operator-valued and product-space formulations. Across five families of parametric partial differential equations, the proposed methods achieve competitive or state-of-the-art predictive accuracy while reducing training and inference costs relative to neural operator architectures and deep learning based models, offering an efficient and lightweight alternative.

Adrien Weihs, Chunyang Liao, Jingmin Sun +1

Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA. · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701, USA. · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA.

Kernel Methods PDE Operator Learning Operator Learning

Ask PDF