Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

Mike Nguyen; Nicole Mücke

stat.MLMar 1, 2026

Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

Authors: Mike Nguyen, Nicole Mücke

Organizations: Technical University of Braunschweig

Abstract

In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework enables a rigorous theoretical analysis of neural operators and neural networks through the lens of the Neural Tangent Kernel (NTK). In particular, it allows us to establish optimal learning rates and provides a good understanding of how many neurons are required to achieve a given accuracy. Furthermore, we establish minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space. These results sharpen and complete earlier findings for specific kernel algorithms.

Figures & tables

References	$M$	Method	Smoothness
[1]	$O(n)$	KRR (rvk)	$r\in[0.5,1]$
[2]	$O(\sqrt{n}\log n)$	KRR (rvk)	$r\in[0.5,1]$
[3]	$O(\sqrt{n})$	KRR (vvk)	$r=0.5$
Our	$O(\sqrt{n}\log n)$	Spec. (vvk)	$2r+b>1$

Table 1: Comparison of random feature requirements (

M

) for achieving generalization error of order

O(n^{-\frac{1}{2}})

. The last column indicates the smoothness classes where optimal rates are known; see Assumptions 3.2 , 3.3 for the meaning of

r,b>0

. [1]= Rahimi and Recht (2007) , [2]= Rudi and Rosasco (2016) , [3]= Lanthaler and Nelsen (2023)

Explore similar work

Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels
Sep 12, 2026Lei Shi, Jia-Qi Yang, Ding-Xuan ZhouOperator Learning Kernel Methods
School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China. · School of Mathematics and Statistics, the University of Sydney, Sydney, New South Wales 2006, Australia.
A short tour of operator learning theory: Convergence rates, statistical limits, and open questions
Feb 28, 2026Simone Brugiapaglia, Nicola Rares Franco, Nicholas H. NelsenOperator Learning Statistical Learning Theory
Concordia University · Politecnico di Milano · Cornell University and The University of Texas at Austin
Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces
Jun 16, 2026Yahong Yang, Zecheng Zhang, Wei Zhu +2Neural Network Approximation Theory Operator Learning
School of Mathematics, Georgia Institute of Technology, 686 Cherry Street, Atlanta, GA 30332-0160, USA. · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA. · Department of Mathematics, Hong Kong Baptist University, FSC1202, Fong Shu Chuen Building, Hong Kong Baptist University, Kowloon Tong, Hong Kong.

Sep 12, 2026J/K move · Enter open · S save

Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels

Lei Shi, Jia-Qi Yang, Ding-Xuan Zhou

School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China. · School of Mathematics and Statistics, the University of Sydney, Sydney, New South Wales 2006, Australia.

Operator Learning Kernel Methods Autoencoders Reproducing Kernel Hilbert Spaces Neural Tangent Kernel

Open paper AskPDF arXiv

Operator learning is formulated on function spaces, but training data are typically available only through finite-dimensional representations. In encoder--decoder architectures, a matrix-valued kernel on the encoded space induces an operator-valued kernel on the original function spaces, and the corresponding reproducing kernel Hilbert spaces are isometrically isomorphic. As the input and output resolutions increase, the induced kernels converge to a limiting kernel, in the sense of operator-norm convergence of their associated integral operators, allowing regularity assumptions to be stated independently of the encoding resolution. For regularized stochastic gradient descent, we establish upper bounds for decreasing and fixed step sizes, separating the encoding and regularization terms from optimization terms of order

t^{-θ}

and

T^{-θ'}

, respectively, for any

θ,θ'\in(0,1)

. We further prove lower bounds showing that these encoding-induced terms are generally unavoidable. The analysis is further extended to encoder--decoder neural networks through the limiting neural tangent kernel (NTK), yielding error bounds with an additional finite-width term and polynomial parameter and sample complexity guarantees when the encoding errors decay algebraically. The framework covers matrix-valued kernels constructed from radial and dot product kernels, NTKs arising from wide encoder--decoder neural networks, and encoder--decoder pairs based on Fourier, Legendre polynomial, wavelet, PCA, or pointwise sampling representations.

Abstract

Explore similar work

Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels

A short tour of operator learning theory: Convergence rates, statistical limits, and open questions

Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces