Training-point consistency of kernel ridge regression with varying correlation scales
Organizations: School of Science, Beijing University of Civil Engineering and Architecture, Beijing, China · SKLMS, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China · School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, China
Abstract
The scale parameter in kernel ridge regression plays a pivotal role in modeling the correlation structure of data. In modern applications, sampling density and the underlying correlation range often change simultaneously, making a fixed-scale assumption highly restrictive. However, establishing asymptotic theories under varying scale parameters is fundamentally challenging. To address this gap, this paper develops a unified theoretical framework for kernel ridge regression under varying scale parameters. We establish that, on uniform tensor grids with product exponential kernels, the consistency of the training-point risk holds if and only if the effective local sample size diverges to infinity. This sharp criterion universally covers all regimes along arbitrary deterministic scale sequences, seamlessly unifying the long-range, fixed-scale, and short-range. Furthermore, we provide an explicit, signal-independent construction of a deterministic regularization penalty that guarantees this consistency. Numerical experiments illustrate the criterion and the corresponding penalty across all the three regimes.
Figures & tables
Kernel Ridge Regression
Kernel ridge regression estimates nonlinear relationships by fitting data with kernel similarities while regularizing model complexity.
See all 33 papers