Kernel Ridge Regression

Follow this topic Ask about this topic

Kernel ridge regression (KRR) fits nonlinear functions in a reproducing-kernel Hilbert space by balancing data fit against a norm penalty, making predictions from similarities between examples. Work studies how kernel spectra, target smoothness, input dimension, and alignment shape learning curves, interpolation, benign overfitting, and generalization. To scale Gram-matrix methods, approaches use Nyström sampling with ridge leverage scores, random features, sketching, and preconditioned iterative solvers; other variants learn or reweight kernels or add unpenalized features. KRR is also used for dynamical systems, treatment-effect estimation, and covariate shift, while a central tension is preserving statistical guarantees and useful inductive bias as kernels and datasets become more complex.

Latest papers

29 papers4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

On the Optimality of Random Features under General Conditions: Misspecified Cases and Robust Learning
Oct 13, 2026Caixing WangRandom Feature Methods Kernel Ridge Regression
School of Statistics and Data Science, Southeast University, Nanjing, China
BLOKS: Scaling Kernel Solvers with Sketched Block Preconditioning
Oct 9, 2026Giordano Vitale, Giacomo Meanti, Marco Rando +2Nyström Method Kernel Ridge Regression
MaLGa Center, DIBRIS, Universit`a degli Studi di Genova, Genoa, Italy · Universit´e Cˆote d’Azur, INRIA, CNRS, LJAD, Nice, France · Dipartimento di Matematica e Applicazioni, Universit`a degli Studi Milano–Bicocca, Italy +2
Learning Closure of Dynamical Systems with Kernel Ridge Regression
Oct 1, 2026Evan Habbershaw, John Harlim, Senwei LiangPDE Surrogate Modeling Nonlinear System Identification
Department of Mathematics, The Pennsylvania State University · Department of Mathematics and Institute for Computational and Data Sciences, The Pennsylvania State University · Department of Mathematics and Statistics, Texas Tech University
Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders
Sep 29, 2026Giovanni Marraffini, Victoria Shevchenko, Carlo Alberto Barbano +1Kernel Ridge Regression Medical Imaging Foundation Models
UNITO · MIND
Structured Extrema Errors in Classical Surrogates for Viscous Burgers: A Physics-Consistent Interpretation
Sep 7, 2026Youssef OubariKernel Ridge Regression Surrogate Modeling
IMT Atlantique, France
Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy
Aug 28, 2026Lorenzo Rizzi, Arie Wortsman Zurich, Bruno LoureiroKernel Ridge Regression Kernel Methods
Departement d’Informatique, École Normale Supérieure, PSL & CNRS · Dipartimento di Fisica e Astronomia, Università degli Studi di Padova · Scuola Galileiana di Studi Superiori, Università degli Studi di Padova
Adaptive Regularization for Random Features: A Neighboring Early-Stopping Rule with Oracle-Rate Guarantees
Aug 26, 2026Caixing Wang, Zhibo Chen, Yue WangRandom Feature Methods Kernel Ridge Regression
School of Statistics and Data Science, Southeast University · Faculty of Business for Science and Technology, School of Management, University of Science and Technology of China
Functional linear regression from sparse to dense designs: a pooling-ridge method and minimax optimality
Aug 26, 2026Shunxing Yan, Fang YaoLinear Regression Minimax Estimation
Department of Statistics and Data Science, Tsinghua University · School of Mathematical Sciences, Center for Statistical Science, Peking University
Nonparametric Goodness-of-fit Testing under Covariate Shift
Aug 5, 2026Zhen Hou, Dong XiaKernel Ridge Regression Covariate Shift
Department of Mathematics Hong Kong University of Science and Technology
Learning dynamical systems from noisy data with Weak-form Kernel Ridge Regression
Jun 30, 2026Max Kreider, John Harlim, Daning HuangKernel Ridge Regression Kernel Regression
Department of Mathematics The Pennsylvania State University, University Park, PA 16802, USA · Department of Mathematics, Institute for Computational and Data Sciences The Pennsylvania State University, University Park, PA 16802, USA · Department of Aerospace Engineering The Pennsylvania State University, University Park, PA 16802, USA
Bernstein-Schur Kernels: Random Features by Sketched Modulation and Radial Randomization
Jun 8, 2026Taha BouhsineFourier Feature Embeddings Kernel Ridge Regression
Azetta AI
Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression
May 30, 2026Kwangho Kim, Jisu KimKernel Ridge Regression Distributional Regression
Department of Statistics, Korea University, Seoul, Korea · Department of Statistics, Seoul National University, Seoul, Korea
Perturbative methods for non-parametric instrumental variable
May 29, 2026Wei Bu, Arthur GrettonKernel Ridge Regression Instrumental Variable Estimation
Harvard University, Northeastern University, Physics Department · University College London.
Conditional KRR: Injecting Unpenalized Features into Kernel Methods with Applications to Kernel Thresholding
May 25, 2026Rustem Takhanov, Zhenisbek AssylbekovKernel Ridge Regression Kernel Regression
Department of Mathematics, Nazarbayev University, Astana, Kazakhstan · Nazarbayev University Research Administration, Astana, Kazakhstan · Purdue University Fort Wayne, Indiana, USA.
Canonical Regularisation of Wide Feature-Learning Neural Networks
May 18, 2026George Whittle, Pranav Vaidhyanathan, Juliusz Ziomek +2Kernel Ridge Regression Representation Learning
Department of Engineering Science University of Oxford
Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models
May 14, 2026Libin Zhu, Damek Davis, Dmitriy Drusvyatskiy +1Kernel Ridge Regression Representation Learning
Department of Mathematics, University of Washington, Seattle, WA 98195 · Wharton Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA 19104, USA · Department of Mathematics, U. Washington, Seattle, WA 98195 +1
Large Dimensional Kernel Ridge Regression: Extending to Product Kernels
May 14, 2026Yang Zhou, Yicheng Li, Yuqian Cheng +1Generalization Bounds Kernel Ridge Regression
Department of Statistics and Data Science Tsinghua University Beijing, 100084, China · Department of Mathematical Science Tsinghua University Beijing, 100084, China
On Kernel Eigen-alignments of KRR: Reconstruction and Generalization
May 14, 2026Yang Liu, Ernest Fokoue, Richard Lange +1Kernel Ridge Regression Generalization Bounds
Galisano College of Coumputing and Information Science · Rochester Institute of Technology · Rochester, NY +1
Bayesian Optimization with Structured Measurements: A Vector-Valued RKHS Framework
May 10, 2026Wenbin Wang, Colin N. JonesKernel Ridge Regression Bayesian Optimization
Automatic Control Laboratory, EPFL
Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression
May 8, 2026Mingsong Yan, Dongyang Li, Charles Kulick +1Softmax Attention Transformer Interpretability
Department of Mathematics, University of California, Santa Barbara, CA
Cubit: Token Mixer with Kernel Ridge Regression
May 7, 2026Chuanyang Zheng, Jiankai Sun, Yihang Gao +6Kernel Ridge Regression Transformer Attention
Differentiable Kernel Ridge Regression for Deep Learning Pipelines
May 4, 2026Jean-Marc Mercier, Gabriele SantinKernel Ridge Regression Kernel Regression
1MPG-Partners, Paris, France · Department of Environmental Sciences, Informatics and Statistics, Ca’ Foscari University of Venice, Italy
Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions
Apr 25, 2026Weihao Lu, Qian Lin, Yingcun Xia +1Kernel Ridge Regression Benign Overfitting
Department of Statistics and Data Science, National University of Singapore. · Department of Statistics and Data Science, Tsinghua University.
Pack only the essentials: Adaptive dictionary learning for kernel ridge regression
Apr 24, 2026Daniele Calandriello, Alessandro Lazaric, Michal ValkoKernel Ridge Regression Kernel Regression
SequeL team, INRIA Lille - Nord Europe, France
Analysis of Nystrom method with sequential ridge leverage scores
Apr 22, 2026Daniele Calandriello, Alessandro Lazaric, Michal ValkoKernel Ridge Regression Kernel Regression
SequeL team INRIA Lille - Nord Europe
Estimating Continuous Treatment Effects with Two-Stage Kernel Ridge Regression
Apr 15, 2026Seok-Jin Kim, Kaizheng WangCausal Effect Estimation Kernel Ridge Regression
Department of IEOR, Columbia University · Department of IEOR and Data Science Institute, Columbia University
Loop Corrections in Random Feature Models: Training Error and Generalization Gap
Apr 14, 2026Taeyoung KimRandom Feature Methods Kernel Ridge Regression
Center for AI and Natural Sciences, Korea Institute for Advanced Study, Seoul 02455, South Korea
Learning Multi-Index Models with Hyper-Kernel Ridge Regression
Oct 2, 2025Shuo Huang, Hippolyte Labarrière, Ernesto De Vito +2Kernel Ridge Regression Kernel Regression
Istituto Italiano di Tecnologia, Genoa, Italy · MaLGa - DIBRIS - Universit`a di Genova, Genoa, Italy · MaLGa - DIMA - Universit`a di Genova, Genoa, Italy +2
A Compositional Kernel Model for Feature Learning
Sep 17, 2025Feng Ruan, Keli Liu, Michael JordanKernel Ridge Regression Kernel Regression
*Department of Statistics and Data Science, Northwestern University. · *Company E · *Department of Statistics and Computer Science, University of California, Berkeley.

Oct 13, 2026J/K move · Enter open · S save

On the Optimality of Random Features under General Conditions: Misspecified Cases and Robust Learning

Caixing Wang

School of Statistics and Data Science, Southeast University, Nanjing, China

Random Feature Methods Kernel Ridge Regression Statistical Learning Theory Model Misspecification

Open paper AskPDF arXiv

Random features provide a standard approach to scaling kernel methods to large datasets by approximating the kernel with a finite-dimensional feature map. Existing works of random feature methods primarily focus on well-specified models and light-tailed noise conditions, leaving a gap in understanding their performance under more general conditions, such as misspecified models and heavy-tailed noise. In this paper, we provide a comprehensive and improved analysis of kernel ridge regression with random features (KRR-RF) under general conditions. In the light-tailed noise setting, we improve upon the worst-case uniform bounds of the feature map used in the previous works and lead to the minimax optimal learning rates with

2r+γ>α_0

, where

(r,γ,α_0)

denote the regularity of the target function, the effective dimension of the kernel, and the embedding index of the RKHS, respectively. In the heavy-tailed noise setting, we leverage a recently developed Hilbert-valued Fuk-Nagaev inequality and extend the same optimal results under finite weak moment conditions. The additional cost to achieve robustness is a mildly stronger explicit sample-size requirement than in the light-tailed case. For both settings, our results recover the conditions random features budgets (up to log factors) required in the previous works. In the most benign case when

α_0=γ

, KRR-RF are minimax optimal for all

r \in (0,1]

. To the best of our knowledge, this is the first work to establish optimal learning rates for KRR-RF under weak moment conditions in the misspecified setting. Numerical experiments and a real data analysis validate the derived learning rates and demonstrate their practical relevance in the presence of mis-specification and heavy-tailed noise.