Kernel Regression

Kernel regression predicts outputs by comparing inputs with a kernel, often fitting a smooth function in a reproducing kernel Hilbert space through kernel ridge regression or gradient-based methods. Researchers study how kernel spectra, regularization, noise, and data structure affect prediction and inference, while methods such as Nyström approximation and random features reduce computational costs. Learned and compositional kernels extend these methods to structured, high-dimensional data and settings including dynamical systems, physics-informed learning, and distribution shift. A central tension is that fixed kernels offer tractable guarantees but can be computationally expensive or poorly matched to complex targets, motivating adaptive kernels and comparisons with feature-learning neural networks.