Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalences

Motonobu Kanagawa; Philipp Hennig; Dino Sejdinovic; Bharath K. Sriperumbudur

stat.MLJun 20, 2025

Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalences

Authors: Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, Bharath K. Sriperumbudur

Abstract

This monograph studies the relations between two approaches using positive definite kernels: probabilistic methods using Gaussian processes, and non-probabilistic methods using reproducing kernel Hilbert spaces (RKHS). They are widely studied and used in machine learning, statistics, and numerical analysis. We study connections and equivalences for fundamental topics such as regression, interpolation, numerical integration, distributional discrepancies, and statistical dependence, as well as sample path properties of Gaussian processes. A unifying perspective for these equivalences is established, based on the equivalence between the Gaussian Hilbert space and the RKHS. The monograph serves as a basis to bridge many other methods based on Gaussian processes and reproducing kernels, which are developed in parallel by the two research communities.

Explore similar work

Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA
Jul 23, 2026Sergio A. AlvarezKernel Hilbert Spaces Principal Component Analysis
Gaussian mixture models in Hilbert spaces via kernel methods
May 7, 2026Daniel López-Montero, Antonio Álvarez-López, Marcos MatabuenaKernel Hilbert Spaces Gaussian Mixture Models
Scalable Gaussian process inference via neural feature maps
May 11, 2026Anthony StephensonGaussian Process Kernel Method

Jul 23, 2026 · cs.LGJ/K move · Enter open · S save

Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA

Sergio A. Alvarez

We show that, up to isotropic scaling, the Gaussian RBF reproducing kernel Hilbert space (RKHS) is asymptotically isometric to Euclidean space in the large bandwidth limit. This strongly suggests that kernel-based constructions reliant on metric properties of the RKHS will yield results for Gaussian RBF kernels that similarly approach those of linear kernels for large bandwidths. The asymptotic behavior of Gaussian CKA can be understood in this light. We further consider kernel PCA, showing that Gaussian RBF eigenvalues, eigenprojections, and principal components all converge to those of classical (linear) PCA as bandwidth

σ\rightarrow \infty

. For a given data representation, both the RKHS feature embeddings and the orthogonal PCA eigenframes of the two kernel types differ asymptotically by a geometric similarity transformation, up to a residual of size

O \left (\fracρσ \right )^2

, where

ρ

is a measure of geometric eccentricity of the representation, equal to the ratio of maximum to median pairwise distance between data examples. Experiments over a diverse collection of data sets demonstrate that

ρ

provides a simple and reliable predictor of dataset-specific convergence behavior in the top principal directions.

Kernel Hilbert Spaces Principal Component Analysis Radial Basis Function Asymptotics Near-Earth Objects

Open paper Ask about it PDF

Abstract

Explore similar work

Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA

Gaussian mixture models in Hilbert spaces via kernel methods

Scalable Gaussian process inference via neural feature maps