Kernel Methods

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 125% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 168

All topics

Predicting kernel regression learning curves from only raw data statistics
Oct 16, 2025Dhruva Karkada, Joseph Turnbull, Yuxi Liu +1Kernel Regression Kernel Methods
UC Berkeley · Imbue
Data-efficient Kernel Methods for Learning Hamiltonian Systems
Sep 21, 2025Yasamin Jalalian, Mostafa Samir, Boumediene Hamzi +2Dynamical Systems Nonlinear System Identification
Department of Computing and Mathematical Sciences, Caltech, CA, USA. · Beyond Limits, CA, USA. · The Alan Turing Institute, London, UK. +1
A Compositional Kernel Model for Feature Learning
Sep 17, 2025Feng Ruan, Keli Liu, Michael JordanKernel Ridge Regression Kernel Regression
*Department of Statistics and Data Science, Northwestern University. · *Company E · *Department of Statistics and Computer Science, University of California, Berkeley.
A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning
Sep 14, 2025Jia-Qi Yang, Lei ShiStochastic Approximation Reproducing Kernel Hilbert Spaces
School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.
The kernel of graph indices for vector search
Jun 25, 2025Mariano Tepper, Ted WillkeGraph-Based Retrieval Kernel Methods
IBM WatsonX
Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalences
Jun 20, 2025Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic +1Gaussian Processes Reproducing Kernel Hilbert Spaces
EURECOM · University of Tübingen, Tübingen AI Center Max Planck Institute for Intelligent Systems · Adelaide University Australian Institute for Machine Learning +1
Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation
May 21, 2025Kun Fang, Qinghua Tao, Mingzhen He +6Principal Component Analysis Kernel Methods
Department of Automation, Shanghai Jiao Tong University · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University · School of Automation, Beijing Institute of Technology +2
Out-of-Sample Embedding with Proximity Data: Projection versus Restricted Reconstruction
May 10, 2025Michael W. Trosset, Kaiyi Tan, Minh Tang +1Kernel Methods Dimensionality Reduction
Department of Statistics, Indiana University. · Department of Statistics, North Carolina State University. · Department of Applied Mathematics & Statistics, Johns Hopkins University.
Benign Overfitting with Quantum Kernels
Mar 21, 2025Joachim Tomasi, Sandrine Anthoine, Hachem KadriBenign Overfitting Quantum Machine Learning
Aix-Marseille Univ, CNRS, LIS, Marseille, France. · Aix Marseille Univ, CNRS, I2M, Marseille, France.
Physics-Informed Support Vector Kernels via Green-Function Analogies and Jackson-Chebyshev Spectral Design
Feb 16, 2025Nan-Hong Kuo, Renata WongSupport Vector Machines Kernel Methods
Department of Physics, National Taiwan University, Taipei, Taiwan · Department of Artificial Intelligence, Chang Gung University, Taoyuan, Taiwan · Department of Neurology, Chang Gung Memorial Hospital, Keelung, Taiwan +1
Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer
Jan 18, 2025Haotian Lin, Matthew ReimherrSpectral Regularization Concept Drift
Department of Statistics, The Pennsylvania State University.
Which Spaces can be Embedded in $L_p$ -type Reproducing Kernel Banach Space? A Characterization via Metric Entropy
Oct 14, 2024Yiping Lu, Daozhe Lin, Qiang DuReproducing Kernel Hilbert Spaces Kernel Methods
Industrial Engineering & Management Sciences, Northwestern University · Department of Applied Physics & Applied Mathematics, Columbia University
Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent
Sep 13, 2024Sayan Banerjee, Krishnakumar Balasubramanian, Promit GhosalKernel Methods Variational Inference
Department of Statistics and Operations Research, University of North Carolina, Chapel Hill. · Department of Statistics, University of California, Davis. · Department of Statistics, University of Chicago.
Nonparametric Control Koopman Operators
May 12, 2024Petar Bevanda, Bas Driessen, Lucian Cristian Iacob +3Koopman Operator Learning System Identification
Chair of Information-oriented Control, TU München, Germany · Control Systems Group, TU/e, Eindhoven, The Netherlands · Systems and Control Laboratory, HUNREN Institute for Computer Science and Control, Budapest, Hungary +2
Consistent Distributed Ranking of Generative Models via Kernel Distances
Oct 18, 2023Zixiao Wang, Farzan Farnia, Zhenghao Lin +2Kernel Methods Image Generation Evaluation
Distribution-Free Uncertainty Quantification for Kernel Methods by Gradient Perturbations
Dec 23, 2018Balázs Csanád Csáji, Krisztián Balázs KisConfidence Region Estimation Uncertainty Quantification
EPIC Centre of Excellence MTA SZTAKI: Institute for Computer Science and Control Hungarian Academy of Sciences, Budapest, Hungary
Classical and quantum kernel fusion for two-sample testing
Date pendingYu Terada, Yugo Ogio, Ken Arai +2Two-Sample Testing Maximum Mean Discrepancy
Advanced Research Laboratory, Sony Group Corporation, 1-7-1 Konan, Minato-ku, Tokyo 108-0075, Japan · Deloitte Tohmatsu Consulting LLC.
When do cheap embeddings beat protein language models? A theoretically-grounded hashing sketch for biological sequence classification
Date pendingSarwan Ali, Taslim Murad, Imdadullah Khan +1Protein Language Models Randomized Sketching
Columbia University, NY, USA · Institute of Business Administration, Karachi, Pakistan · Lahore University of Management Sciences, Lahore, Pakistan +1

Oct 16, 2025, cs.LGJ/K move · Enter open · S save

Predicting kernel regression learning curves from only raw data statistics

Dhruva Karkada, Joseph Turnbull, Yuxi Liu, James B. Simon

UC Berkeley · Imbue

Kernel Regression Kernel Methods Statistical Learning Theory

Open paper AskPDF arXiv

We study kernel regression with common rotation-invariant kernels on real datasets including CIFAR-5m, SVHN, and ImageNet. We give a theoretical framework that predicts learning curves (test risk vs. sample size) from only two measurements: the empirical data covariance matrix and an empirical polynomial decomposition of the target function

f_*

. The key new idea is an analytical approximation of a kernel's eigenvalues and eigenfunctions with respect to an anisotropic data distribution. The eigenfunctions resemble Hermite polynomials of the data, so we call this approximation the Hermite eigenstructure ansatz (HEA). We prove the HEA for Gaussian data, but we find that real image data is often "Gaussian enough" for the HEA to hold well in practice, enabling us to predict learning curves by applying prior results relating kernel eigenstructure to test risk. Extending beyond kernel regression, we empirically find that MLPs in the feature-learning regime learn Hermite polynomials in the order predicted by the HEA. Our HEA framework is a proof of concept that an end-to-end theory of learning which maps dataset structure all the way to model performance is possible for nontrivial learning algorithms on real datasets.

Momentum

Related topics

Latest papers 168