Kernel Regression
Kernel regression is a machine learning technique that uses kernel functions to estimate a function from data, aiming to achieve accurate and efficient prediction. Current research focuses on improving the adaptability and generalization capabilities of kernel regression, exploring variations like physics-informed kernel learning and over-parameterized gradient descent, as well as analyzing its behavior in high-dimensional settings and under various data distributions. These advancements are significant for enhancing the performance and understanding of kernel methods in diverse applications, including protein variant effect prediction, Hamiltonian system learning, and even improving the efficiency of large language models.
Papers
FAVOR#: Sharp Attention Kernel Approximations via New Classes of Positive Random Features
Valerii Likhosherstov, Krzysztof Choromanski, Avinava Dubey, Frederick Liu, Tamas Sarlos, Adrian Weller
Bayes-optimal Learning of Deep Random Networks of Extensive-width
Hugo Cui, Florent Krzakala, Lenka Zdeborová