Implicit Kernel

Implicit kernels represent a powerful approach to designing efficient and expressive machine learning models by implicitly defining kernel functions rather than explicitly storing their parameters. Current research focuses on leveraging implicit kernels within neural networks, particularly through novel architectures like hyper-convolutions and coordinate-based MLPs, to achieve full context interaction and improved performance with fewer parameters. This approach is proving valuable in diverse applications, including time series analysis, image classification (especially in medical imaging), and modeling spatio-temporal geophysical data, by enabling the incorporation of prior knowledge and the handling of nonlinear relationships. The resulting models offer advantages in terms of parameter efficiency, training speed, and the ability to capture complex patterns in data.

Papers

March 8, 2024

Exploring the Links between the Fundamental Lemma and Kernel Regression
Oleksii Molodchyk, Timm Faulwasser
Kernel Regression Fundamental Lemma Internet Link Natural Extension Implicit Kernel

January 31, 2024

HyperZ$\cdot$Z$\cdot$W Operator Connects Slow-Fast Networks for Full Context Interaction
Harvie Zhang
Weight Matrix Coordinate Based Task Specific Weight Context Interaction Implicit Kernel

December 15, 2022

Let's consider more general nonlinear approaches to study teleconnections of climate variables
D. Bueso, M. Piles, G. Camps-Valls
Principal Component Analysis Nonlinear Model PCa Detection Weather Variable Spatiotemporal Signal Hilbert Transform Implicit Kernel

May 15, 2022

Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernel
Ziyang Jiang, Tongshu Zheng, Yiling Liu, David Carlson
Neural Network Deep Learning Convolutional Neural Network Prior Knowledge Kernel Function Multiple Kernel Implicit Kernel

February 6, 2022

Hyper-Convolutions via Implicit Kernels for Medical Imaging
Tianyu Ma, Alan Q. Wang, Adrian V. Dalca, Mert R. Sabuncu
Convolutional Neural Network Medical Imaging Convolutional Kernel Hyper Convolution Implicit Kernel

Implicit Kernel

Papers

Exploring the Links between the Fundamental Lemma and Kernel Regression

HyperZ$\cdot$Z$\cdot$W Operator Connects Slow-Fast Networks for Full Context Interaction

Let's consider more general nonlinear approaches to study teleconnections of climate variables

Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernel

Hyper-Convolutions via Implicit Kernels for Medical Imaging