Deep Kernel

Deep kernel learning (DKL) combines the power of deep neural networks with the probabilistic framework of Gaussian processes, aiming to improve the efficiency and accuracy of machine learning models, particularly in high-dimensional spaces. Current research focuses on developing DKL methods for various applications, including Bayesian optimization, independence testing, and image reconstruction, often employing architectures that integrate deep kernels with Gaussian processes or other models like genetic algorithms. This approach offers advantages in handling complex data, providing robust uncertainty quantification, and improving sample efficiency in optimization problems, impacting fields ranging from materials science and medical imaging to robotics and time series analysis.

Papers

October 13, 2024

A resource-efficient model for deep kernel learning
Luisa D'Amore
Model Based Learning Model Efficient Model Deep Kernel Formula Based Computation

October 4, 2024

Rapid optimization in high dimensional space by deep kernel learning augmented genetic algorithms
Mani Valleti, Aditya Raghavan, Sergei V. Kalinin
Bayesian Optimization Genetic Algorithm Deep Kernel Learning Deep Kernel Deep Learning Based Surrogate

September 10, 2024

July 18, 2024

Data-driven Forecasting of Deep Learning Performance on GPUs
Seonho Lee, Amar Phanishayee, Divya Mahajan
Deep Learning Deep Learning Model Data Driven Single GPU Deep Learning Workload Deep Kernel

March 30, 2024

Bayesian Exploration of Pre-trained Models for Low-shot Image Classification
Yibo Miao, Yu Lei, Feng Zhou, Zhijie Deng
Pre Trained Model Pre Trained Convolutional Neural Network Deep Kernel Low Shot

December 13, 2023

Meta-learning to Calibrate Gaussian Processes with Deep Kernels for Regression Uncertainty Estimation
Tomoharu Iwata, Atsutoshi Kumagai
Gaussian Process Deep Kernel Regression Uncertainty Estimation

October 11, 2023

Deep Kernel and Image Quality Estimators for Optimizing Robotic Ultrasound Controller using Bayesian Optimization
Deepak Raina, SH Chandrashekhara, Richard Voyles, Juan Wachs, Subir Kumar Saha
High Dimensional Bayesian Optimization Image Quality Assessment Deep Kernel Robotic Ultrasound System Autonomous Robotic Ultrasound Ultrasound Image Quality Ultrasound Probe

October 9, 2023

Integration-free Training for Spatio-temporal Multimodal Covariate Deep Kernel Point Processes
Yixuan Zhang, Quyu Kong, Feng Zhou
Training Free Point Process Deep Kernel

June 14, 2023

MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting
Felix Biggs, Antonin Schrab, Arthur Gretton
LeArning Abstract Maximum Mean Discrepancy Data Splitting Two Sample Deep Kernel Kernel Test

June 12, 2023

Combining Primal and Dual Representations in Deep Restricted Kernel Machines Classifiers
Francesco Tonin, Panagiotis Patrinos, Johan A. K. Suykens
Kernel Principal Component Analysis Kernel Machine Deep Kernel

May 23, 2023

DF2M: An Explainable Deep Bayesian Nonparametric Model for High-Dimensional Functional Time Series
Yirui Liu, Xinghao Qiao, Yulong Pei, Liying Wang
Deep Learning Model High Dimensional Time Series Factor Model Bayesian Nonparametric Deep Kernel

February 20, 2023

Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation
Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L Smith, Yee Whye Teh
Self Attention Propagation Environment Deep Transformer Normalization Layer Shortcut Feature Vanilla Transformer Deep Kernel

February 19, 2023

Guided Deep Kernel Learning
Idan Achituve, Gal Chechik, Ethan Fetaya
Deep Kernel Learning Deep Kernel Infinite Width Neural Network Neural Network Gaussian Process Kernel Optimization

November 20, 2022

Counterfactual Learning with Multioutput Deep Kernels
Alberto Caron, Gianluca Baio, Ioanna Manolopoulou
Causal Effect Structural Causal Model Causal Effect Estimation Counterfactual Inference Counterfactual Learning Deep Kernel

May 13, 2022

A hybrid data driven-physics constrained Gaussian process regression framework with deep kernel for uncertainty quantification
Cheng Chang, Tieyong Zeng
Uncertainty Quantification Gaussian Process Regression Boltzmann Distribution Deep Kernel Hybrid Physic Machine Learning

March 18, 2022

Active learning in open experimental environments: selecting the right information channel(s) based on predictability in deep kernel learning
Maxim Ziatdinov, Yongtao Liu, Sergei V. Kalinin
Active Learning Predictability Problem Quantitative Structure Activity Relationship Deep Kernel Learning Deep Kernel Structure Property Relationship

March 15, 2022

Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers
Guodong Zhang, Aleksandar Botev, James Martens
Deep Learning Deep Neural Network Synchronous Generator Shallow Network Kernel Extension Normalization Layer Shortcut Feature Deep Kernel Vanilla Convolutional Neural Network

Deep Kernel

Papers

A resource-efficient model for deep kernel learning

Rapid optimization in high dimensional space by deep kernel learning augmented genetic algorithms

Learning Deep Kernels for Non-Parametric Independence Testing

Deep kernel representations of latent space features for low-dose PET-MR imaging robust to variable dose reduction

Data-driven Forecasting of Deep Learning Performance on GPUs

Bayesian Exploration of Pre-trained Models for Low-shot Image Classification

Meta-learning to Calibrate Gaussian Processes with Deep Kernels for Regression Uncertainty Estimation

Deep Kernel and Image Quality Estimators for Optimizing Robotic Ultrasound Controller using Bayesian Optimization

Integration-free Training for Spatio-temporal Multimodal Covariate Deep Kernel Point Processes

MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting

Combining Primal and Dual Representations in Deep Restricted Kernel Machines Classifiers

DF2M: An Explainable Deep Bayesian Nonparametric Model for High-Dimensional Functional Time Series

Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation

Guided Deep Kernel Learning

Counterfactual Learning with Multioutput Deep Kernels

A hybrid data driven-physics constrained Gaussian process regression framework with deep kernel for uncertainty quantification

Active learning in open experimental environments: selecting the right information channel(s) based on predictability in deep kernel learning

Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers