Kernel Optimization

Kernel optimization focuses on improving the efficiency and performance of kernel-based algorithms, primarily in machine learning, by finding optimal implementations for various operations on data tensors. Current research emphasizes automated search strategies, such as combining exploration and exploitation phases, and adapting these techniques for specific hardware architectures (CPUs, GPUs, MCUs) and model types (deep neural networks, large language models). These advancements are crucial for deploying computationally intensive models on resource-constrained devices and accelerating inference speed, impacting fields ranging from edge computing to scientific simulations.

Papers

October 31, 2024

Scalable Kernel Inverse Optimization
Youyuan Long, Tolga Ok, Pedro Zattoni Scroccaro, Peyman Mohajerin Esfahani
Kernel Method Nonconvex Optimization Sequential Selection Kernel Optimization

October 14, 2024

Liger Kernel: Efficient Triton Kernels for LLM Training
Byron (Pin-Lun)Hsu, Yun Dai, Vignesh Kothapalli, Qingquan Song, Shao Tang, Siyu Zhu, Steven Shimizu, Shivam Sahni, Haowen Ning, Yanning Chen
Small Kernel Kernel Extension LLM Training Large Language Model Training Kernel Optimization

June 28, 2024

Explore as a Storm, Exploit as a Raindrop: On the Benefit of Fine-Tuning Kernel Schedulers with Coordinate Descent
Michael Canesche, Gaurav Verma, Fernando Magno Quintao Pereira
Coordinate Descent Kernel Optimization Kernel Trick

June 13, 2024

Center-Sensitive Kernel Optimization for Efficient On-Device Incremental Learning
Dingwen Zhang, Yan Li, De Cheng, Nannan Wang, Junwei Han
Edge Intelligence Device Training Incremental Learning Framework Kernel Optimization Device Continual Learning

May 1, 2024

vMCU: Coordinated Memory Management and Kernel Optimization for DNN Inference on MCUs
Size Zheng, Renze Chen, Meng Li, Zihao Ye, Luis Ceze, Yun Liang
Neural Network Low Power DNN Inference Dynamic Memory Kernel Optimization

November 1, 2023

Efficient LLM Inference on CPUs
Haihao Shen, Hanwen Chang, Bo Dong, Yu Luo, Hengyu Meng
Large Language Model LLM Inference Kernel Optimization INT4 Quantization Optimal Inference

October 9, 2023

Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels
Da Long, Wei W. Xing, Aditi S. Krishnapriyan, Robert M. Kirby, Shandian Zhe, Michael W. Mahoney
Kernel Regression Equation Discovery Kernel Optimization Spike and Slab

September 16, 2023

Accelerating In-Browser Deep Learning Inference on Diverse Edge Clients through Just-in-Time Kernel Optimizations
Fucheng Jia, Shiqi Jiang, Ting Cao, Wei Cui, Tianrui Xia, Xu Cao, Yuanchun Li, Deyu Zhang, Ju Ren, Yunxin Liu, Lili Qiu, Mao Yang
Tensor Program Tensor Compiler Kernel Optimization Online Deep Learning

July 7, 2023

QIGen: Generating Efficient Kernels for Quantized Inference on Large Language Models
Tommaso Pegolotti, Elias Frantar, Dan Alistarh, Markus Püschel
Code Generation Generative Modeling Kernel Optimization Accuracy Constraint Based Inference

February 19, 2023

Guided Deep Kernel Learning
Idan Achituve, Gal Chechik, Ethan Fetaya
Deep Kernel Learning Deep Kernel Neural Network Gaussian Process Infinite Width Neural Network Kernel Optimization

February 14, 2023

ML-driven Hardware Cost Model for MLIR
Dibyendu Das, Sandya Mannarswamy
Cost Model Deep Learning Compiler Kernel Optimization

August 22, 2022

Scale invariant process regression: Towards Bayesian ML with minimal assumptions
Matthias Wieler
Visual Analogue Scale Regularization Model Gaussian Process Regression Modeling Assumption Kernel Regime Kernel Optimization Extrapolation Performance Invariant Smoothing

March 29, 2022

Kernel Modulation: A Parameter-Efficient Method for Training Convolutional Neural Networks
Yuhuang Hu, Shih-Chii Liu
Convolutional Neural Network Parameter Efficient High Performing 1D ConvNet Kernel Optimization

Kernel Optimization

Papers

Scalable Kernel Inverse Optimization

Liger Kernel: Efficient Triton Kernels for LLM Training

Explore as a Storm, Exploit as a Raindrop: On the Benefit of Fine-Tuning Kernel Schedulers with Coordinate Descent

Center-Sensitive Kernel Optimization for Efficient On-Device Incremental Learning

vMCU: Coordinated Memory Management and Kernel Optimization for DNN Inference on MCUs

Efficient LLM Inference on CPUs

Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels

Accelerating In-Browser Deep Learning Inference on Diverse Edge Clients through Just-in-Time Kernel Optimizations

QIGen: Generating Efficient Kernels for Quantized Inference on Large Language Models

Guided Deep Kernel Learning

ML-driven Hardware Cost Model for MLIR

Scale invariant process regression: Towards Bayesian ML with minimal assumptions

Kernel Modulation: A Parameter-Efficient Method for Training Convolutional Neural Networks