Sparse Kernel

Sparse kernel methods aim to improve the efficiency and effectiveness of machine learning algorithms by reducing computational complexity through strategic sparsity in kernel matrices or neural network weights. Current research focuses on developing efficient algorithms and architectures for sparse convolution, including optimized kernels for GPUs and CPUs, and exploring novel sparse kernel designs for specific applications like 3D point cloud processing and graph neural networks. These advancements are crucial for enabling the application of computationally intensive machine learning models to large-scale datasets and resource-constrained environments, impacting fields such as autonomous driving, computer vision, and recommendation systems.

Papers

March 22, 2024

LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels
Tuo Feng, Wenguan Wang, Fan Ma, Yi Yang
3D Perception Selective Kernel Sparse Kernel 3D Kernel

January 30, 2024

Robust Kernel Sparse Subspace Clustering
Ivica Kopriva
Subspace Clustering Sparse Kernel Kernel Spectral Clustering

October 25, 2023

TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
Haotian Tang, Shang Yang, Zhijian Liu, Ke Hong, Zhongming Yu, Xiuyu Li, Guohao Dai, Yu Wang, Song Han
Graph Convolution Single GPU Efficient Training Inference Framework Sparse Convolution Sparse Kernel

September 30, 2023

The Sparsity Roofline: Understanding the Hardware Limits of Sparse Neural Networks
Cameron Shinn, Collin McCarthy, Saurav Muralidharan, Muhammad Osama, John D. Owens
Sparsity Increase Sparse Neural Network Sparsity Pattern Hardware Constraint Sparse Kernel

July 8, 2023

Rosko: Row Skipping Outer Products for Sparse Matrix Multiplication Kernels
Vikas Natesh, Andrew Sabot, H. T. Kung, Mark Ting
Parameter Efficient Sparsity Sparse Matrix Multiplication Sparse Kernel Matrix Multiplication Kernel Kernel Trick Scheduling Method Row Skipping Outer Product

May 21, 2023

Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces
Rui Wang, Yuesheng Xu, Mingsong Yan
LeArning Abstract Sparsity Increase Kernel Banach Space Sparse Kernel Representer Theorem Sparse Learning Method

December 19, 2022

Anticancer Peptides Classification using Kernel Sparse Representation Classifier
Ehtisham Fazal, Muhammad Sohail Ibrahim, Seongyong Park, Imran Naseem, Abdul Wahab
Sparse Representation Amino Acid Synthetic Minority Oversampling Technique Sparse Kernel Anticancer Peptide

December 10, 2022

Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation
Nicolas Nytko, Ali Taghibakhshi, Tareq Uz Zaman, Scott MacLachlan, Luke N. Olson, Matt West
Sparse Representation Sparse Matrix Dense Representation Mode Automatic Differentiation Sparse Kernel

October 4, 2022

Log-Linear-Time Gaussian Processes Using Binary Tree Kernels
Michael K. Cohen, Samuel Daulton, Michael A. Osborne
Gaussian Process Gaussian Process Regression Sparse Kernel Subtree Kernel Sparse GP

March 15, 2022

Distributed-Memory Sparse Kernels for Machine Learning
Vivek Bharadwaj, Aydin Buluç, James Demmel
Machine Learning Sparse Matrix Sparse Kernel Matrix Multiplication Kernel Dense Matrix Multiplication