Sparsity Search

Sparsity search focuses on identifying and utilizing the most crucial components within complex systems, such as neural networks or large datasets, to improve efficiency and performance without significant loss of accuracy. Current research emphasizes developing efficient algorithms, including those based on single-stage importance and sparsity evaluations and iterative neural networks, to find optimal sparse subnetworks or representations. This field is significant because it addresses the computational challenges associated with large-scale models and datasets, leading to improved model compression, faster training times, and enhanced interpretability in various applications, from image processing to online retail analytics.

Papers

March 23, 2024

Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression
Hancheng Ye, Chong Yu, Peng Ye, Renqiu Xia, Yansong Tang, Jiwen Lu, Tao Chen, Bo Zhang
Importance Aware Mask Guided Single Stage Dynamic Mask Masked Image Vision Transformer Compression Various Vision Transformer Sparsity Search

March 13, 2024

Random Search as a Baseline for Sparse Neural Network Architecture Search
Rezsa Farahani
Baseline Result Local Search Sparse Network Sparse Subnetworks Sparsity Search

December 19, 2023

New Classes of the Greedy-Applicable Arm Feature Distributions in the Sparse Linear Bandit Problem
Koji Ichikawa, Shinji Ito, Daisuke Hatano, Hanna Sumita, Takuro Fukunaga, Naonori Kakimura, Ken-ichi Kawarabayashi
Greedy Algorithm Contextual Bandit Problem Feature Distribution Arm Selection Virtual ARM Sparse Linear Bandit Sparsity Search

June 11, 2022

A Theoretical Understanding of Neural Network Compression from Sparse Linear Approximation
Wenjing Yang, Ganghua Wang, Jie Ding, Yuhong Yang
Theoretical Understanding Model Compression Linear Compression Neural Network Compression Sparse Function Sparsity Search

June 9, 2022

Convolutional Dictionary Learning by End-To-End Training of Iterative Neural Networks
Andreas Kofler, Christian Wald, Tobias Schaeffter, Markus Haltmeier, Christoph Kolbitsch
Dictionary Learning Blind Source Separation Iterative Neural Network End to End Training Dynamic MR Physic Informed Convolutional Neural Network Sparsity Search Sparsifying Transform

March 2, 2022

A density peaks clustering algorithm with sparse search and K-d tree
Yunxiao Shan, Shu Li, Fuxiang Li, Yuxin Cui, Shuai Li, Ming Zhou, Xiang Li
Practical Algorithm Sparse Matrix Density Peak Clustering Sparsity Search Density Peak Sparse Learning Method

November 11, 2021

Characterization of Frequent Online Shoppers using Statistical Learning with Sparsity
Rajiv Sambasivan, Mark Burgess, Jörg Schad, Arthur Keen, Christopher Woodward, Alexander Geenen, Sachin Sharma
Sparsity Increase E Commerce New Characterization Statistical Learning Sparsity Search Shopping Intent Retail Analytics

Sparsity Search

Papers

Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression

Random Search as a Baseline for Sparse Neural Network Architecture Search

New Classes of the Greedy-Applicable Arm Feature Distributions in the Sparse Linear Bandit Problem

A Theoretical Understanding of Neural Network Compression from Sparse Linear Approximation

Convolutional Dictionary Learning by End-To-End Training of Iterative Neural Networks

A density peaks clustering algorithm with sparse search and K-d tree

Characterization of Frequent Online Shoppers using Statistical Learning with Sparsity