Sparsity Constraint

Sparsity constraints in machine learning aim to reduce model complexity by limiting the number of non-zero parameters, thereby improving efficiency and potentially generalization. Current research focuses on developing efficient algorithms for imposing sparsity during training (e.g., using iterative pruning, gradient-based methods, and combinatorial optimization) and applying these techniques to various model architectures, including convolutional neural networks and large language models, within federated learning settings. This work is significant because it addresses the computational burden of large models, enhances privacy in distributed learning, and improves the interpretability of models by identifying important features.

Papers

December 21, 2022

Efficient First-order Methods for Convex Optimization with Strongly Convex Function Constraints
Zhenwei Lin, Qi Deng
Convex Optimization Primal Dual Sparsity Constraint Strong Convexity Convex Constraint Concave Saddle Point Problem

November 1, 2022

Batch Active Learning from the Perspective of Sparse Approximation
Maohao Shen, Bowen Jiang, Jacky Yibo Zhang, Oluwasanmi Koyejo
Active Learning Visual Perspective Model Training Sparsity Constraint Sparse Approximation Batch Active Learning

October 14, 2022

Neural Network Compression by Joint Sparsity Promotion and Redundancy Reduction
Tariq M. Khan, Syed S. Naqvi, Antonio Robles-Kelly, Erik Meijering
Convolutional Neural Network Model Neural Network Compression Sparse Structure Feature Redundancy Sparsity Constraint Redundancy Reduction

October 6, 2022

A Novel Maximum-Entropy-Driven Technique for Low-Rank Orthogonal Nonnegative Matrix Factorization with $\ell_0$-Norm sparsity Constraint
Salar Basiri, Srinivasa Salapaka
Matrix Factorization Low Rank Maximum Entropy Sparsity Constraint Non Negative Matrix

July 15, 2022

Sparse Relational Reasoning with Object-Centric Representations
Alex F. Spies, Alessandra Russo, Murray Shanahan
CNN Model Object Centric Representation Sparsity Constraint Task Relationship Relational Representation Learning Sparse Knowledge Graph

July 8, 2022

L$_0$onie: Compressing COINs with L$_0$-constraints
Juan Ramirez, Jose Gallego-Posada
Sparsity Increase Compression Technique Sparsity Constraint Functional Compression

May 30, 2022

Holistic Generalized Linear Models
Benjamin Schwendinger, Florian Schwendinger, Laura Vana
Generalized Linear Model Sparsity Constraint Task Constraint Linear Constraint

May 24, 2022

Medical Scientific Table-to-Text Generation with Human-in-the-Loop under the Data Sparsity Constraint
Heng-Yi Wu, Jingqing Zhang, Julia Ive, Tong Li, Vibhor Gupta, Bingyuan Chen, Yike Guo
Text Generation Structured Data Language Generation Model Sparsity Constraint Validation Report Table Summarization

April 13, 2022

Generalization Error Bounds for Multiclass Sparse Linear Classifiers
Tomer Levy, Felix Abramovich
High Dimensional Generalization Error Bound Sparsity Constraint Sparse Logistic Regression

February 24, 2022

Support Recovery in Mixture Models with Sparse Parameters
Arya Mazumdar, Soumyabrata Pal
Latent Space Mixture Model Sparsity Constraint Multivariate Gaussian

January 7, 2022

Sparse PCA on fixed-rank matrices
Alberto Del Pia
Many Sparse PCa Detection Sparsity Constraint Sparse Principal Component Analysis Principal Component

November 10, 2021

Efficient Neural Network Training via Forward and Backward Propagation Sparsification
Xiao Zhou, Weizhong Zhang, Zonghao Chen, Shizhe Diao, Tong Zhang
Neural Network Sparse Training Sparse Structure Backward Propagation Sparsity Constraint