Relative Sparsity

Relative sparsity focuses on creating models with a minimal number of parameters that differ from a pre-existing, often established model (e.g., a standard-of-care policy in healthcare or a baseline neural network). Current research emphasizes developing algorithms that achieve this "relative" sparsity while maintaining or even improving performance, often employing techniques like iterative momentum-based pruning or incorporating knowledge distillation within structured pruning frameworks applied to architectures such as ResNet-50. This approach is particularly valuable in applications like healthcare, where interpretability and explainability of model changes are crucial, and in resource-constrained settings where model compression is essential for efficient deployment.

Papers

March 11, 2024

Enhanced Sparsification via Stimulative Training
Shengji Tang, Weihao Lin, Hancheng Ye, Peng Ye, Chong Yu, Baopu Li, Tao Chen
Model Compression Random Sparsification Training Method Sparsity Penalty Relative Sparsity

June 25, 2023

Inference for relative sparsity
Samuel J. Weisenthal, Sally W. Thurston, Ashkan Ertefaie
Scientific Inference Relative Sparsity

November 29, 2022

Relative Sparsity for Medical Decision Problems
Samuel J. Weisenthal, Sally W. Thurston, Ashkan Ertefaie
Data Driven Relevant Covariates Data Driven Decision Clinical Decision Trust Region Policy Optimization Relative Sparsity

August 11, 2022

WeightMom: Learning Sparse Networks using Iterative Momentum-based pruning
Elvis Johnson, Xiaochen Tang, Sriramacharyulu Samudrala
Neural Network Deep Neural Network Sparse Network Image Classification Datasets Sparsity Pruning Weight Monitoring Relative Sparsity

Relative Sparsity

Papers

Enhanced Sparsification via Stimulative Training

Inference for relative sparsity

Relative Sparsity for Medical Decision Problems

WeightMom: Learning Sparse Networks using Iterative Momentum-based pruning