Sparse Regularization

Sparse regularization techniques aim to improve the efficiency and robustness of machine learning models by encouraging sparsity in their parameters, leading to simpler, more interpretable models and reduced computational costs. Current research focuses on applying sparse regularization to various model architectures, including deep neural networks and vision transformers, often employing algorithms like ADMM and LARS to efficiently solve the resulting optimization problems. This approach is proving valuable in diverse applications, from image processing and speech recognition to time series prediction and solving partial differential equations, by enhancing model performance, generalizability, and interpretability while reducing computational burden.

Papers

October 19, 2024

2D Basement Relief Inversion using Sparse Regularization
Francisco Márcio Barboza, Arthur Anthony da Cunha Romão E Silva, Bruno Motta de Carvalho
Inverse Problem Regularization Technique Sparse Regularization

August 28, 2024

Evaluating Model Robustness Using Adaptive Sparse L0 Regularization
Weiyou Liu, Zhenyang Li, Weitong Chen
Adversarial Attack Adversarial Example Model Robustness Sparse Regularization Norm Attack

June 25, 2024

MSRS: Training Multimodal Speech Recognition Models from Scratch with Sparse Mask Optimization
Adriana Fernandez-Lopez, Honglie Chen, Pingchuan Ma, Lu Yin, Qiao Xiao, Stavros Petridis, Shiwei Liu, Maja Pantic
Speech Recognition Scratch Project Audio Visual Speech Recognition Sparse Mask Sparse Regularization

June 12, 2024

Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality
William Kengne, Modou Wade
Loss Function Evolving Observation Minimax Optimality Explicit Regularization Sparse Regularization Deep Neural Network Estimator

November 30, 2023

Piecewise Linearity of Min-Norm Solution Map of a Nonconvexly Regularized Convex Sparse Model
Yi Zhang, Isao Yamada
Sparse Model General Nonconvex Solution Space Sparse Regularization LASSO Regression Regularization Path

November 22, 2023

Single Image Compressed Sensing MRI via a Self-Supervised Deep Denoising Approach
Marlon Bran Lorenzana, Feng Liu, Shekhar S. Chandra
Noisy Image Self Supervised Denoising Reconstruction Model Sparse Regularization Compressed Sensing MRI Sparse Reconstruction

August 17, 2023

Semi-sparsity Priors for Image Structure Analysis and Extraction
Junqing Huang, Haihui Wang, Michael Ruzhansky
Regularization Model Process Extraction Image Decomposition Sparse Regularization Image Structure Sparsity Prior Text Mining Model Cartoon Texture

July 22, 2023

Sparse then Prune: Toward Efficient Vision Transformers
Yogi Prasetyo, Novanto Yudistira, Agus Wahyu Widodo
Vision Transformer Many Sparse Edge Pruning Model Pruning Vision Transformer Architecture Vision Transformer Model Sparse Regularization

July 7, 2023

Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization
Chris Kolb, Christian L. Müller, Bernd Bischl, David Rügamer
Manipulation Edge Detection Module Derivative Free Optimization Sparse Regularization Parameter Efficient Sparsity Hadamard Layer Smooth Optimization

January 23, 2023

Deep Learning Meets Sparse Regularization: A Signal Processing Perspective
Rahul Parhi, Robert D. Nowak
Deep Learning Deep Neural Network Signal Processing Sparse Regularization Decay Regularization Mathematical Discovery

January 18, 2023

A Novel Sparse Regularizer
Hovig Tigran Bayandorian
Sparse Regularization L2 Regularization

October 24, 2022

Revisiting Sparse Convolutional Model for Visual Recognition
Xili Dai, Mingyang Li, Pengyuan Zhai, Shengbang Tong, Xingjian Gao, Shao-Lun Huang, Zhihui Zhu, Chong You, Yi Ma
Visual Recognition Sparse Regularization Sparse Convolutional Convolutional Sparse

October 21, 2022

Geometric Sparse Coding in Wasserstein Space
Marshall Mueller, Shuchin Aeron, James M. Murphy, Abiy Tasissa
Wasserstein Space Wasserstein Barycenter Sparse Regularization Barycentric Coordinate Sparse Intensity Change

September 13, 2022

Sparse deep neural networks for modeling aluminum electrolysis dynamics
Erlend Torje Berg Lundby, Adil Rasheed, Ivar Johan Halvorsen, Jan Tommy Gravdahl
Deep Neural Network Many Sparse Regularization Model Sparse Model Sparse Neural Network Sparse Regularization Modular Electrolysis Plant

July 27, 2022

Sparse Deep Neural Network for Nonlinear Partial Differential Equations
Yuesheng Xu, Taishan Zeng
Sparse Deep Sparse Regularization Multi Scale Sparse Sparsity Structure Adaptive Approximation

July 13, 2022

Majorization-minimization for Sparse Nonnegative Matrix Factorization with the $\beta$-divergence
Arthur Marmin, José Henrique de Morais Goulart, Cédric Févotte
Matrix Factorization Majorization Minimization Beta Divergence Sparse Regularization Norm Regularization Log Gradient

April 23, 2022

Distributed Dynamic Safe Screening Algorithms for Sparse Regularization
Runxue Bao, Xidong Wu, Wenhan Xian, Heng Huang
High Dimensional Large Scale Learning Sparse Regularization Safe Screening Dynamic Screening

April 19, 2022

Quaternion Optimized Model with Sparse Regularization for Color Image Recovery
Liqiao Yang, Yang Liu, Kit Ian Kou
Full Model Low Rank Color Image Sparse Regularization Quaternion Domain Quaternion Matrix Quaternion Matrix Completion

March 5, 2022

$\ell_1$DecNet+: A new architecture framework by $\ell_1$ decomposition and iteration unfolding for sparse feature segmentation
Yumeng Ren, Yiming Gao, Chunlin Wu, Xue-cheng Tai
Microbial Decomposition Deep Unfolding Sparse Feature Sparse Regularization Architecture Framework Sparse Segmentation