Regularization Model
Regularization models aim to improve the performance and robustness of machine learning models by constraining their complexity and preventing overfitting. Current research focuses on understanding the impact of various regularization techniques—including L1, L2, and other non-convex penalties, label smoothing, and data augmentation methods—on model calibration, robustness to noise and adversarial attacks, and generalization to unseen data, often within the context of specific architectures like GANs, ResNets, and RNNs. These investigations are crucial for advancing the reliability and applicability of machine learning across diverse fields, from medical imaging and natural language processing to control systems and inverse problems. Improved regularization strategies lead to more accurate, stable, and efficient models, particularly in scenarios with limited data or high dimensionality.
Papers
Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes
Dan Qiao, Kaiqi Zhang, Esha Singh, Daniel Soudry, Yu-Xiang Wang
Geometric sparsification in recurrent neural networks
Wyatt Mackey, Ioannis Schizas, Jared Deighton, David L. Boothe,, Vasileios Maroulas
Decoupling regularization from the action space
Sobhan Mohammadpour, Emma Frejinger, Pierre-Luc Bacon
Enhancing Performance for Highly Imbalanced Medical Data via Data Regularization in a Federated Learning Setting
Georgios Tsoumplekas, Ilias Siniosoglou, Vasileios Argyriou, Ioannis D. Moscholios, Panagiotis Sarigiannidis
Dual sparse training framework: inducing activation map sparsity via Transformed $\ell1$ regularization
Xiaolong Yu, Cong Tian