Explicit Regularization

Explicit regularization in machine learning involves adding penalty terms to optimization objectives to constrain model complexity and improve generalization. Recent research focuses on understanding the interplay between explicit and implicit regularization, particularly in deep learning models like convolutional neural networks and transformers, and how this interplay affects performance in various tasks such as image restoration, matrix completion, and continual learning. This research is crucial for developing more robust and efficient machine learning algorithms, addressing challenges like overfitting and catastrophic forgetting, and ultimately improving the reliability and performance of AI systems across diverse applications.

Papers

December 14, 2022

ECON: Explicit Clothed humans Optimized via Normal integration
Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, Michael J. Black
Real Human Surface Reconstruction 3D Human Body Model Explicit Regularization Explicit Content Normal Distribution Free Form

December 1, 2022

Regularization Trade-offs with Fake Features
Martin Hellkvist, Ayça Özçelikkale, Anders Ahlén
Implicit Regularization Ridge Regression Model Misspecification Explicit Regularization Fake Document

August 11, 2022

Adaptive and Implicit Regularization for Matrix Completion
Zhemin Li, Tao Sun, Hongxia Wang, Bao Wang
Adaptive Importance Implicit Regularization Matrix Completion Explicit Regularization Low Rank Regularization

July 20, 2022

Structural Causal 3D Reconstruction
Weiyang Liu, Zhen Liu, Liam Paull, Adrian Weller, Bernhard Schölkopf
Latent Space Causal Graph Implicit Regularization Explicit Regularization

June 16, 2022

Towards Better Understanding with Uniformity and Explicit Regularization of Embeddings in Embedding-based Neural Topic Models
Wei Shao, Lei Huang, Shuqi Liu, Shihua Ma, Linqi Song
Jina Embeddings Topic Analysis Deep Understanding Neural Topic Model Uniformity Metric Explicit Regularization

June 9, 2022

Explicit Regularization in Overparametrized Models via Noise Injection
Antonio Orvieto, Anant Raj, Hans Kersting, Francis Bach
Gradient Descent Parameterized Model Explicit Regularization Noise Injection Vanilla Gradient Descent

June 8, 2022

Generative Myocardial Motion Tracking via Latent Space Exploration with Biomechanics-informed Prior
Chen Qin, Shuo Wang, Chen Chen, Wenjia Bai, Daniel Rueckert
Latent Space Motion Tracking Explicit Regularization Cardiac Motion Cardiac MRI Dataset Myocardial Strain

May 25, 2022

On the Interpretability of Regularisation for Neural Networks Through Model Gradient Similarity
Vincent Szolnoky, Viktor Andersson, Balazs Kulcsar, Rebecka Jörnsten
Neural Network Inherent Interpretability Regularization Model Explicit Regularization Regularization Property Gradient Similarity

May 18, 2022

Large Neural Networks Learning from Scratch with Very Few Data and without Explicit Regularization
Christoph Linse, Thomas Martinetz
Neural Network Convolutional Neural Network Training Data CNN Model Scratch Project Fine Grained Image Classification Explicit Regularization

March 7, 2022

Regularising for invariance to data augmentation improves supervised learning
Aleksander Botev, Matthias Bauer, Soham De
LeArning Abstract Data Augmentation Supervised ImageNet Alpha Invariance Explicit Regularization Regularization Property Augmentation Composition

March 6, 2022

Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization
Liwei Jiang, Yudong Chen, Lijun Ding
Gradient Descent Matrix Factorization Nonconvex Optimization Low Rank Approximation Explicit Regularization

February 22, 2022

Explicit Regularization via Regularizer Mirror Descent
Navid Azizan, Sahin Lale, Babak Hassibi
Stochastic Gradient Descent Implicit Regularization Mirror Descent Explicit Regularization Stochastic Mirror Descent

February 11, 2022

Fast Adversarial Training with Noise Augmentation: A Unified Perspective on RandStart and GradAlign
Axi Niu, Kang Zhang, Chaoning Zhang, Chenshuang Zhang, In So Kweon, Chang D. Yoo, Yanning Zhang
Unified View Gradient Alignment Explicit Regularization Fast Adversarial Training Catastrophic Overfitting Adversarial Training Algorithm Noise Augmentation

January 27, 2022

Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks
Noam Razin, Asaf Maman, Nadav Cohen
Deep Convolutional Neural Network Implicit Regularization Tensor Factorization Explicit Regularization Factor Tensor

December 9, 2021

DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
Aviral Kumar, Rishabh Agarwal, Tengyu Ma, Aaron Courville, George Tucker, Sergey Levine
Deep Reinforcement Learning Implicit Regularization Explicit Regularization Effective Regularization Implicit Regularization Effect Value Based Deep Reinforcement Learning