Dropout Regularization

Dropout regularization is a technique used to prevent overfitting in neural networks by randomly ignoring neurons during training, forcing the network to learn more robust and generalizable features. Current research focuses on understanding the theoretical properties of dropout, particularly its impact on model convergence and the resulting distributions, and on optimizing its application within various architectures, including Bayesian neural networks and convolutional neural networks, as well as exploring its use in conjunction with other regularization methods like prompt tuning and simultaneous learning. This research is significant because it improves the reliability and efficiency of neural networks across diverse applications, from image classification and natural language processing to resource-constrained hardware implementations.

Papers

September 11, 2024

Asymptotics of Stochastic Gradient Descent with Dropout Regularization in Linear Models
Jiaqi Li, Johannes Schmidt-Hieber, Wei Biao Wu
Stochastic Gradient Descent Covariance Matrix Linear Model Asymptotic Behavior Online Inference Dropout Regularization

August 21, 2024

Distributional Properties of Subword Regularization
Marco Cognetta, Vilém Zouhar, Naoaki Okazaki
Training Corpus Subword Tokenization Distributional Property Dropout Regularization Subword Regularization

June 23, 2024

Hardware-Aware Neural Dropout Search for Reliable Uncertainty Prediction on FPGA
Zehuan Zhang, Hongxiang Fan, Hao Mark Chen, Lukasz Dudziak, Wayne Luk
Field Programmable Gate Array Uncertainty Prediction Dropout Regularization Inference Based Adaptive Dropout Dropout Based

October 30, 2023

Lyapunov-Based Dropout Deep Neural Network (Lb-DDNN) Controller
Saiedeh Akbari, Emily J. Griffis, Omkar Sudhir Patil, Warren E. Dixon
Structured Dropout M$ Base Controller Adaptive Controller Dropout Regularization

July 27, 2023

Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models
Kecheng Zheng, Wei Wu, Ruili Feng, Kai Zhu, Jiawei Liu, Deli Zhao, Zheng-Jun Zha, Wei Chen, Yujun Shen
Fine Tuning Pre Trained Vision Language Model Parameter Efficient Tuning Hidden Knowledge Binary Mask Dropout Regularization

July 13, 2023

On the ability of CNNs to extract color invariant intensity based features for image classification
Pradyumna Elavarthi, James Lee, Anca Ralescu
Convolutional Neural Network Image Classification Feature Wise Color Object Image Classification Task Intensity Feature Intensity Function Dropout Regularization

May 22, 2023

Regularization Through Simultaneous Learning: A Case Study on Plant Classification
Pedro Henrique Nascimento Castro, Gabriel Cássia Fortuna, Rafael Alves Bonfim de Queiroz, Gladston Juliano Prates Moreira, Eduardo José da Silva Luz
Transfer Learning Case Study Multi Task Learning Feature Selection Plant Identification Simultaneous Learning Dropout Regularization

May 11, 2023

Dropout Regularization in Extended Generalized Linear Models based on Double Exponential Families
Benedikt Lütke Schwienhorst, Lucas Kock, David J. Nott, Nadja Klein
Regularization Model Generalized Linear Model Regularization Method Exponential Family Dropout Regularization

March 9, 2023

Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts
Rohit Agarwal, Deepak Gupta, Alexander Horsch, Dilip K. Prasad
Online Learning Ambiguous Input Dropout Regularization

September 30, 2022

An In-depth Study of Stochastic Backpropagation
Jun Fang, Mingze Xu, Hao Chen, Bing Shuai, Zhuowen Tu, Joseph Tighe
Deep Neural Network Stochastic Gradient Comprehensive Study Backward Propagation Stochastic Neural Network Dropout Regularization

Dropout Regularization

Papers

Asymptotics of Stochastic Gradient Descent with Dropout Regularization in Linear Models

Distributional Properties of Subword Regularization

Hardware-Aware Neural Dropout Search for Reliable Uncertainty Prediction on FPGA

Lyapunov-Based Dropout Deep Neural Network (Lb-DDNN) Controller

Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models

On the ability of CNNs to extract color invariant intensity based features for image classification

Regularization Through Simultaneous Learning: A Case Study on Plant Classification

Dropout Regularization in Extended Generalized Linear Models based on Double Exponential Families

Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts

An In-depth Study of Stochastic Backpropagation