Divisive Normalization

Divisive normalization (DN) is a computational process inspired by neuroscience, aiming to improve the robustness and efficiency of neural networks by normalizing neural activations relative to a local or global context. Current research focuses on integrating DN into various architectures, including recurrent neural networks (RNNs) and U-Nets for image segmentation, to enhance stability during training and improve generalization across diverse datasets and environmental conditions. This approach shows promise in addressing challenges like dataset bias in face recognition, improving the performance of image segmentation in varying lighting and weather conditions, and accelerating inference in transformer models, ultimately leading to more robust and efficient machine learning systems.