Deep Residual Learning

Deep residual learning focuses on improving the training and performance of deep neural networks by incorporating "skip connections" that allow information to bypass certain layers, mitigating the vanishing gradient problem and enabling the training of significantly deeper architectures. Current research explores applications across diverse fields, including image classification (with variations like densely additive connections in spiking neural networks), financial modeling (precision matrix estimation), and medical image registration (Metamorphosis implementation). This technique's success stems from its ability to enhance model accuracy and efficiency in various complex tasks, leading to improvements in diverse areas such as gravitational wave modeling and medical image analysis.

Papers

November 9, 2023

Rethinking Residual Connection in Training Large-Scale Spiking Neural Networks
Yudong Li, Yunlin Lei, Xu Yang
Training Data Large Scale Spiking Neural Network Residual Connection Deep Residual Learning

September 9, 2022

Deep Learning Based Residuals in Non-linear Factor Models: Precision Matrix Estimation of Returns with Low Signal-to-Noise Ratio
Mehmet Caner, Maurizio Daniele
Deep Neural Network Deep Learning Framework Customer Service Consistent Estimator Precision Matrix Stochastic Ratio Deep Residual Learning

August 1, 2022

Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization
David Peer, Bart Keulen, Sebastian Stabinger, Justus Piater, Antonio Rodríguez-Sánchez
Deep Neural Network Deep Network Layer Wise Deep Residual Learning

March 16, 2022

Deep Residual Error and Bag-of-Tricks Learning for Gravitational Wave Surrogate Modeling
Styliani-Christina Fragkouli, Paraskevi Nousi, Nikolaos Passalis, Panagiotis Iosif, Nikolaos Stergioulas, Anastasios Tefas
Surrogate Model Gravitational Wave Bag of Word Binary Black Hole Merger Deep Residual Learning Surrogate Signal

February 1, 2022

A deep residual learning implementation of Metamorphosis
Matthis Maillard, Anton François, Joan Glaunès, Isabelle Bloch, Pietro Gori
Pathology Image Deep Residual Learning

Deep Residual Learning

Papers

Rethinking Residual Connection in Training Large-Scale Spiking Neural Networks

Deep Learning Based Residuals in Non-linear Factor Models: Precision Matrix Estimation of Returns with Low Signal-to-Noise Ratio

Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization

Deep Residual Error and Bag-of-Tricks Learning for Gravitational Wave Surrogate Modeling

A deep residual learning implementation of Metamorphosis