Neural Network Generalization

Neural network generalization focuses on understanding why and how deep learning models trained on a limited dataset can accurately predict outcomes on unseen data. Current research investigates factors influencing generalization, such as model architecture (including modular networks and transformers), training optimization techniques (like sharpness-aware minimization and data augmentation strategies), and the role of biases and noise in both data and model parameters. These investigations are crucial for improving the reliability and robustness of AI systems across diverse applications, ranging from medical image analysis to robotics and cybersecurity, where generalization to real-world scenarios is paramount.

Papers

May 25, 2023

PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusion
Yige Yuan, Bingbing Xu, Bo Lin, Liang Hou, Fei Sun, Huawei Shen, Xueqi Cheng
Strong Generalization Partial Differential Equation Neural Network Generalization Adaptive Diffusion

May 19, 2023

Deep Neural Networks Generalization and Fine-Tuning for 12-lead ECG Classification
Aram Avetisyan, Shahane Tigranyan, Ariana Asatryan, Olga Mashkova, Sergey Skorik, Vladislav Ananev, Yury Markin
Deep Learning Deep Neural Network Fine Tuning Neural Network Generalization Electrocardiogram Dataset

February 23, 2023

On Statistical Properties of Sharpness-Aware Minimization: Provable Guarantees
Kayhan Behdin, Rahul Mazumder
Deep Neural Network Gradient Descent Sharpness Aware Minimization Neural Network Generalization Provable Guarantee Statistical Property

February 14, 2023

Do Neural Networks Generalize from Self-Averaging Sub-classifiers in the Same Way As Adaptive Boosting?
Michael Sun, Peter Chatain
Neural Network Strong Generalization Neural Network Generalization Generalization Technique Adaptive Boosting Meta Classifier

December 7, 2022

Improved Deep Neural Network Generalization Using m-Sharpness-Aware Minimization
Kayhan Behdin, Qingquan Song, Aman Gupta, David Durfee, Ayan Acharya, Sathiya Keerthi, Rahul Mazumder
Adversarial Perturbation Generalization Performance Sharpness Aware Minimization Neural Network Generalization Modern Deep Learning Model

December 1, 2022

Task Discovery: Finding the Tasks that Neural Networks Generalize on
Andrei Atanov, Andrei Filatov, Teresa Yeo, Ajay Sohmshetty, Amir Zamir
Deep Learning Model Inductive Bias New Task Neural Network Generalization High Probability Generalization

November 18, 2022

SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness
Gonçalo Mordido, Sébastien Henwood, Sarath Chandar, François Leduc-Primeau
Native Robustness Sharpness Aware Minimization Neural Network Generalization Deep Neural Network Performance Sharpness Aware Training

November 17, 2022

Why Deep Learning Generalizes
Benjamin L. Badger
Deep Learning Strong Generalization Gradient Descent Neural Network Generalization Large Deep Learning Model

October 21, 2022

A New Perspective for Understanding Generalization Gap of Deep Neural Networks Trained with Large Batch Sizes
Oyebade K. Oyedotun, Konstantinos Papadopoulos, Djamila Aouada
Deep Neural Network New Perspective Generalization Gap Neural Network Generalization Mini Batch Gradient Descent

October 18, 2022

Generalizing in the Real World with Representation Learning
Tegan Maharaj
Machine Learning Artificial Intelligence Strong Generalization Representation Learning Gradient Descent Real World Deep Network Neural Network Generalization

August 10, 2022

Semantic Self-adaptation: Enhancing Generalization with a Single Sample
Sherwin Bahmani, Oliver Hahn, Eduard Zamfir, Nikita Araslanov, Daniel Cremers, Stefan Roth
Semantic Segmentation Strong Generalization Convolutional Layer Neural Network Generalization Self Adaptation Single Sample

July 5, 2022

Neural Networks and the Chomsky Hierarchy
Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, Li Kevin Wenliang, Elliot Catt, Chris Cundy, Marcus Hutter, Shane Legg, Joel Veness, Pedro A. Ortega
Neural Network Traditional RNNs Neural Network Generalization Easy to Hard Generalization

March 21, 2022

The activity-weight duality in feed forward neural networks: The geometric determinants of generalization
Yu Feng, Yuhai Tu
Neural Network Strong Generalization Generalization Performance Neural Network Generalization Feed Ranking Log Determinant Point Wise Weight

March 19, 2022

Deep Learning Generalization, Extrapolation, and Over-parameterization
Roozbeh Yousefzadeh
Neural Network Strong Generalization Deep Network Adversarial Vulnerability Extrapolation Framework Neural Network Generalization Benign Over Parameterization Imperceptible Adversarial Perturbation

February 3, 2022

Mapping DNN Embedding Manifolds for Network Generalization Prediction
Molly O'Brien, Julia Bukowski, Mathias Unberath, Aria Pezeshk, Greg Hager
Deep Neural Network DNN Framework Class Manifold Neural Network Generalization Deep Neural Network Performance

February 1, 2022

Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization
Pedro R. A. S. Bassi, Sergio S. J. Dertkigil, Andrea Cavalli
Native Robustness Shortcut Learning Neural Network Generalization Layer Wise Relevance Propagation Background Bias

January 6, 2022

Diversity-boosted Generalization-Specialization Balancing for Zero-shot Learning
Yun Li, Zhe Liu, Xiaojun Chang, Julian McAuley, Lina Yao
Strong Generalization Meta Learning Zero Shot Learning Feature Diversity Neural Network Generalization