Generalization Property

Generalization, a machine learning model's ability to perform well on unseen data, is a central research focus, aiming to understand why and how models generalize beyond their training data. Current research investigates this through various lenses, including analyzing the impact of training schedules, loss landscape sharpness (e.g., using SAM), and model architectures like ResNets and GFlowNets, as well as exploring the role of data variability and the effects of quantization. Improved understanding of generalization properties is crucial for building more reliable and robust machine learning systems across diverse applications, from scientific computing to medical diagnosis.

Papers

February 17, 2022

Limitations of Neural Collapse for Understanding Generalization in Deep Learning
Like Hui, Mikhail Belkin, Preetum Nakkiran
Deep Learning Strong Generalization Fundamental Limitation Neural Collapse Generalization Property

February 4, 2022

Improved Information Theoretic Generalization Bounds for Distributed and Federated Learning
L. P. Barnes, Alex Dytso, H. V. Poor
Mutual Information Generalization Error Information Theoretic Generalization Property Clean Distribution Information Theoretic Generalization Bound

January 30, 2022

Generalizing similarity in noisy setups: the DIBS phenomenon
Nayara Fonseca, Veronica Guidetti
Contrastive Learning Label Noise High Similarity Siamese Network Generalization Property Noisy Environment Similarity Learning Offline Training

December 7, 2021

Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural Networks
Pascal Mattia Esser, Leena Chennuru Vankadara, Debarghya Ghoshdastidar
Graph Neural Network Strong Generalization Graph Convolution Generalization Property Rademacher Complexity Transductive Inference Learning Theory

November 29, 2021

Self-Training of Halfspaces with Generalization Guarantees under Massart Mislabeling Noise Model
Lies Hadjadj, Massih-Reza Amini, Sana Louhichi, Alexis Deschamps
Semi Supervised Label Noise Self Training Generalization Property Generalization Guarantee Margin Halfspaces Pseudo Labeled Sample

August 11, 2021

Approximation and generalization properties of the random projection classification method
Mireille Boutin, Evzenie Coupkova
Average Approximation Learning Algorithm Generalization Gap Generalization Property Linear Classifier Random Projection

Generalization Property

Papers

Limitations of Neural Collapse for Understanding Generalization in Deep Learning

Improved Information Theoretic Generalization Bounds for Distributed and Federated Learning

Generalizing similarity in noisy setups: the DIBS phenomenon

Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural Networks

Self-Training of Halfspaces with Generalization Guarantees under Massart Mislabeling Noise Model

Approximation and generalization properties of the random projection classification method