Generalization Error
Generalization error, the difference between a model's performance on training and unseen data, is a central challenge in machine learning. Current research focuses on understanding and mitigating this error across various model architectures, including linear models, neural networks (especially deep and overparameterized ones), and graph neural networks, often employing techniques like stochastic gradient descent, early stopping, and ensemble methods such as bagging. This research aims to develop tighter theoretical bounds on generalization error and improve model selection and assessment, particularly under conditions like data scarcity, distribution shifts, and adversarial attacks. Improved understanding of generalization error is crucial for building more reliable and robust machine learning systems across diverse applications.
Papers
Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures
Emmanuel Abbe, Samy Bengio, Elisabetta Cornacchia, Jon Kleinberg, Aryo Lotfi, Maithra Raghu, Chiyuan Zhang
Learning ReLU networks to high uniform accuracy is intractable
Julius Berner, Philipp Grohs, Felix Voigtlaender
On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions
Daniel Beaglehole, Mikhail Belkin, Parthe Pandit