Generalization Property

Generalization, a machine learning model's ability to perform well on unseen data, is a central research focus, aiming to understand why and how models generalize beyond their training data. Current research investigates this through various lenses, including analyzing the impact of training schedules, loss landscape sharpness (e.g., using SAM), and model architectures like ResNets and GFlowNets, as well as exploring the role of data variability and the effects of quantization. Improved understanding of generalization properties is crucial for building more reliable and robust machine learning systems across diverse applications, from scientific computing to medical diagnosis.

Papers

October 18, 2022

Generalization Properties of Decision Trees on Real-valued and Categorical Features
Jean-Samuel Leboeuf, Frédéric LeBlanc, Mario Marchand
Decision Tree Generalization Property Real Data Categorical Feature

October 6, 2022

Generalization Properties of Retrieval-based Models
Soumya Basu, Ankit Singh Rawat, Manzil Zaheer
Generalization Property Local Learning Retrieval Method Retrieval Based Model

September 30, 2022

Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery
Yifei Wang, Yixuan Hua, Emmanuel Candés, Mert Pilanci
Compressed Sensing ReLU Network Generalization Property Sparse Model Two Layer ReLU Simple Model Exact Community Recovery

September 15, 2022

August 29, 2022

Generalization In Multi-Objective Machine Learning
Peter Súkeník, Christoph H. Lampert
Strong Generalization Multi Objective Generalization Bound Pareto Optimal Generalization Property Multi Objective Learning

August 27, 2022

Information FOMO: The unhealthy fear of missing out on information. A method for removing misleading data for healthier models
Ethan Pickering, Themistoklis P. Sapsis
Machine Learning Machine Learning Model Full Information Generalization Property FEAR Speech

August 8, 2022

Generalization and Overfitting in Matrix Product State Machine Learning Architectures
Artem Strashko, E. Miles Stoudenmire
Strong Generalization Model Overfitting Generalization Property Matrix Product State

June 18, 2022

On the Role of Generalization in Transferability of Adversarial Examples
Yilin Wang, Farzan Farnia
Adversarial Attack Strong Generalization Adversarial Example Integral Role Task Transferability Generalization Property

June 16, 2022

Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence
Margalit Glasgow, Colin Wei, Mary Wootters, Tengyu Ma
Strong Generalization Generalization Bound Generalization Property Margin Maximization Margin Classifier Margin Based Uniform Convergence

June 13, 2022

Towards Understanding Sharpness-Aware Minimization
Maksym Andriushchenko, Nicolas Flammarion
Sharpness Aware Minimization Generalization Property Improved Generalization Standard Gradient Descent

June 10, 2022

Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias
Navid Ardeshir, Daniel Hsu, Clayton Sanford
Inductive Bias Two Layer Neural Network Generalization Property P$ Norm Optimal Generalization Intrinsic Dimensionality

June 9, 2022

On Margins and Generalisation for Voting Classifiers
Felix Biggs, Valentina Zantedeschi, Benjamin Guedj
Strong Generalization Generalization Property PAC Bayesian Margin Maximization Margin Based Voting Classifier

June 6, 2022

Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees
Haotian Ju, Dongyue Li, Hongyang R. Zhang
Deep Neural Network Generalization Bound Fine Tuned Model Generalization Gap Generalization Property Generalization Guarantee Robust Fine Tuning

June 4, 2022

Guided Deep Metric Learning
Jorge Gonzalez-Zapata, Ivan Reyes-Amezcua, Daniel Flores-Araiza, Mauricio Mendez-Ruiz, Gilberto Ochoa-Ruiz, Andres Mendez-Vazquez
Manifold Learning Deep Metric Learning Generalization Property Similarity Learning

June 2, 2022

April 13, 2022

Local Feature Swapping for Generalization in Reinforcement Learning
David Bertoin, Emmanuel Rachelson
Reinforcement Learning Strong Generalization Neural Architecture Generalization Property Visual Input Feature Swapping

February 27, 2022

Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond
Matan Schliserman, Tomer Koren
Gradient Descent Core Stability Generalization Bound Implicit Bias Gradient Method Generalization Property Gradient Free Stochastic Convex Optimization Separable Data

Generalization Property

Papers

Generalization Properties of Decision Trees on Real-valued and Categorical Features

Generalization Properties of Retrieval-based Models

Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery

Generalization Properties of NAS under Activation and Skip Connection Search

On Generalization of Decentralized Learning with Separable Data

Generalization In Multi-Objective Machine Learning

Information FOMO: The unhealthy fear of missing out on information. A method for removing misleading data for healthier models

Generalization and Overfitting in Matrix Product State Machine Learning Architectures

On the Role of Generalization in Transferability of Adversarial Examples

Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence

Towards Understanding Sharpness-Aware Minimization

Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias

On Margins and Generalisation for Voting Classifiers

Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees

Guided Deep Metric Learning

Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares

Score-Based Generative Models Detect Manifolds

Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules

Local Feature Swapping for Generalization in Reinforcement Learning

Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond