Generalization Property

Generalization, a machine learning model's ability to perform well on unseen data, is a central research focus, aiming to understand why and how models generalize beyond their training data. Current research investigates this through various lenses, including analyzing the impact of training schedules, loss landscape sharpness (e.g., using SAM), and model architectures like ResNets and GFlowNets, as well as exploring the role of data variability and the effects of quantization. Improved understanding of generalization properties is crucial for building more reliable and robust machine learning systems across diverse applications, from scientific computing to medical diagnosis.

Papers

February 5, 2024

Generalization Properties of Adversarial Training for $\ell_0$-Bounded Adversarial Attacks
Payam Delgosha, Hamed Hassani, Ramtin Pedarsani
Adversarial Attack Adversarial Training Adversarial Perturbation Generalization Property Hypothesis Class

February 1, 2024

Multi-group Learning for Hierarchical Groups
Samuel Deng, Daniel Hsu
Generalization Property Group Structure Multi Group Learning

January 22, 2024

Generalization and Informativeness of Conformal Prediction
Matteo Zecchin, Sangwoo Park, Osvaldo Simeone, Fredrik Hellström
Strong Generalization Conformal Prediction Generalization Property Prediction Set Conformal Prediction Set Base Editor

December 29, 2023

Generalization properties of contrastive world models
Kandan Ramakrishnan, R. James Cotton, Xaq Pitkow, Andreas S. Tolias
Object Centric Representation Object Representation Generalization Property Human Like Generalization

December 21, 2023

Learned reconstruction methods for inverse problems: sample error estimates
Luca Ratti
Inverse Problem Regularization Model Statistical Learning Generalization Property Reconstruction Technique

December 14, 2023

Less is more -- the Dispatcher/ Executor principle for multi-task Reinforcement Learning
Martin Riedmiller, Tim Hertweck, Roland Hafner
Strong Generalization Generalization Property Design Principle Multi Task Reinforcement Learning Near Optimal Executor

November 3, 2023

On the Generalization Properties of Diffusion Models
Puheng Li, Zhong Li, Huishuai Zhang, Jiang Bian
Diffusion Model Generative Model Model Generalization Score Based Diffusion Model Generalization Gap Generalization Property

October 26, 2023

Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates
Miao Lu, Beining Wu, Xiaodong Yang, Difan Zou
Stochastic Gradient Descent Generalization Property Large Learning Rate Non Converging Artificial Oscillation

October 23, 2023

Understanding the Inner Workings of Language Models Through Representation Dissimilarity
Davis Brown, Charles Godfrey, Nicholas Konz, Jonathan Tu, Henry Kvinge
Language Model Language Modeling Generalization Property Dissimilarity Measure

September 14, 2023

How many Neurons do we need? A refined Analysis for Shallow Networks trained with Gradient Descent
Mike Nguyen, Nicole Mücke
Gradient Descent Neural Tangent Kernel Kernel Hilbert Space Two Layer Neural Network Generalization Property Shallow Network Improved Analysis Many Neuron

August 7, 2023

Studying Large Language Model Generalization with Influence Functions
Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilė Lukošiūtė, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, Samuel R. Bowman
Generalization Property Influence Function Human Like Generalization Influence Analysis Language Model Generalization

July 31, 2023

A theory of data variability in Neural Network Bayesian inference
Javed Lindner, David Dahmen, Michael Krämer, Moritz Helias
Theoretical Understanding Kernel Method Generalization Property Kernel Matrix Non Pathological Variability Neural Network Gaussian Process Bayesian Neural

July 18, 2023

Batched Predictors Generalize within Distribution
Andreas Loukas, Pan Kessel
Product Distribution Generalization Property Generalization Guarantee Easy to Hard Generalization

June 30, 2023

June 26, 2023

Balanced Filtering via Disclosure-Controlled Proxies
Siqi Deng, Emily Diana, Michael Kearns, Aaron Roth
Generalization Property Optimization Proxy Proxy Dataset

May 30, 2023

Embedding Inequalities for Barron-type Spaces
Lei Wu
Generalization Property Societal Inequality Barron Space

May 9, 2023

Towards Understanding Generalization of Macro-AUC in Multi-label Learning
Guoqiang Wu, Chongxuan Li, Yilong Yin
Strong Generalization Multi Label Generalization Property AUC Metric Label Imbalance

February 20, 2023

Stability-based Generalization Analysis for Mixtures of Pointwise and Pairwise Learning
Jiahuan Wang, Jun Chen, Hong Chen, Bin Gu, Weifu Li, Xin Tang
Stochastic Gradient Descent Mixture Component Generalization Property Pairwise Learning Point Wise Operation Pairwise Loss

February 1, 2023

Gradient Descent in Neural Networks as Sequential Learning in RKBS
Alistair Shilton, Sunil Gupta, Santu Rana, Svetha Venkatesh
Neural Network Gradient Descent Neural Tangent Kernel Kernel Hilbert Space Sequential Learning Generalization Property Taylor Expansion Weak Convergence Kernel Banach Space