Generalization Error Bound

Generalization error bounds aim to quantify how well a machine learning model trained on a finite dataset will perform on unseen data. Current research focuses on deriving tighter bounds for various model architectures, including neural networks (especially deep networks with weight decay and specific activation functions), and algorithms like stochastic gradient descent and federated learning, often leveraging techniques from information theory and optimal transport. These bounds are crucial for understanding model robustness and reliability, informing algorithm design, and providing theoretical guarantees for applications ranging from natural language processing to quantum machine learning. Improved bounds contribute to more efficient and trustworthy machine learning systems.

Papers

November 17, 2024

Infinite Width Limits of Self Supervised Neural Networks
Maximilian Fleissner, Gautham Govind Anil, Debarghya Ghoshdastidar
Self Supervised Wide Neural Network Kernel Regression Generalization Error Bound Infinite Depth and Width Limit

October 26, 2024

Robust Model Evaluation over Large-scale Federated Networks
Amir Najafi, Samin Mahdizadeh Sani, Farzan Farnia
Native Robustness Network Programming Generalization Error Bound Robust Evaluation Unseen Target

October 15, 2024

On Rank-Dependent Generalisation Error Bounds for Transformers
Lan V. Truong
Transformer Megatron Decepticons Generalization Bound Generalization Error Efficient Transformer Low Rank Matrix Generalization Error Bound

October 9, 2024

A Generalization Bound for a Family of Implicit Networks
Samy Wu Fung, Benjamin Berkels
Neural Network Strong Generalization Generalization Error Bound Single Parent Family

October 3, 2024

September 11, 2024

July 9, 2024

A Generalization Bound for Nearly-Linear Networks
Eugene Golikov
Neural Network Generalization Bound Generalization Error Bound Linear Network Non Vacuous Generalization Bound

June 27, 2024

Estimating Long-term Heterogeneous Dose-response Curve: Generalization Bound Leveraging Optimal Transport Weights
Zeqin Yang, Weilin Chen, Ruichu Cai, Yuguang Yan, Zhifeng Hao, Zhipeng Yu, Zhichao Zou, Zhen Peng, Jiecheng Guo
Long Term Causal Effect Estimation Unobserved Confounders Generalization Error Bound Dose Response Heterogeneous Effect

June 2, 2024

Generalization Bound and New Algorithm for Clean-Label Backdoor Attack
Lijia Yu, Shuang Liu, Yibo Miao, Xiao-Shan Gao, Lijun Zhang
Strong Generalization Backdoor Attack Adversarial Learning Novel Algorithm Generalization Error Bound Clean Label Backdoor Attack

May 30, 2024

Domain Adaptation with Cauchy-Schwarz Divergence
Wenzhe Yin, Shujian Yu, Yicong Lin, Jie Liu, Jan-Jakob Sonke, Efstratios Gavves
Domain Adaptation Unsupervised Domain Adaptation Inverse Divergence Generalization Error Bound Cauchy Schwarz Divergence

April 26, 2024

Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets
Benjamin Dupuis, Paul Viallard, George Deligiannidis, Umut Simsekli
Generalization Bound PAC Bayesian Upper Bound Generalization Error Bound Hypothesis Space Random Set

April 17, 2024

Improved Generalization Bounds for Communication Efficient Federated Learning
Peyman Gholami, Hulya Seferoglu
Strong Generalization Federated Learning Generalization Bound Generalization Error Bound Communication Efficient Federated Learning

April 14, 2024

Generalization Error Bounds for Learning under Censored Feedback
Yifan Yang, Ali Payani, Parinaz Naghizadeh
Generalization Error Error Bound Generalization Error Bound Generalization Guarantee

February 8, 2024

Tighter Generalization Bounds on Digital Computers via Discrete Optimal Transport
Anastasis Kratsios, A. Martina Neuman, Gudmund Pammer
Generalization Gap Data Discretization Generalization Error Bound Digital Computer Discrete Optimal Transport

November 25, 2023

A unified framework for learning with nonlinear model classes from arbitrary linear samples
Ben Adcock, Juan M. Cardenas, Nick Dexter
Generative Model Unified Framework Nonlinear Model Generalization Error Bound Hilbert Space Learning Guarantee

November 10, 2023

Aggregation Weighting of Federated Learning via Generalization Bound Estimation
Mingwei Xu, Xiaofeng Cao, Ivor W. Tsang, James T. Kwok
Generalization Performance Generalization Bound Generalization Error Bound Distributional Robustness Weight Aggregation

November 2, 2023

Time-Independent Information-Theoretic Generalization Bounds for SGLD
Futoshi Futami, Masahiro Fujisawa
Langevin Dynamic Kullback Leibler Divergence Generalization Error Bound Information Theoretic Generalization Bound Information Theoretic Generalization Time Invariant

October 18, 2023

A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm
Haizhou Shi, Hao Wang
Practical Algorithm Unified Framework Theoretical Understanding Incremental Learning Memory Trace Generalization Error Bound Domain Incremental Learning