Generalization Benefit

Generalization benefit in machine learning focuses on improving a model's ability to perform well on unseen data, a crucial aspect for real-world applications. Current research investigates various techniques to enhance generalization, including optimizing training procedures (e.g., adjusting learning rates, employing sharpness-aware minimization), leveraging multi-task learning and contrastive methods, and designing models with inherent invariances to data transformations. These efforts aim to understand the underlying mechanisms driving generalization and develop more robust and reliable machine learning models across diverse tasks and datasets, ultimately impacting the reliability and applicability of AI systems.

Papers

October 30, 2024

Why Fine-grained Labels in Pretraining Benefit Generalization?
Guan Zhe Hong, Yin Cui, Ariel Fuxman, Stanely Chan, Enming Luo
Neural Network Deep Neural Network Fine Grained Label Information Better Generalization Generalization Benefit Coarse Label

January 21, 2024

Understanding the Generalization Benefits of Late Learning Rate Decay
Yinuo Ren, Chao Ma, Lexing Ying
Learning Rate Better Generalization Large Learning Rate Optimal Generalization Generalization Benefit

August 25, 2023

Attending Generalizability in Course of Deep Fake Detection by Exploring Multi-task Learning
Pranav Balaji, Abhijit Das, Srijan Das, Antitza Dantcheva
Multi Task Learning Deep Fake Stronger Generalizability Contrastive Method Course Specific Context Deep Fake Detection Generalization Benefit

August 16, 2023

Benchmarking Neural Network Generalization for Grammar Induction
Nur Lan, Emmanuel Chemla, Roni Katzir
Neural Network Neural Network Generalization Grammar Induction Generalization Benefit Generalization Benchmark

May 19, 2023

Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape
Yan Sun, Li Shen, Shixiang Chen, Liang Ding, Dacheng Tao
Optimization Purpose Sharpness Aware Minimization Local Optimum Global Consistency Generalization Benefit Return Landscape

January 30, 2023

ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning
Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Liu Dapeng, Jie Jiang, Mingsheng Long
Auxiliary Task Negative Transfer Human Like Generalization Task Gradient Generalization Benefit

August 3, 2022

Improving Meta-Learning Generalization with Activation-Based Early-Stopping
Simon Guiroy, Christopher Pal, Gonçalo Mordido, Sarath Chandar
Shot Learning Meta Learning Meta Training Generalization Benefit

June 21, 2022

On the Maximum Hessian Eigenvalue and Generalization
Simran Kaur, Jeremy Cohen, Zachary C. Lipton
Strong Generalization Stochastic Gradient Descent Hessian Matrix Batch Normalization Generalization Benefit

June 14, 2022

Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction
Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora
Batch Normalization Normalization Dictionary Layer Normalization Normalization Layer Generalization Benefit Sharpness Reduction

March 19, 2022

The Sandbox Environment for Generalizable Agent Research (SEGAR)
R Devon Hjelm, Bogdan Mazoure, Florian Golemo, Samira Ebrahimi Kahou, Pedro Braga, Felipe Frujeri, Mihai Jalobeanu, Andrey Kolobov
Generalist Agent Sequential Decision Making Task Generalization Benefit Sandbox Environment

November 10, 2021

Understanding the Generalization Benefit of Model Invariance from a Data Perspective
Sicheng Zhu, Bang An, Furong Huang
Strong Generalization Better Generalization Data Transformation Data Perspective Model Invariance Generalization Benefit