Generalization Capacity

Generalization capacity in machine learning refers to a model's ability to accurately predict outcomes on unseen data, a crucial aspect for reliable real-world applications. Current research focuses on understanding and improving generalization across diverse model architectures, including deep neural networks (DNNs), transformers, and graph neural networks, often employing techniques like regularization, multi-task learning, and data-centric approaches to enhance performance. This research is vital for improving the robustness and reliability of AI systems across various domains, from medical image analysis and earth observation to natural language processing and federated learning, where models must handle data heterogeneity and potential adversarial attacks. A key challenge remains developing accurate and practical metrics for evaluating generalization, particularly in out-of-distribution scenarios.

Papers

September 18, 2023

Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust?
Minsu Kim, Walid Saad
Strong Generalization Continual LEArning General Analysis Generalization Performance AI Agent Limited Memorization Robust Generalization Generalization Capacity Robust Continual Learning Unseen Target

August 9, 2023

JEDI: Joint Expert Distillation in a Semi-Supervised Multi-Dataset Student-Teacher Scenario for Video Action Recognition
Lucian Bicsi, Bogdan Alexe, Radu Tudor Ionescu, Marius Leordeanu
Semi Supervised Action Recognition Video Action Recognition Generalization Capacity Multi Dataset Supervised Training Map Distillation

June 19, 2023

Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models
Tomoki Sugimoto, Yasumasa Onoe, Hitomi Yanaka
Language Model Natural Language Inference Generalization Capacity Temporal Inference

June 9, 2023

Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often!
Milad Sefidgaran, Romain Chor, Abdellatif Zaidi, Yijun Wan
Strong Generalization Loss Function Generalization Error Critical Lesson Generalization Capacity

February 25, 2023

Explaining Generalization Power of a DNN Using Interactive Concepts
Huilin Zhou, Hao Zhang, Huiqi Deng, Dongrui Liu, Wen Shen, Shih-Han Chan, Quanshi Zhang
Deep Neural Network Strong Generalization Generalization Capacity Low Generalization

February 3, 2023

Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Models
Byounggyu Lew, Donghyun Son, Buru Chang
Pre Trained Model Gradient Estimation Large Scale Pre Trained Model Generalization Capacity

December 19, 2022

VC dimensions of group convolutional neural networks
Philipp Christian Petersen, Anna Sepliarskaia
Convolutional Neural Network Group Setting Convolutional Kernel Generalization Capacity VC Dimension Model Family

November 10, 2022

October 17, 2022

Test-Time Training for Graph Neural Networks
Yiqi Wang, Chaozhuo Li, Wei Jin, Rui Li, Jianan Zhao, Jiliang Tang, Xing Xie
Graph Neural Network Graph Classification GNN Model Test Time Test Time Training Generalization Capacity

July 5, 2022

June 4, 2022

On the Generalization Power of the Overfitted Three-Layer Neural Tangent Kernel Model
Peizhong Ju, Xiaojun Lin, Ness B. Shroff
Generalization Performance Neural Tangent Kernel Model Overfitting Hidden Layer Generalization Capacity Layer NTK

March 23, 2022

Predicting the generalization gap in neural networks using topological data analysis
Rubén Ballester, Xavier Arnal Clemente, Carles Casacuberta, Meysam Madadi, Ciprian A. Corneanu, Sergio Escalera
Neural Network Topological Data Analysis Generalization Gap Generalization Capacity

March 22, 2022

Improving Generalization in Federated Learning by Seeking Flat Minima
Debora Caldarola, Barbara Caputo, Marco Ciccone
Strong Generalization Sharpness Aware Minimization Generalization Gap Flat Minimum Generalization Capacity

February 10, 2022

Monotone Learning
Olivier Bousquet, Amit Daniely, Haim Kaplan, Yishay Mansour, Shay Moran, Uri Stemmer
Generalization Capacity PAC Learning Monotone Function Monotone Neural Network

January 23, 2022

A Generalized Weighted Optimization Method for Computational Learning and Inversion
Björn Engquist, Kui Ren, Yunan Yang
Generalization Error Kernel Regression Generalization Capacity Eta Inversion Computational Learning

December 9, 2021

Effective dimension of machine learning models
Amira Abbas, David Sutter, Alessio Figalli, Stefan Woerner
Machine Learning Machine Learning Model Generalization Error Generalization Capacity Effective Dimension

November 17, 2021

Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data
Rui Hu, Jitao Sang, Jinqiang Wang, Rui Hu, Chaoquan Jiang
Deep Learning Model Deep Network Human Understanding Distribution Data Generalization Capacity Class Out of Distribution

Generalization Capacity

Papers

Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust?

JEDI: Joint Expert Distillation in a Semi-Supervised Multi-Dataset Student-Teacher Scenario for Video Action Recognition

Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models

Lessons from Generalization Error Analysis of Federated Learning: You May Communicate Less Often!

Explaining Generalization Power of a DNN Using Interactive Concepts

Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Models

VC dimensions of group convolutional neural networks

Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities

Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization

Test-Time Training for Graph Neural Networks

Predicting Out-of-Domain Generalization with Neighborhood Invariance

On A Mallows-type Model For (Ranked) Choices

On the Generalization Power of the Overfitted Three-Layer Neural Tangent Kernel Model

Predicting the generalization gap in neural networks using topological data analysis

Improving Generalization in Federated Learning by Seeking Flat Minima

Monotone Learning

A Generalized Weighted Optimization Method for Computational Learning and Inversion

Effective dimension of machine learning models

Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data