Strong Generalization

Strong generalization, the ability of machine learning models to perform well on unseen data, is a central objective in current research. Active areas of investigation include improving the robustness of self-supervised learning, understanding the optimization dynamics of transformers and other architectures (including CNNs and RNNs), and developing methods to enhance generalization through data augmentation, regularization techniques (e.g., logical regularization, consistency regularization), and improved training strategies (e.g., few-shot learning, meta-learning). These advancements are crucial for building reliable and adaptable AI systems across diverse applications, from image classification and natural language processing to healthcare and robotics.

Papers

February 24, 2024

Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models
Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, Ge Li
Strong Generalization Limited Memorization Data Contamination Contamination Detection Inaccurate Contamination Ratio

February 23, 2024

Deep Networks Always Grok and Here is Why
Ahmed Imtiaz Humayun, Randall Balestriero, Richard Baraniuk
Deep Neural Network Strong Generalization Adversarial Example Deep Network Grokking Phenomenon Delay Resilient

February 22, 2024

A Self-supervised Pressure Map human keypoint Detection Approch: Optimizing Generalization and Computational Efficiency Across Datasets
Chengzhang Yu, Xianjun Yang, Wenxia Bao, Shaonan Wang, Zhiming Yao
Data Set Strong Generalization Keypoint Detection Computational Efficiency Pressure Map

February 20, 2024

Spurious Correlations in Machine Learning: A Survey
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma, Aidong Zhang
Machine Learning Timely Survey Strong Generalization Machine Learning Model Spurious Correlation Total Correlation

February 18, 2024

Graph Out-of-Distribution Generalization via Causal Intervention
Qitian Wu, Fan Nie, Chenxiao Yang, Tianyi Bao, Junchi Yan
Strong Generalization Causal Inference Causal Intervention Graph Out of Distribution Generalization Ego Graph

February 17, 2024

Beyond Generalization: A Survey of Out-Of-Distribution Adaptation on Graphs
Shuhan Liu, Kaize Ding
Timely Survey Strong Generalization Graph Drawing Graph Out of Distribution Graph Out of Distribution Adaptation

February 16, 2024

Instruction Diversity Drives Generalization To Unseen Tasks
Dylan Zhang, Justin Wang, Francois Charton
Strong Generalization Pre Trained Language Model Instruction Tuning Model Generalization Unseen Task Diverse Instruction

February 15, 2024

February 14, 2024

February 13, 2024

Strong Generalization

Papers

Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models

Deep Networks Always Grok and Here is Why

A Self-supervised Pressure Map human keypoint Detection Approch: Optimizing Generalization and Computational Efficiency Across Datasets

TOOLVERIFIER: Generalization to New Tools via Self-Verification

Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains

Zero-shot generalization across architectures for visual classification

Spurious Correlations in Machine Learning: A Survey

Revisiting Data Augmentation in Deep Reinforcement Learning

Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures

The effect of Leaky ReLUs on the training and generalization of overparameterized networks

Graph Out-of-Distribution Generalization via Causal Intervention

Beyond Generalization: A Survey of Out-Of-Distribution Adaptation on Graphs

Instruction Diversity Drives Generalization To Unseen Tasks

$f$-MICL: Understanding and Generalizing InfoNCE-based Contrastive Learning

Any-Shift Prompting for Generalization over Distributions

Quantified Task Misalignment to Inform PEFT: An Exploration of Domain Generalization and Catastrophic Forgetting in CLIP

Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization

Generalization in Healthcare AI: Evaluation of a Clinical Large Language Model

Improving Generalization in Semantic Parsing by Increasing Natural Language Variation

A PAC-Bayesian Link Between Generalisation and Flat Minima