Strong Generalization

Strong generalization, the ability of machine learning models to perform well on unseen data, is a central objective in current research. Active areas of investigation include improving the robustness of self-supervised learning, understanding the optimization dynamics of transformers and other architectures (including CNNs and RNNs), and developing methods to enhance generalization through data augmentation, regularization techniques (e.g., logical regularization, consistency regularization), and improved training strategies (e.g., few-shot learning, meta-learning). These advancements are crucial for building reliable and adaptable AI systems across diverse applications, from image classification and natural language processing to healthcare and robotics.

Papers

October 18, 2023

Learning under Label Proportions for Text Classification
Jatin Chauhan, Xiaoxuan Wang, Wei Wang
LeArning Abstract Strong Generalization Text Classification NLP Research Self Supervised Objective Label Proportion

October 16, 2023

CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization
Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, Peter Clark
Reinforcement Learning Strong Generalization Language Agent Episodic Memory Task Adaptation

October 15, 2023

KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Models
Yuyang Bai, Shangbin Feng, Vidhisha Balachandran, Zhaoxuan Tan, Shiqi Lou, Tianxing He, Yulia Tsvetkov
Large Language Model Strong Generalization Knowledge Intensive Task Open Ended Generation Encoded Knowledge Human Aligned Benchmark Knowledge Generalization

October 14, 2023

Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations
Alexa R. Tartaglini, Sheridan Feucht, Michael A. Lepori, Wai Keen Vong, Charles Lovering, Brenden M. Lake, Ellie Pavlick
Convolutional Neural Network Deep Neural Network Strong Generalization Distribution Generalization Visual Relation

October 13, 2023

October 11, 2023

October 10, 2023

October 9, 2023

October 6, 2023

Understanding prompt engineering may not require rethinking generalization
Victor Akinwande, Yiding Jiang, Dylan Sam, J. Zico Kolter
Vision Language Model Strong Generalization Zero Shot Learning Prompt Engineering Generalization Bound PAC Bayes Bound ImageNet Classifier

October 4, 2023

Generalization in diffusion models arises from geometry-adaptive harmonic representations
Zahra Kadkhodaie, Florentin Guth, Eero P. Simoncelli, Stéphane Mallat
Diffusion Model Deep Neural Network Strong Generalization Image Denoising Diffusion Generated Image Learning Based Denoising

October 2, 2023

Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learning
Debora Caldarola, Barbara Caputo, Marco Ciccone
Strong Generalization Heterogeneous Federated Learning Stable Learning Model Averaging Concrete Aggregate

Strong Generalization

Papers

Learning under Label Proportions for Text Classification

CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization

KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Models

Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations

PAGE: Equilibrate Personalization and Generalization in Federated Learning

Adaptivity and Modularity for Efficient Generalization Over Task Complexity

Feature Learning and Generalization in Deep Networks with Orthogonal Weights

Why Does Sharpness-Aware Minimization Generalize Better Than SGD?

Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization

Robustness May be More Brittle than We Think under Different Degrees of Distribution Shifts

Understanding the Effects of RLHF on LLM Generalisation and Diversity

Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness

Grokking as the Transition from Lazy to Rich Training Dynamics

Grokking as Compression: A Nonlinear Complexity Perspective

A Generalization Bound of Deep Neural Networks for Dependent Data

Causal Reasoning through Two Layers of Cognition for Improving Generalization in Visual Question Answering

What do larger image classifiers memorise?

Understanding prompt engineering may not require rethinking generalization

Generalization in diffusion models arises from geometry-adaptive harmonic representations

Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learning