Strong Generalization

Strong generalization, the ability of machine learning models to perform well on unseen data, is a central objective in current research. Active areas of investigation include improving the robustness of self-supervised learning, understanding the optimization dynamics of transformers and other architectures (including CNNs and RNNs), and developing methods to enhance generalization through data augmentation, regularization techniques (e.g., logical regularization, consistency regularization), and improved training strategies (e.g., few-shot learning, meta-learning). These advancements are crucial for building reliable and adaptable AI systems across diverse applications, from image classification and natural language processing to healthcare and robotics.

Papers

October 11, 2022

October 10, 2022

October 6, 2022

State-of-the-art generalisation research in NLP: A taxonomy and review
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske, Ryan Cotterell, Zhijing Jin
Natural Language Processing Strong Generalization Narrative Review Comprehensive Taxonomy

October 5, 2022

October 3, 2022

September 29, 2022

Learning Parsimonious Dynamics for Generalization in Reinforcement Learning
Tankred Saanum, Eric Schulz
Reinforcement Learning Strong Generalization Model Based Reinforcement Learning Latent Variable Policy Learning Latent Dynamic Parsimonious Machine Learning

September 28, 2022

Generalization in Deep Reinforcement Learning for Robotic Navigation by Reward Shaping
Victor R. F. Miranda, Armando A. Neto, Gustavo M. Freitas, Leonardo A. Mozelli
Deep Reinforcement Learning Strong Generalization Collision Avoidance Robot Navigation Reward Shaping DRL Algorithm Exteroceptive Perception COLlision Avoidance

September 24, 2022

Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptation
Kang Xu, Yan Ma, Bingsheng Wei, Wei Li
Reinforcement Learning Domain Adaptation Strong Generalization Free Counterpart Diverse Policy Diverse Solution Behavioral Diversity

September 21, 2022

Periodic Extrapolative Generalisation in Neural Networks
Peter Belcák, Roger Wattenhofer
Neural Network Strong Generalization Extrapolation Framework Periodic Signal Periodicity Detection

Strong Generalization

Papers

C-Mixup: Improving Generalization in Regression

Transformers generalize differently from information stored in context vs in weights

The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective

Everything is Varied: The Surprising Impact of Individual Variation on ML Robustness in Medicine

State-of-the-art generalisation research in NLP: A taxonomy and review

Bayesian Prompt Learning for Image-Language Model Generalization

The Calibration Generalization Gap

Tree Mover's Distance: Bridging Graph Metrics and Stability of Graph Neural Networks

Neural-Symbolic Recursive Machine for Systematic Generalization

A Study on the Efficiency and Generalization of Light Hybrid Retrievers

Omnigrok: Grokking Beyond Algorithmic Data

On Stability and Generalization of Bilevel Optimization Problem

Information-Theoretic Analysis of Unsupervised Domain Adaptation

On the optimization and generalization of overparameterized implicit neural networks

Emergent Communication: Generalization and Overfitting in Lewis Games

Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel

Learning Parsimonious Dynamics for Generalization in Reinforcement Learning

Generalization in Deep Reinforcement Learning for Robotic Navigation by Reward Shaping

Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptation

Periodic Extrapolative Generalisation in Neural Networks