Better Out of Distribution Generalization

Improving the out-of-distribution (OOD) generalization of machine learning models—their ability to perform well on data differing significantly from their training data—is a major research focus. Current efforts concentrate on mitigating biases that lead to overfitting on training data, exploring the use of modular architectures and memory-based methods, and developing techniques like consistency training and invariant representation learning to enhance robustness. These advancements aim to create more reliable and adaptable AI systems, with significant implications for applications where data variability is inherent, such as personalized medicine and federated learning.

Papers

October 9, 2023

Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness
Bhavya Vasudeva, Kameron Shahabi, Vatsal Sharan
Deep Learning Native Robustness Strong Generalization Distribution Generalization Simplicity Bias Biased Prediction Better Out of Distribution Generalization

June 16, 2023

Studying Generalization on Memory-Based Methods in Continual Learning
Felipe del Rio, Julio Hurtado, Cristian Buc, Alvaro Soto, Vincenzo Lomonaco
Strong Generalization Continual LEArning Distribution Generalization Memory Replay Memory Based Better Out of Distribution Generalization

February 13, 2023

PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees
Chulin Xie, De-An Huang, Wenda Chu, Daguang Xu, Chaowei Xiao, Bo Li, Anima Anandkumar
Personalized Federated Learning Generalization Guarantee pFL Method Better Out of Distribution Generalization

November 21, 2022

Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning
Xueyang Tang, Song Guo, Jie Zhang
Distribution Generalization Better Out of Distribution Generalization

November 1, 2022

Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks
Sadegh Mahdavi, Kevin Swersky, Thomas Kipf, Milad Hashemi, Christos Thrampoulidis, Renjie Liao
Deep Neural Network Message Passing Neural Algorithmic Reasoning Algorithmic Reasoning Better Out of Distribution Generalization

October 23, 2022

Functional Indirection Neural Estimator for Better Out-of-distribution Generalization
Kha Pham, Hung Le, Man Ngo, Truyen Tran
Strong Generalization Distribution Generalization Functional Learning Better Out of Distribution Generalization

June 9, 2022

On the Generalization and Adaption Performance of Causal Models
Nino Scherrer, Anirudh Goyal, Stefan Bauer, Yoshua Bengio, Nan Rosemary Ke
Strong Generalization Causal Structure Neural Causal Model Differentiable Causal Discovery AdaPtation Performance Better Out of Distribution Generalization

June 6, 2022

Is a Modular Architecture Enough?
Sarthak Mittal, Yoshua Bengio, Guillaume Lajoie
Modularity Function Data Distribution Modular Architecture Better Out of Distribution Generalization

May 30, 2022

Better Out of Distribution Generalization

Papers

Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness

Studying Generalization on Memory-Based Methods in Continual Learning

PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees

Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning

Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks

Functional Indirection Neural Estimator for Better Out-of-distribution Generalization

On the Generalization and Adaption Performance of Causal Models

Is a Modular Architecture Enough?

Segmentation Consistency Training: Out-of-Distribution Generalization for Medical Image Segmentation

PAC Generalization via Invariant Representations