Better Generalization

Better generalization in machine learning aims to improve a model's ability to perform well on unseen data, a crucial aspect for real-world applications. Current research focuses on techniques like weight decay, sharpness minimization, and various continual learning strategies, often applied to neural networks including transformers and convolutional models, to achieve this goal. These efforts are driven by the need for more robust and reliable AI systems across diverse domains, impacting fields ranging from medical image analysis and natural language processing to climate modeling and robotics. Improved generalization ultimately leads to more trustworthy and effective AI deployments.

Papers

March 10, 2023

Ignorance is Bliss: Robust Control via Information Gating
Manan Tomar, Riashat Islam, Matthew E. Taylor, Sergey Levine, Philip Bachman
Strong Generalization Information Theoretic Better Generalization Robust Control Gating Mechanism Parsimonious Machine Learning Parsimony Learning

January 30, 2023

On student-teacher deviations in distillation: does it pay to disobey?
Vaishnavh Nagarajan, Aditya Krishna Menon, Srinadh Bhojanapalli, Hossein Mobahi, Sanjiv Kumar
Knowledge Distillation Implicit Bias Better Generalization Low Temperature Distillation Improved Generalization Teacher Student

December 12, 2022

Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging
Peng Lu, Ivan Kobyzev, Mehdi Rezagholizadeh, Ahmad Rashid, Ali Ghodsi, Philippe Langlais
Language Model Knowledge Distillation Strong Generalization Pre Trained Language Model Better Generalization Stochastic Weight Averaging

November 29, 2022

A3T: Accuracy Aware Adversarial Training
Enes Altinisik, Safa Messaoud, Husrev Taha Sencar, Sanjay Chawla
Adversarial Training Adversarial Sample Better Generalization Adversarial Evaluation Labeled Sample

November 20, 2022

Towards Generalizable Graph Contrastive Learning: An Information Theory Perspective
Yige Yuan, Bingbing Xu, Huawei Shen, Qi Cao, Keting Cen, Wen Zheng, Xueqi Cheng
Contrastive Learning Strong Generalization Graph Representation Learning Graph Contrastive Learning Information Theory Better Generalization

November 3, 2022

FedGen: Generalizable Federated Learning for Sequential Data
Praveen Venkateswaran, Vatche Isahagian, Vinod Muthusamy, Nalini Venkatasubramanian
Federated Learning Better Generalization Sequential Data Invariant Risk Minimization Training Distribution Training Data Distribution Generalization Strategy

October 11, 2022

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models
Qihuang Zhong, Liang Ding, Li Shen, Peng Mi, Juhua Liu, Bo Du, Dacheng Tao
Language Model Sharpness Aware Minimization Model Generalization Better Generalization Better Optimizers Sparse Perturbation

September 29, 2022

Learning Gradient-based Mixup towards Flatter Minima for Domain Generalization
Danni Peng, Sinno Jialin Pan
LeArning Abstract Domain Generalization Unseen Domain Better Generalization Spatio Temporal Mixup Mechanism Flat Minimum Gradient Similarity

September 26, 2022

Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization
Gábor Melis
Adaptive Importance Stochastic Optimization Better Generalization Averaging Algorithm Tail Risk Polyak Ruppert

September 14, 2022

Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networks
Hyungeun Lee, Kijung Yoon
Generalization Performance Better Generalization GNN Framework Multimodal Distribution Graph Generator

August 19, 2022

Intersection of Parallels as an Early Stopping Criterion
Ali Vardasbi, Maarten de Rijke, Mostafa Dehghani
Strong Generalization Supervised Learning Intersection Scenario Model Overfitting Better Generalization Early Stopping Parallel Text

July 15, 2022

Bootstrap State Representation using Style Transfer for Better Generalization in Deep Reinforcement Learning
Md Masudur Rahman, Yexiang Xue
Deep Reinforcement Learning Strong Generalization Style Transfer Policy Learning Better Generalization State Representation Human Thinking

June 28, 2022

Domain Agnostic Few-shot Learning for Speaker Verification
Seunghan Yang, Debasmit Das, Janghoon Cho, Hyoungwoo Park, Sungrack Yun
Speaker Verification Better Generalization Domain Few Shot New Domain

June 13, 2022

Invariant Structure Learning for Better Generalization and Causal Explainability
Yunhao Ge, Sercan Ö. Arik, Jinsung Yoon, Ao Xu, Laurent Itti, Tomas Pfister
Causal Structure Better Generalization Invariant Causal Causal Structure Discovery

May 6, 2022

SKILL-IL: Disentangling Skill and Knowledge in Multitask Imitation Learning
Bian Xihan, Oscar Mendez, Simon Hadfield
Knowledge Based Better Generalization Unseen Environment Skill Transfer Shared Knowledge Lifelong Learning Latent Memory

April 26, 2022

Self-scalable Tanh (Stan): Faster Convergence and Better Generalization in Physics-informed Neural Networks
Raghav Gnanasambandam, Bo Shen, Jihoon Chung, Xubo Yue, Zhenyu, Kong
Physic Informed Neural Network Better Generalization Faster Convergence Thermal Conductivity

April 17, 2022

Unsupervised Cross-Task Generalization via Retrieval Augmentation
Bill Yuchen Lin, Kangmin Tan, Chris Miller, Beiwen Tian, Xiang Ren
Multi Task Retrieval Augmentation Better Generalization Multi Task Model Cross Task Generalization

March 16, 2022

Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation
Shivanshu Gupta, Sameer Singh, Matt Gardner
NLP Model Structured Output Comprehensive Evaluation Sample Efficient Better Generalization Improved Generalization Compositional Split

February 18, 2022

Towards better understanding and better generalization of few-shot classification in histology images with contrastive learning
Jiawei Yang, Hanbo Chen, Jiangpeng Yan, Xiaoyu Chen, Jianhua Yao
Contrastive Learning Shot Learning Shot Classification Image Analysis Deep Understanding Better Generalization Histology Image Space Augmentation

January 4, 2022

Learning to Generate Novel Classes for Deep Metric Learning
Kyungmoon Lee, Sungyeon Kim, Seunghoon Hong, Suha Kwak
LeArning Abstract Deep Metric Learning Conditional Generative Better Generalization Novel Class

Better Generalization

Papers

Ignorance is Bliss: Robust Control via Information Gating

On student-teacher deviations in distillation: does it pay to disobey?

Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging

A3T: Accuracy Aware Adversarial Training

Towards Generalizable Graph Contrastive Learning: An Information Theory Perspective

FedGen: Generalizable Federated Learning for Sequential Data

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models

Learning Gradient-based Mixup towards Flatter Minima for Domain Generalization

Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization

Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networks

Intersection of Parallels as an Early Stopping Criterion

Bootstrap State Representation using Style Transfer for Better Generalization in Deep Reinforcement Learning

Domain Agnostic Few-shot Learning for Speaker Verification

Invariant Structure Learning for Better Generalization and Causal Explainability

SKILL-IL: Disentangling Skill and Knowledge in Multitask Imitation Learning

Self-scalable Tanh (Stan): Faster Convergence and Better Generalization in Physics-informed Neural Networks

Unsupervised Cross-Task Generalization via Retrieval Augmentation

Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation

Towards better understanding and better generalization of few-shot classification in histology images with contrastive learning

Learning to Generate Novel Classes for Deep Metric Learning