Strong Generalization

Strong generalization, the ability of machine learning models to perform well on unseen data, is a central objective in current research. Active areas of investigation include improving the robustness of self-supervised learning, understanding the optimization dynamics of transformers and other architectures (including CNNs and RNNs), and developing methods to enhance generalization through data augmentation, regularization techniques (e.g., logical regularization, consistency regularization), and improved training strategies (e.g., few-shot learning, meta-learning). These advancements are crucial for building reliable and adaptable AI systems across diverse applications, from image classification and natural language processing to healthcare and robotics.

Papers

March 3, 2023

February 28, 2023

DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks
Samyak Jain, Sravanti Addepalli, Pawan Sahu, Priyam Dey, R. Venkatesh Babu
Neural Network Strong Generalization Training Data Domain Generalization Improved Generalization

February 26, 2023

Training neural networks with structured noise improves classification and generalization
Marco Benedetti, Enrico Ventura
Neural Network Strong Generalization Classification Code Noise Injection Neural System Hebbian Learning Rule

February 25, 2023

February 23, 2023

Generalization of Auto-Regressive Hidden Markov Models to Non-Linear Dynamics and Unit Quaternion Observation Space
Michele Ginesi, Paolo Fiorini
Strong Generalization Markov Chain Latent Variable Model Auto Regressive Non Linear Dynamic Semi Markov

February 22, 2023

Learning to Generalize Provably in Learning to Optimize
Junjie Yang, Tianlong Chen, Mingkang Zhu, Fengxiang He, Dacheng Tao, Yingbin Liang, Zhangyang Wang
LeArning Abstract Strong Generalization Learned Optimizers Optimal Generalization

February 20, 2023

February 19, 2023

Generalization in Visual Reinforcement Learning with the Reward Sequence Distribution
Jie Wang, Rui Yang, Zijie Geng, Zhihao Shi, Mingxuan Ye, Qi Zhou, Shuiwang Ji, Bin Li, Yongdong Zhang, Feng Wu
Strong Generalization Markov Decision Process State Space Visual Reinforcement Learning Partially Observed Markov Decision Process Reward Distribution

February 18, 2023

Generalization and Stability of Interpolating Neural Networks with Minimal Width
Hossein Taheri, Christos Thrampoulidis
Neural Network Strong Generalization Gradient Descent Core Stability Learning Rate Generalization Bound Shallow Neural Network Minimum Width

February 16, 2023

Learning Multi-Object Positional Relationships via Emergent Communication
Yicheng Feng, Boshi An, Zongqing Lu
LeArning Abstract Strong Generalization Emergent Communication Emergent Language Referential Game

February 15, 2023

On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it
Andrea Bonfanti, Roberto Santana, Marco Ellero, Babak Gholami
Neural Network Strong Generalization Physic Informed Neural Network Related Hyperparameters Training Domain Domain Decomposition PINN Model Consistent Prediction

February 14, 2023

Strong Generalization

Papers

Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization

Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models

Over-training with Mixup May Hurt Generalization

The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks

DSD$^2$: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?

DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks

Training neural networks with structured noise improves classification and generalization

nnUNet RASPP for Retinal OCT Fluid Detection, Segmentation and Generalisation over Variations of Data Sources

Explaining Generalization Power of a DNN Using Interactive Concepts

Generalization of Auto-Regressive Hidden Markov Models to Non-Linear Dynamics and Unit Quaternion Observation Space

Learning to Generalize Provably in Learning to Optimize

Progressive Ensemble Distillation: Building Ensembles for Efficient Inference

Towards Unbounded Machine Unlearning

On the Stability and Generalization of Triplet Learning

Generalization in Visual Reinforcement Learning with the Reward Sequence Distribution

Generalization and Stability of Interpolating Neural Networks with Minimal Width

Learning Multi-Object Positional Relationships via Emergent Communication

On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it

AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models

A Modern Look at the Relationship between Sharpness and Generalization