Strong Generalization

Strong generalization, the ability of machine learning models to perform well on unseen data, is a central objective in current research. Active areas of investigation include improving the robustness of self-supervised learning, understanding the optimization dynamics of transformers and other architectures (including CNNs and RNNs), and developing methods to enhance generalization through data augmentation, regularization techniques (e.g., logical regularization, consistency regularization), and improved training strategies (e.g., few-shot learning, meta-learning). These advancements are crucial for building reliable and adaptable AI systems across diverse applications, from image classification and natural language processing to healthcare and robotics.

Papers

July 18, 2023

Can Neural Network Memorization Be Localized?
Pratyush Maini, Michael C. Mozer, Hanie Sedghi, Zachary C. Lipton, J. Zico Kolter, Chiyuan Zhang
Strong Generalization Limited Memorization Generalization Gap Training Example

July 17, 2023

Neural Image Compression: Generalization, Robustness, and Spectral Biases
Kelsey Lieberman, James Diffenderfer, Charles Godfrey, Bhavya Kailkhura
Native Robustness Strong Generalization Image Compression Neural Image Compression Performance Comparison Spectral Bias

July 16, 2023

MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning
Zhenwen Liang, Dian Yu, Xiaoman Pan, Wenlin Yao, Qingkai Zeng, Xiangliang Zhang, Dong Yu
Language Model Knowledge Distillation Strong Generalization Mathematical Reasoning MATH Dataset Reasoning Domain

July 15, 2023

Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks
Dominik Schnaus, Jongseok Lee, Daniel Cremers, Rudolph Triebel
Neural Network Strong Generalization Uncertainty Estimation Generalization Bound Generalization Guarantee Expressive Neural Non Vacuous Generalization Bound Learned Prior

July 14, 2023

Improving Zero-Shot Generalization for CLIP with Synthesized Prompts
Zhengbo Wang, Jian Liang, Ran He, Nan Xu, Zilei Wang, Tieniu Tan
Vision Language Model Strong Generalization Variational Autoencoders Single CLIP Generative Approach Generalized Zero Shot Learning Diverse Prompt Synthetic Feature

July 13, 2023

A Causal Framework to Unify Common Domain Generalization Approaches
Nevin L. Zhang, Kaican Li, Han Gao, Weiyan Xie, Zhi Lin, Zhenguo Li, Luning Wang, Yongxiang Huang
Machine Learning Strong Generalization Domain Generalization

July 3, 2023

MoVie: Visual Model-Based Policy Adaptation for View Generalization
Sizhe Yang, Yanjie Ze, Huazhe Xu
Strong Generalization Real World Visual Reinforcement Learning Reward Signal Movie Review

July 1, 2023

June 30, 2023

Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networks
Martin Ferianc, Ondrej Bohdal, Timothy Hospedales, Miguel Rodrigues
Neural Network Strong Generalization Study Feature Label Noise Calibration Performance Learned Representation Massart Noise Label Invariant Augmentation Generalisation Ability

June 28, 2023

On information captured by neural networks: connections with memorization and generalization
Hrayr Harutyunyan
Neural Network Deep Learning Strong Generalization Deep Network Full Information Limited Memorization Generalization Gap New Connection Neural Network Generalization

June 27, 2023

Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost
Parikshit Bansal, Amit Sharma
Large Language Model Strong Generalization Active Learning NLP Model Individual Annotator LLM Annotation Minimal Cost

June 23, 2023

Strong Generalization

Papers

Can Neural Network Memorization Be Localized?

Neural Image Compression: Generalization, Robustness, and Spectral Biases

MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning

Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks

Improving Zero-Shot Generalization for CLIP with Synthesized Prompts

A Causal Framework to Unify Common Domain Generalization Approaches

SAR: Generalization of Physiological Agility and Dexterity via Synergistic Action Representation

Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation

Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms

GIT: Detecting Uncertainty, Out-Of-Distribution and Adversarial Samples using Gradients and Invariance Transformations

Jailbroken: How Does LLM Safety Training Fail?

FAM: Relative Flatness Aware Minimization

MoVie: Visual Model-Based Policy Adaptation for View Generalization

Sparsity-aware generalization theory for deep neural networks

Recursive Algorithmic Reasoning

Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networks

On information captured by neural networks: connections with memorization and generalization

Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost

Understanding quantum machine learning also requires rethinking generalization

Catching Image Retrieval Generalization