Generalization Performance

Generalization performance in machine learning focuses on a model's ability to accurately predict outcomes on unseen data, a crucial aspect for real-world applications. Current research investigates this through various lenses, including mitigating overfitting in self-supervised and federated learning, improving the robustness of models to out-of-distribution data (e.g., using dropout or orthogonal regularization), and enhancing the efficiency of fine-tuning large pre-trained models (e.g., via low-rank adaptation). Understanding and improving generalization is vital for building reliable and adaptable AI systems across diverse domains, impacting fields from image recognition and natural language processing to control of biological neural networks.

Papers

March 1, 2022

Contrasting random and learned features in deep Bayesian linear regression
Jacob A. Zavatone-Veth, William L. Tong, Cengiz Pehlevan
Deep Network Generalization Performance Feature Wise Deep Linear Deep Learning Theory Deep Random Feature

February 26, 2022

SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalization
Austin W. Hanjie, Ameet Deshpande, Karthik Narasimhan
Zero Shot Generalization Performance Zero Shot Learning Natural Language Description Text Supervision Semantic Self Supervision

February 23, 2022

Better Modelling Out-of-Distribution Regression on Distributed Acoustic Sensor Data Using Anchored Hidden State Mixup
Hasan Asyari Arief, Peter James Thomas, Tomasz Wiktorski
Generalization Performance Acoustic Data Regularization Penalty Manifold Mixup

February 15, 2022

Generalisation and the Risk--Entropy Curve
Dominic Belcher, Antonia Marcu, Adam Prügel-Bennett
Machine Learning Deep Learning Model Strong Generalization Generalization Performance Aggregated Curve Entropic Risk

February 8, 2022

Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning
Yang Zhao, Hao Zhang, Xiuyuan Hu
Deep Learning Strong Generalization Loss Function Gradient Descent Generalization Performance Gradient Norm Model Generalization

February 6, 2022

January 28, 2022

Bioinspired Cortex-based Fast Codebook Generation
Meric Yucel, Serdar Bagis, Ahmet Sertbas, Mehmet Sarikaya, Burak Berk Ustundag
Machine Learning Generalization Performance Brain Inspired Bio Inspired

January 24, 2022

Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search
Yao Shu, Zhongxiang Dai, Zhaoxuan Wu, Bryan Kian Hsiang Low
Neural Architecture Search Neural Architecture Generalization Performance Anti Unification Token Boosting Free Neural Architecture Search Training Free Metric

January 21, 2022

Distance-Ratio-Based Formulation for Metric Learning
Hyeongji Kim, Pekka Parviainen, Ketil Malde
Generalization Performance Metric Learning Softmax Function New Formulation Mini ImageNet

January 17, 2022

ExpertNet: A Symbiosis of Classification and Clustering
Shivin Srivastava, Kenji Kawaguchi, Vaibhav Rajan
Classification Code Representation Learning Latent Representation Generalization Performance Clustering Friendly Representation Symbiotic Transformer

January 6, 2022

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, Vedant Misra
Neural Network Deep Learning Strong Generalization Generalization Performance Model Overfitting Small Datasets Grokking Phenomenon

January 3, 2022

Novelty-based Generalization Evaluation for Traffic Light Detection
Arvind Kumar Shekar, Laureen Lake, Liang Gou, Liu Ren
Convolutional Neural Network Strong Generalization Generalization Performance Traffic Light Detection Novel Class Generalization

December 21, 2021

Learning Positional Embeddings for Coordinate-MLPs
Sameera Ramasinghe, Simon Lucey
Generalization Performance Graph Laplacian Coordinate Based Positional Embeddings

December 15, 2021

Robust Neural Network Classification via Double Regularization
Olof Zetterqvist, Rebecka Jörnsten, Johan Jonasson
Generalization Performance Pseudo Labelling

December 14, 2021

On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training
Chen Liu, Zhichao Huang, Mathieu Salzmann, Tong Zhang, Sabine Süsstrunk
Adversarial Attack Adversarial Training Global Impact Adversarial Learning Generalization Performance Adversarial Instance

December 7, 2021

A generalization gap estimation for overparameterized models via the Langevin functional variance
Akifumi Okuno, Keisuke Yano
Full Model Generalization Performance Langevin Dynamic Generalization Gap Sample Variance

December 3, 2021

Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting
Ryo Karakida, Shotaro Akaho
Neural Network Strong Generalization Transfer Learning Continual LEArning Generalization Performance Sequential Learning Learning Curve

November 28, 2021

Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Nets
Shao-Bo Lin, Yao Wang, Ding-Xuan Zhou
Generalization Performance Empirical Risk Minimization Deep ReLU Network Optimal Generalization

November 9, 2021

Learning Rates for Nonconvex Pairwise Learning
Shaojie Li, Yong Liu
Gradient Descent Generalization Performance Learning Rate Pairwise Learning