Gaussian Universality

Gaussian universality in high-dimensional statistics explores whether the performance of machine learning models, particularly in classification and regression tasks, is largely insensitive to the specific data distribution, instead converging to predictions based on Gaussian assumptions. Current research focuses on identifying conditions under which this universality holds, examining its validity across various model architectures like two-layer neural networks and generalized linear models, and investigating the role of regularization and data characteristics like heavy tails. Understanding the limits of Gaussian universality is crucial for improving the robustness and generalizability of machine learning algorithms and for developing more accurate theoretical analyses of their performance.

Papers

October 8, 2024

The Breakdown of Gaussian Universality in Classification of High-dimensional Mixtures
Xiaoyi Mai, Zhenyu Liao
Classification Code Gaussian Mixture Gaussian Model Gaussian Data Dimensional Gaussian Mixture Gaussian Universality

February 7, 2024

Asymptotics of feature learning in two-layer networks after one gradient-step
Hugo Cui, Luca Pesce, Yatin Dandi, Florent Krzakala, Yue M. Lu, Lenka Zdeborová, Bruno Loureiro
High Dimensional Feature Learning Two Layer Neural Network Asymptotic Behavior Two Layer Network Kernel Regime Gradient Step Gaussian Universality

April 6, 2023

Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approach
Urte Adomaityte, Gabriele Sicuro, Pierpaolo Vivo
Classification Code Regularization Model High Dimension Heavy Tailed Gaussian Distribution Convex Regularization General Convex Loss Gaussian Universality Self Fitting Method

February 17, 2023

Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation
Luca Pesce, Florent Krzakala, Bruno Loureiro, Ludovic Stephan
Continuum Limit RSD Difference of Gaussian Gaussian Mixture K$ Universality Generalized Linear Model Gaussian Data Mixture Signal Gaussian Universality

February 1, 2023

Deterministic equivalent and error universality of deep random features learning
Dominik Schröder, Hugo Cui, Daniil Dmitriev, Bruno Loureiro
Covariance Neural Network Gaussian Universality Deep Random Feature

December 21, 2022

A Nearly Tight Bound for Fitting an Ellipsoid to Gaussian Random Points
Daniel M. Kane, Ilias Diakonikolas
Spherical Surface Statistical Query Tight Bound Discrete Logarithm Gaussian Universality Mathematical Constant

May 26, 2022

Gaussian Universality of Perceptrons with Random Labels
Federica Gerace, Florent Krzakala, Bruno Loureiro, Ludovic Stephan, Lenka Zdeborová
Linear Classifier Gaussian Model Gaussian Data Random Label Elementary Perceptron Gaussian Universality

December 16, 2021

Non-Gaussian Component Analysis via Lattice Basis Reduction
Ilias Diakonikolas, Daniel M. Kane
Non Gaussian Gaussian Universality Lattice Basis Reduction