High-Dimensional Asymptotics

Latest papers

26 papers8 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Multiclass Classification with Rare Useful Features: Fundamental Limits and the Optimality of Diversity Pursuit Higher Criticism Thresholding
Nov 1, 2026Gabriel Moryoussef, Alon KipnisMulticlass Classification Feature Selection
Department of Statistics and Data Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Efi Arazi School of Computer Science, Reichman University, Herzliya, Israel
New Multiple Descent Phenomena Induced by Heterogeneous Data
Oct 27, 2026Mark-Anthony Moisescu-Pareja, Elvis DohmatobDouble Descent Statistical Learning Theory
Concordia University · Mila – Quebec AI Institute
Approximating the Euclidean Ball by Polytopes: Surprising Rates in High Dimensions
Oct 26, 2026Yegor Gorodzha, Steven Hoehner, Gil KurHigh-Dimensional Asymptotics
ETH Zürich · Longwood University
Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis
Oct 22, 2026Yuansi Chen, Francesco Pedrotti, Peter A. WhalleyLangevin Dynamics High-Dimensional Asymptotics
Seminar for Statistics, ETH Zürich · Department of Statistics, University of Warwick
Silent Alignment in Two-Layer Linear Networks: A High-Dimensional Perspective
Oct 21, 2026Simon Martin, Francis Bach, Giulio BiroliTwo-Layer Neural Networks Neural Network Training Dynamics
DI-ENS, LPENS, SIERRA · DI-ENS, SIERRA · LPENS
Hard Way, The Only Way: Provable Merits of Training on Hardest Examples
Oct 15, 2026Elvis DohmatobHigh-Dimensional Asymptotics Dataset Pruning
Concordia University · Mila – Quebec AI Institute · FAIR – Meta
High-Dimensional Statistical Inference for Sparse Support Vector Machines
Oct 6, 2026Peng Zeng, Hanwen HuangSupport Vector Machines Feature Selection
Department of Mathematics and Statistics Auburn University, Auburn, AL 36849 · Department of Biostatistics, Data Science and Epidemiology Medical College of Georgia, Augusta University, Augusta, GA, 30912
Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Oct 6, 2026Kaito Takanami, Takashi Takahashi, Yoshiyuki KabashimaStatistical Physics of Learning High-Dimensional Asymptotics
Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo
High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning
Oct 1, 2026Filip Kovačević, Edwige Cyffers, Stefano Sarao Mannelli +1High-Dimensional Asymptotics Transfer Learning
Institute of Science and Technology Austria. · CNRS Dauphine PSL. · Computer Science and Engineering Department, Chalmers University of Technology and University of Gothenburg. +1
Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model
Sep 30, 2026Yugo NakayamaSupport Vector Machines Classification
Independent researcher
Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy
Aug 28, 2026Lorenzo Rizzi, Arie Wortsman Zurich, Bruno LoureiroKernel Ridge Regression Kernel Methods
Departement d’Informatique, École Normale Supérieure, PSL & CNRS · Dipartimento di Fisica e Astronomia, Università degli Studi di Padova · Scuola Galileiana di Studi Superiori, Università degli Studi di Padova
Influence Diagnostics in High-dimensional M-estimation: Precise Asymptotics
Jul 10, 2026Hugo CuiInfluence Functions Training Data Attribution
Université Paris-Saclay, CNRS, Laboratoire de mathématiques d’Orsay, 91405, Orsay, France
Homogenization of $\ell_2$ -Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent
Jun 30, 2026Fabrizzio SabelliAdversarial Training High-Dimensional Asymptotics
Université de Montréal
Effective Covariance Dynamics in Solvable High-Dimensional GANs
Jun 25, 2026Andrew Bond, Zafer DoğanHigh-Dimensional Asymptotics Neural Network Training Dynamics
MLIP Group, KUIS AI Center, Koç University, İstanbul, Turkey. · Dept. of EE Engineering, Koç University, İstanbul, Turkey.
Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
May 27, 2026Katie Everett, Elliot PaquetteHigh-Dimensional Asymptotics Momentum Methods
Google DeepMind & MIT · McGill University & Mila
High-dimensional Limit of SGD for Diagonal Linear Networks
May 16, 2026Begoña García Malaxechebarría, Courtney Paquette, Maryam Fazel +1Stochastic Optimization Convergence Neural Network Training Dynamics
University of Washington · McGill University · Google DeepMind +2
Large Dimensional Kernel Ridge Regression: Extending to Product Kernels
May 14, 2026Yang Zhou, Yicheng Li, Yuqian Cheng +1Generalization Bounds Kernel Ridge Regression
Department of Statistics and Data Science Tsinghua University Beijing, 100084, China · Department of Mathematical Science Tsinghua University Beijing, 100084, China
Characterizing the Generalization Error of Random Feature Regression with Arbitrary Data-Augmentation
May 11, 2026Lucas Morisset, Alain Durmus, Adrien HardyNeural Network Generalization High-Dimensional Asymptotics
Qube Research & Technologies · École Polytechnique
Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization
Apr 3, 2026Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko +1Empirical Risk Minimization High-Dimensional Asymptotics
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · Huawei Noah’s Ark Lab, Huawei Technologies, Paris, France · School of Electronic Information and Communications, Huazhong University of Science & Technology, China
Precise Performance of Linear Denoisers in the Proportional Regime
Mar 19, 2026Reza Ghane, Danil Akhtiamov, Babak HassibiHigh-Dimensional Asymptotics
California Institute of Technology, Pasadena, CA, 91125
Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval
Feb 19, 2026Antoine Maillard, Tony Bonnaire, Giulio BiroliLoss Landscape Loss Landscape Geometry
INRIA Paris & DI ENS, PSL University, Paris, France. · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France
High-dimensional learning dynamics of multi-pass Stochastic Gradient Descent in multi-index models
Jan 28, 2026Zhou Fan, Leda WangDynamical Mean-Field Theory Stochastic Gradient Descent
Department of Statistics and Data Science, Yale University
Spectral Thresholds in Correlated Spiked Models and Fundamental Limits of Partial Least Squares
Oct 20, 2025Pierre Mergny, Lenka ZdeborováHigh-Dimensional Asymptotics Phase Transitions
Information, Learning and Physics Laboratory (IdePHICS), EPFL, 1015 Lausanne, Switzerland · Statistical Physics of Computation Laboratory (SPOC), EPFL, 1015 Lausanne, Switzerland
Exact Dynamics of Multi-class Stochastic Gradient Descent
Oct 15, 2025Elizabeth Collins-Woodfin, Inbar SeroussiLogistic Regression High-Dimensional Asymptotics
Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
Jun 16, 2025Zhenyu Liao, Michael W. MahoneyRandom Matrix Theory Neural Network Generalization
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China · ICSI, LBNL, and Department of Statistics, University of California, Berkeley, USA
Universality of empirical risk minimization
Feb 17, 2022Andrea Montanari, Basil SaeedStatistical Learning Theory Empirical Risk Minimization
Department of Statistics and Department of Mathematics, Stanford University · Department of Electrical Engineering, Stanford University

Nov 1, 2026J/K move · Enter open · S save

Multiclass Classification with Rare Useful Features: Fundamental Limits and the Optimality of Diversity Pursuit Higher Criticism Thresholding

Gabriel Moryoussef, Alon Kipnis

Department of Statistics and Data Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Efi Arazi School of Computer Science, Reichman University, Herzliya, Israel

Multiclass Classification Feature Selection High-Dimensional Asymptotics

Open paper AskPDF arXiv

We consider classification in high-dimensional feature spaces where only a vanishing fraction of features are useful and the number of classes may be large. While this rare-and-weak useful-feature regime is well understood in the binary setting, its multiclass version raises new questions: a feature can be useful for class discrimination without separating any specific pair of classes. We focus on a rare feature diversity model, in which most features are useless, but a small fraction promote discrimination among classes in a way that can be detected by a per-feature diversity test, such as one-way ANOVA or the Kruskal-Wallis test. For this setting, we propose Diversity Pursuit Higher Criticism Thresholding (DP-HCT), a feature-screening procedure that applies a higher-criticism threshold to the collection of p-values obtained from per-feature diversity tests. We justify DP-HCT under a Gaussian multiclass Rare-and-Weak Useful Feature model with class-mean contrasts of fixed magnitude in random directions, extending the binary model of Donoho and Jin (2008). When the number of classes is fixed, our model recovers the binary phase transition. When the number of classes grows logarithmically with the number of features, however, we derive a new and strictly harder phase transition, driven by the growing degrees of freedom and their effect on the useful-feature tail rate. We show that whenever classification beyond chance is possible, DP-HCT with ANOVA, followed by a simple classifier based on similarity to class means, achieves asymptotically perfect accuracy. Experiments on real multiclass classification benchmarks demonstrate substantial improvements, and numerical simulations confirm that the derived phase-transition curve accurately predicts the finite-sample behavior of DP-HCT in the growing-class regime.