High-Dimensional Asymptotics

Latest papers

19 papers5 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Hard Way, The Only Way: Provable Merits of Training on Hardest Examples
Oct 15, 2026Elvis DohmatobHigh-Dimensional Asymptotics Dataset Pruning
Concordia University · Mila – Quebec AI Institute · FAIR – Meta
High-Dimensional Statistical Inference for Sparse Support Vector Machines
Oct 6, 2026Peng Zeng, Hanwen HuangSupport Vector Machines Feature Selection
Department of Mathematics and Statistics Auburn University, Auburn, AL 36849 · Department of Biostatistics, Data Science and Epidemiology Medical College of Georgia, Augusta University, Augusta, GA, 30912
Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Oct 6, 2026Kaito Takanami, Takashi Takahashi, Yoshiyuki KabashimaStatistical Physics of Learning High-Dimensional Asymptotics
Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo
High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning
Oct 1, 2026Filip Kovačević, Edwige Cyffers, Stefano Sarao Mannelli +1High-Dimensional Asymptotics Transfer Learning
Institute of Science and Technology Austria. · CNRS Dauphine PSL. · Computer Science and Engineering Department, Chalmers University of Technology and University of Gothenburg. +1
Asymptotic Properties of Support Vector Machines in High-Dimension, Low-Sample-Size Settings under a Spiked Model
Sep 30, 2026Yugo NakayamaSupport Vector Machines Classification
Independent researcher
Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy
Aug 28, 2026Lorenzo Rizzi, Arie Wortsman Zurich, Bruno LoureiroKernel Ridge Regression Kernel Methods
Departement d’Informatique, École Normale Supérieure, PSL & CNRS · Dipartimento di Fisica e Astronomia, Università degli Studi di Padova · Scuola Galileiana di Studi Superiori, Università degli Studi di Padova
Influence Diagnostics in High-dimensional M-estimation: Precise Asymptotics
Jul 10, 2026Hugo CuiInfluence Functions Training Data Attribution
Université Paris-Saclay, CNRS, Laboratoire de mathématiques d’Orsay, 91405, Orsay, France
Homogenization of $\ell_2$ -Adversarial Training in High-Dimensions: Exact Dynamics under Stochastic Gradient Descent
Jun 30, 2026Fabrizzio SabelliAdversarial Training High-Dimensional Asymptotics
Université de Montréal
Effective Covariance Dynamics in Solvable High-Dimensional GANs
Jun 25, 2026Andrew Bond, Zafer DoğanHigh-Dimensional Asymptotics Neural Network Training Dynamics
MLIP Group, KUIS AI Center, Koç University, İstanbul, Turkey. · Dept. of EE Engineering, Koç University, İstanbul, Turkey.
Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
May 27, 2026Katie Everett, Elliot PaquetteHigh-Dimensional Asymptotics Momentum Methods
Google DeepMind & MIT · McGill University & Mila
High-dimensional Limit of SGD for Diagonal Linear Networks
May 16, 2026Begoña García Malaxechebarría, Courtney Paquette, Maryam Fazel +1Stochastic Optimization Convergence Neural Network Training Dynamics
University of Washington · McGill University · Google DeepMind +2
Large Dimensional Kernel Ridge Regression: Extending to Product Kernels
May 14, 2026Yang Zhou, Yicheng Li, Yuqian Cheng +1Generalization Bounds Kernel Ridge Regression
Department of Statistics and Data Science Tsinghua University Beijing, 100084, China · Department of Mathematical Science Tsinghua University Beijing, 100084, China
Characterizing the Generalization Error of Random Feature Regression with Arbitrary Data-Augmentation
May 11, 2026Lucas Morisset, Alain Durmus, Adrien HardyNeural Network Generalization High-Dimensional Asymptotics
Qube Research & Technologies · École Polytechnique
Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization
Apr 3, 2026Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko +1Empirical Risk Minimization High-Dimensional Asymptotics
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · Huawei Noah’s Ark Lab, Huawei Technologies, Paris, France · School of Electronic Information and Communications, Huazhong University of Science & Technology, China
Precise Performance of Linear Denoisers in the Proportional Regime
Mar 19, 2026Reza Ghane, Danil Akhtiamov, Babak HassibiHigh-Dimensional Asymptotics
California Institute of Technology, Pasadena, CA, 91125
Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval
Feb 19, 2026Antoine Maillard, Tony Bonnaire, Giulio BiroliLoss Landscape Loss Landscape Geometry
INRIA Paris & DI ENS, PSL University, Paris, France. · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France
Exact Dynamics of Multi-class Stochastic Gradient Descent
Oct 15, 2025Elizabeth Collins-Woodfin, Inbar SeroussiLogistic Regression High-Dimensional Asymptotics
Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models
Jun 16, 2025Zhenyu Liao, Michael W. MahoneyRandom Matrix Theory Neural Network Generalization
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China · ICSI, LBNL, and Department of Statistics, University of California, Berkeley, USA
Universality of empirical risk minimization
Feb 17, 2022Andrea Montanari, Basil SaeedStatistical Learning Theory Empirical Risk Minimization
Department of Statistics and Department of Mathematics, Stanford University · Department of Electrical Engineering, Stanford University

Oct 15, 2026J/K move · Enter open · S save

Hard Way, The Only Way: Provable Merits of Training on Hardest Examples

Elvis Dohmatob

Concordia University · Mila – Quebec AI Institute · FAIR – Meta

High-Dimensional Asymptotics Dataset Pruning Statistical Learning Theory Empirical Risk Minimization Hard Negative Mining

Open paper AskPDF arXiv

Despite a surge of empirical interest in the ``unreasonable effectiveness'' of drastic data pruning for AI systems, a unified theoretical understanding of this phenomenon remains elusive. In this work, we provide a precise high-dimensional theory of learning under data pruning via weighted empirical risk minimization (ERM). As in previous work, we analyze a setting where pruning is performed along a one-dimensional subspace of the input space. Our analysis precisely characterizes the limiting test error curves in the regime where the input dimension N and sample size P tend to infinity with a fixed ratio N/P. The resulting theory recovers several recent results in the literature as special cases and provides a general lens to evaluate how pruning strategies interact with the loss function and label noise. Crucially, we characterize the optimal pruning strategy in the high-sample regime where P greatly exceeds N. We demonstrate that strategic pruning can accelerate the decay of classification test error from the classical sqrt(N/P) rate, typical of vanilla ERM with loss functions without exponential tail (e.g., quadratic loss), to the much faster Bayes-optimal scaling of order N/P. In this setup, we prove that, surprisingly, this optimal performance is achieved by a drastic pruning strategy: discarding almost all training data except for a small proportion, consisting strictly of the 2N hardest training examples.