Empirical Risk Minimization

Also known as ERM

Empirical risk minimization chooses a model or decision rule to minimize average loss on a training sample, providing a foundation for regression, classification, and many neural-network objectives. Statistical analyses use localized Rademacher complexity, concentration and small-ball methods, and high-dimensional tools such as Gaussian comparison and approximate message passing to bound test error and study when structure, weighting, or constraints help. Optimization work examines gradient methods, regularization, distributed primal-dual algorithms, and privacy-preserving variants, while extensions address biased or dependent data, distribution shift, heavy tails, and selective labels. A central tension is that low empirical risk or faster fitting need not ensure reliable test performance: guarantees depend on model complexity and data conditions, while computational, privacy, and robustness constraints introduce further tradeoffs.

Latest papers

45 papers5 papers in the last four weeks, down 29% on the four weeks before. 0.0% of all new papers.

Benign population landscapes do not transfer to empirical risk minimization in nonsmooth optimization
Nov 1, 2026Penghao Wang, Mateo Díaz, Liwei JiangNonconvex Optimization Empirical Risk Minimization
Edwardson School of Industrial Engineering, Purdue University. · Department of Statistics & Data Science Institute, University of Chicago.
Two Sample Sizes in Test-Time LLM Ensembling: Buffered ERM and Minimax Theory
Oct 15, 2026Zhi Zhang, Arash A. AminiLanguage Model Ensembles Test-Time Scaling
Department of Statistics and Data Science, UCLA
Hard Way, The Only Way: Provable Merits of Training on Hardest Examples
Oct 15, 2026Elvis DohmatobHigh-Dimensional Asymptotics Dataset Pruning
Concordia University · Mila – Quebec AI Institute · FAIR – Meta
Learning a Group Element from Its Action: Galois Correspondences, Optimal ERM, and an Abel-Ruffini Threshold for the Natarajan Dimension
Oct 13, 2026Xiaoyu LiSample Complexity Multiclass Classification
University of New South Wales
Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Oct 6, 2026Kaito Takanami, Takashi Takahashi, Yoshiyuki KabashimaStatistical Physics of Learning High-Dimensional Asymptotics
Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo
Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations
Sep 24, 2026Mihir Dhanakshirur, Adam Ousherovitch, Ambuj TewariEntropy Regularization LLM Reasoning
Department of Statistics, University of Michigan
Tail-Weight Control and Localized Generalization in Nearly Low-Rank Adversarial Classification
Sep 20, 2026Kunyu Wang, Dehan Wang, Wenjun ChenAdversarial Robustness Robust Loss Functions
The Chinese University of Hong Kong · Columbia University
Weighted Empirical Risk Minimization for Machine Learning under Long-Range Dependence: Exact Pathwise Rates and Learning-Error Geometry
Sep 9, 2026Elina MoldavskayaStatistical Learning Theory Empirical Risk Minimization
Technion–Israel Institute of Technology, Israel
An Exponential Deterministic--Randomized Gap in ERM-Oracle Complexity for Thresholds on an Unknown Order
Sep 9, 2026Xuan LiQuery Complexity Oracle Complexity
University of New South Wales, Sydney, Australia
Weighted Data Selection: Sharp Upper-Half and Five-Dimensional Laws
Sep 8, 2026Zhongxuan Liu, Hongzhi WangData Selection Training Data Selection
Faculty of Computing, Harbin Institute of Technology
Fitting and Learning Basis-Restricted Propositional Formulas
Sep 8, 2026Balder ten CateBoolean Function Learning Empirical Risk Minimization
GPU-Enabled Large-Scale Optimization Using Randomized Linear Algebra
Sep 8, 2026Pratik Rathore, Zachary Frangella, Parth Nobel +2Empirical Risk Minimization
Stanford University
An Agnostic Sample Compression Scheme for Squared Loss of Near-Linear Size in the Fat-Shattering Dimension
Aug 31, 2026Guangjian Zhang, Zhengyi Yang, Xiaoyang WangAgnostic Learning Sample Compression
Exact Risk Ratios for Weighted Data Selection in Linear Regression
Aug 28, 2026Guangjian ZhangTraining Data Selection Linear Regression
University of New South Wales
ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning
Aug 10, 2026Ange-Clément Akazan, Ineza Remy Mugenga, Abebe Geletu +2Chance-Constrained Optimization Classification
School of Agriculture and Science, University of KwaZulu-Natal, Pietermaritzburg, KwaZulu-Natal, South Africa · 2AIMS Research and Innovation Centre, African Institute for Mathematical Sciences, Kigali, Rwanda · 3African Institute for Mathematical Sciences, Kigali, Rwanda
Tight Sample Complexity for Low-Rank Adaptation: Matching Bounds and Rank Selection
Jul 30, 2026Arunan JSample Complexity Low-Rank Adaptation
Independent Researcher, Chennai, India.
Learning switched non-linear dynamical systems from a single trajectory
Jul 26, 2026Sunny G. W. Wang, Hemant TyagiRegime-Switching Dynamical Systems Nonlinear System Identification
Division of Mathematical Sciences, SPMS, NTU Singapore 637371
Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection
Jul 19, 2026Shantanu ThoratAI-Generated Text Detection Data Selection
College Station, Texas, US
Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model
Jun 26, 2026Andrea Montanari, Kangjie ZhouApproximate Message Passing Statistical Learning Theory
Department of Mathematics and Department of Statistics, Stanford University · Department of Statistics, Columbia University
Non-asymptotic estimates of the minimal risk in statistical learning
Jun 22, 2026Liming Wu, Sen YangStatistical Learning Theory Empirical Risk Minimization
Laboratoire de Mathématiques Blaise Pascal, CNRS-UMR 6620, Université Clermont Auvergne · IASM, Harbin Institute of Technology
On the Oracle Complexity of Interpolation-Based Gradient Descent
Jun 18, 2026Dongmin Lee, William Lu, Anuran MakurOracle Complexity Gradient Descent
Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA · Department of Computer Science and the Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA
Focused PU learning from imbalanced data
May 14, 2026Elias Zavitsanos, Georgios PaliourasClass-Imbalanced Learning Binary Classification
Institute of Informatics and Telecommunications, NCSR “Demokritos”, Patriarhou Gregoriou E and 27 Neapoleos St., Agia Paraskevi, 15310, Attica, Greece.
Learning U-Statistics with Active Inference
May 12, 2026Xiaoning Wang, Yuyang Huo, Liuhua Peng +1Active Inference U-Statistics
School of Statistics and Data Sciences, LPMC, KLM-DASR and LEBPS, Nankai University, Tianjin, China · School of Mathematics and Statistics, The University of Melbourne, Melbourne, Australia
Risk-Consistent Multiclass Learning from Random Label-Subset Membership Queries
May 8, 2026Jiaxu Su, Junpeng Li, Changchun Hua +1Multiclass Classification Weakly Supervised Learning
Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, China
Elite-Driven Support Vector Machines for Classification
Apr 28, 2026Mohammad Jafari Jozani, Bahram MoeinianfarBinary Classification Support Vector Machines
Department of Statistics, University of Manitoba, Winnipeg, Manitoba, CANADA, R3T2N2
Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms
Apr 22, 2026Yaiza Bermudez, Samir M. Perlaza, Iñaki EsnaolaDecentralized Learning Statistical Learning Theory
Centre Inria d'Université Côte d'Azur, INRIA, Sophia Antipolis, France. · Laboratoire GAATI, Université de la Polynésie française, Fa'a'ā, French Polynesia. · ECE Dept. Princeton University, Princeton, 08544 NJ, USA. +1
Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization
Apr 3, 2026Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko +1Empirical Risk Minimization High-Dimensional Asymptotics
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · Huawei Noah’s Ark Lab, Huawei Technologies, Paris, France · School of Electronic Information and Communications, Huazhong University of Science & Technology, China
From Cross-Validation to SURE: Asymptotic Risk of Tuned Regularized Estimators
Mar 20, 2026Karun Adusumilli, Maximilian Kasy, Ashia WilsonCross-Validation Shrinkage Estimation
Department of Economics, University of Pennsylvania. · Department of Economics, University of Oxford. · Department of Electrical Engineering and Computer Science, MIT.
Self-Regularized Learning Methods
Mar 17, 2026Max Schölpple, Liu Fanghui, Ingo SteinwartStatistical Learning Theory Implicit Regularization
Institute for Stochastics and Applications, University of Stuttgart · School of Mathematical Sciences, Institute of Natural Sciences and MOE-LSC, Shanghai Jiao Tong University
Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
Mar 16, 2026Hideaki IidukaNonconvex Stochastic Optimization Muon Optimizer
Department of Computer Science Meiji University 1-1-1 Higashimita, Tama-ku, Kawasaki-shi, Kanagawa 214-8571 Japan

Nov 1, 2026J/K move · Enter open · S save

Benign population landscapes do not transfer to empirical risk minimization in nonsmooth optimization

Penghao Wang, Mateo Díaz, Liwei Jiang

Edwardson School of Industrial Engineering, Purdue University. · Department of Statistics & Data Science Institute, University of Chicago.

Nonconvex Optimization Empirical Risk Minimization

Open paper AskPDF arXiv

Benign landscapes provide a geometric explanation for the success of iterative methods on nonconvex problems arising in data science. The landscape is benign if every local minimizer is global and every other critical point is a strict saddle. For methods that avoid strict saddles, convergence to a critical point therefore implies convergence to a global minimizer. For smooth stochastic optimization, a common strategy for establishing such landscapes proceeds in two steps:

(i)

establish benign geometry for the population objective---which is often easier to analyze---and

(ii)

transfer this geometry to the empirical objective using concentration of gradients and Hessians. Recent work has extended saddle-point avoidance guarantees to weakly convex nonsmooth optimization through the notion of active strict saddles. We show, however, that benign population geometry need not transfer to empirical objectives in this setting. Specifically, for robust phase retrieval with Gaussian measurements---a well-studied and highly structured problem---the population objective is benign. Nevertheless, the empirical objective has sharp spurious local minima with probability bounded away from zero as the sample size tends to infinity. This asymptotic probability lower bound is independent of the dimension.