Empirical Risk Minimization

Also known as ERM

Ask about this topic

Overview

Empirical risk minimization chooses a model or decision rule to minimize average loss on a training sample, providing a foundation for regression, classification, and many neural-network objectives. Statistical analyses use localized Rademacher complexity, concentration and small-ball methods, and high-dimensional tools such as Gaussian comparison and approximate message passing to bound test error and study when structure, weighting, or constraints help. Optimization work examines gradient methods, regularization, distributed primal-dual algorithms, and privacy-preserving variants, while extensions address biased or dependent data, distribution shift, heavy tails, and selective labels. A central tension is that low empirical risk or faster fitting need not ensure reliable test performance: guarantees depend on model complexity and data conditions, while computational, privacy, and robustness constraints introduce further tradeoffs.

Latest papers

CardsList

45 papers5 papers in the last four weeks, down 29% on the four weeks before. 0.0% of all new papers.

  1. Interleaved Resampling and Refitting: Data and Compute-Efficient Evaluation of Black-Box Predictors

    Mar 15, 2026Haichen Hu, David Simchi-LeviEmpirical Risk Minimization

    Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA 02139, USA

  2. On the Statistical Optimality of Optimal Decision Trees

    Mar 5, 2026Zineng Xu, Subhro Ghosh, Yan Shuo TanDecision Tree LearningStatistical Learning Theory

    Department of Statistics and Data Science, National University of Singapore · Department of Mathematics, National University of Singapore

  3. A Researcher's Guide to Empirical Risk Minimization

    Feb 25, 2026Lars van der LaanEmpirical Risk MinimizationStatistical Learning Theory

    Department of Statistics, University of Washington

  4. Why ReLU? A Bit-Model Dichotomy for Deep Network Training

    Feb 22, 2026Ilan Doron-Arad, Elchanan MosselEmpirical Risk MinimizationReLU Neural Networks

    MIT, Cambridge, MA, USA

  5. On the Generalization and Robustness in Conditional Value-at-Risk

    Feb 20, 2026Dinesh Karthik Mulumudi, Piyushi Manupriya, Gholamali Aminian +1Conditional Value-at-RiskEmpirical Risk Minimization

    Dept. of Mathematics, IISER, Pune. · Dept. of Computer Science and Automation, Indian Institute of Sciecne, Bangalore. · The Alan Turing Institute, Greater London, England, UK.

  6. Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval

    Feb 19, 2026Antoine Maillard, Tony Bonnaire, Giulio BiroliLoss LandscapeLoss Landscape Geometry

    INRIA Paris & DI ENS, PSL University, Paris, France. · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France

  7. Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks

    Feb 18, 2026Binchuan QiNeural Network GeneralizationNeural Network Optimization

    Tongji University, Siping Street, 200092 Shanghai, China · Zhejiang Yuying College of Vocational Technology, Sihao Street, Hangzhou, 310018, Zhejiang, China

  8. Fast Rates for Nonstationary Weighted Risk Minimization

    Feb 5, 2026Tobias Brock, Thomas NaglerEmpirical Risk MinimizationStatistical Learning Theory

    LMU Munich Munich Center for Machine Learning (MCML)

  9. On the Gradient Complexity of Private Optimization with Private Oracles

    Nov 17, 2025Michael Menart, Aleksandar NikolovQuery ComplexityOracle Complexity

    Department of Computer Science, University of Toronto, Vector Institute · Department of Computer Science, University of Toronto

  10. Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Method

    Oct 27, 2025Diego MarcondesPhysics-Informed MLStatistical Learning Theory

  11. LLM Priors for ERM over Programs

    Oct 16, 2025Shivam Singhal, Priyadarsi Mishra, Eran Malach +1LLM-Based Program SynthesisEmpirical Risk Minimization

    Department of Computer Science & Engineering, Texas A&M University, TX, USA · Kempner Institute, Harvard University, MA, USA

  12. On the Relationship Between CoCoA and ADMM for Distributed Empirical Risk Minimization

    Feb 1, 2025Runxiong Wu, Andi WangPrimal-Dual OptimizationDistributed Optimization

    Department of Industrial & Systems Engineering University of Wisconsin–Madison

  13. A Differentially Private Weighted Empirical Risk Minimization Procedure and its Application to Outcome Weighted Learning

    Jul 24, 2023Spencer Giddens, Yiwang Zhou, Kevin R. Krull +3Privacy-Preserving MLDifferential Privacy

    University of Notre Dame, Notre Dame, IN 46556, USA · St. Jude Children’s Research Hospital, Memphis, TN 38138, USA · University of Michigan, Ann Arbor, MI 48109, USA

  14. Learning from a Biased Sample

    Sep 5, 2022Roshni Sahoo, Lihua Lei, Stefan WagerDistribution Shift RobustnessDistributionally Robust Optimization

    Stanford University

  15. Universality of empirical risk minimization

    Feb 17, 2022Andrea Montanari, Basil SaeedStatistical Learning TheoryEmpirical Risk Minimization

    Department of Statistics and Department of Mathematics, Stanford University · Department of Electrical Engineering, Stanford University