Statistical Learning Theory

Ask about this topic

Overview

Statistical learning theory asks when a learner can infer rules that work beyond its training data, and quantifies sample complexity, prediction error, and limits on what can be learned. Classical tools include VC and Littlestone dimensions, covering numbers, Rademacher complexity, stability, PAC-Bayes bounds, and minimax analysis; these provide guarantees for empirical risk minimization, online learning, kernels, and robust or private learning. Current work also studies high-dimensional and overparameterized regression, including interpolation and double descent, neural networks and Transformers, and learning under distribution shift, dependence, partial labels, or adaptive data. An enduring tension is that statistically optimal rates may be computationally unattainable, while guarantees for modern architectures often rely on idealized assumptions and may not explain practical optimization.

Latest papers

CardsList

584 papers79 papers in the last four weeks, up 58% on the four weeks before. 0.5% of all new papers.

  1. The EM-algorithm and the Method of Moments in Softmax Mixture Models

    Sep 16, 2024Xin Bing, Florentina Bunea, Jonathan Niles-Weed +1Expectation-MaximizationDiscrete Choice Modeling

    Department of Statistical Sciences, University of Toronto · Department of Statistics and Data Science, Cornell University · Courant Institute of Mathematical Sciences and Center for Data Science, New York University +1

  2. Adversarial dynamical systems characterize when data-driven learning succeeds or fails

    Jul 8, 2024Matthew J. Colbrook, Igor Mezić, Alexei StepanenkoKoopman Operator LearningSpectral Methods

    DAMTP, University of Cambridge, Cambridge, CB3 0WA, UK. · University of California, Santa Barbara, CA 93106, USA.

  3. Smoothed Analysis for Learning Concepts with Low Intrinsic Dimension

    Jul 1, 2024Gautam Chandrasekaran, Adam Klivans, Vasilis Kontonis +2Agnostic LearningMargin-Based Learning

    a University of Texas at Austin · b University of California, Los Angeles

  4. Generalization error of min-norm interpolators in transfer learning

    Jun 20, 2024Yanke Song, Kenneth Gu, Sohom Bhattacharya +1OOD GeneralizationCross-Domain Transfer Learning

    Department of Statistics, Harvard University · Department of Statistics, Stanford University · Department of Statistics, University of Florida

  5. Estimation of multiple mean vectors in high dimension

    Mar 22, 2024Gilles Blanchard, Jean-Baptiste Fermanian, Hannah MarienwaldKernel Mean EmbeddingsStatistical Learning Theory

    LMO, DATASHAPE · Université Paris Saclay, Institut Mathématique d’Orsay, France. · LMO +2

  6. A Probabilistic Approach for Model Alignment with Human Comparisons

    Mar 16, 2024Junyu Cao, Mohsen BayatiHuman Preference ModelingStatistical Learning Theory

    McCombs School of Business, The University of Texas at Austin · Graduate School of Business, Stanford University

  7. A Differentially Private Weighted Empirical Risk Minimization Procedure and its Application to Outcome Weighted Learning

    Jul 24, 2023Spencer Giddens, Yiwang Zhou, Kevin R. Krull +3Privacy-Preserving MLDifferential Privacy

    University of Notre Dame, Notre Dame, IN 46556, USA · St. Jude Children’s Research Hospital, Memphis, TN 38138, USA · University of Michigan, Ann Arbor, MI 48109, USA

  8. Reliable learning in challenging environments

    Apr 6, 2023Maria-Florina Balcan, Steve Hanneke, Rattana Pukdee +1Distribution Shift RobustnessAdversarial Robustness

    Carnegie Mellon University · Purdue University

  9. Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation

    Mar 15, 2023Diego Marcondes, Cláudia PeixotoModel SelectionCross-Validation

  10. How many labelers do you have? A closer look at gold-standard labels

    Jun 24, 2022Chen Cheng, Hilal Asi, John DuchiModel CalibrationNoisy-Label Learning

    Department of Statistics, Stanford University · Department of Electrical Engineering, Stanford University · Departments of Statistics and Electrical Engineering, Stanford University

  11. Learning Non-Vacuous Generalization Bounds from Optimization

    Jun 9, 2022Chengli Tan, Jiangshe Zhang, Junmin Liu +1Neural Network GeneralizationStatistical Learning Theory

    Northwestern Polytechnical University, Xi’an, China · Xi’an Jiaotong University, Xi’an, China · SGIT AI Lab, State Grid Corporation of China, Xi’an, China

  12. Universality of empirical risk minimization

    Feb 17, 2022Andrea Montanari, Basil SaeedStatistical Learning TheoryEmpirical Risk Minimization

    Department of Statistics and Department of Mathematics, Stanford University · Department of Electrical Engineering, Stanford University

  13. Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

    Jan 6, 2022Saptarshi Chakraborty, Debolina Paul, Swagatam DasRobust RegressionModel Misspecification

    Department of Statistics, University of California, Berkeley · Department of Statistics, Stanford University · Electronics and Communication Sciences Unit, Indian Statistical Institute, Kolkata, India

  14. A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning

    Sep 6, 2021Yehuda Dar, Vidya Muthukumar, Richard G. BaraniukDouble DescentNeural Network Generalization

    Faculty of Computer and Information Science, Ben-Gurion University · Schools of Electrical and Computer Engineering, and Industrial and Systems Engineering, Georgia Institute of Technology · Electrical and Computer Engineering Department, Rice University