Statistical Learning Theory

Overview

Statistical learning theory asks when a learner can infer rules that work beyond its training data, and quantifies sample complexity, prediction error, and limits on what can be learned. Classical tools include VC and Littlestone dimensions, covering numbers, Rademacher complexity, stability, PAC-Bayes bounds, and minimax analysis; these provide guarantees for empirical risk minimization, online learning, kernels, and robust or private learning. Current work also studies high-dimensional and overparameterized regression, including interpolation and double descent, neural networks and Transformers, and learning under distribution shift, dependence, partial labels, or adaptive data. An enduring tension is that statistically optimal rates may be computationally unattainable, while guarantees for modern architectures often rely on idealized assumptions and may not explain practical optimization.

Latest papers

CardsList

640 papers57 papers in the last four weeks, down 31% on the four weeks before. 0.4% of all new papers.

  1. Estimation of multiple mean vectors in high dimension

    Mar 22, 2024Gilles Blanchard, Jean-Baptiste Fermanian, Hannah MarienwaldKernel Mean EmbeddingsStatistical Learning Theory

  2. A Probabilistic Approach for Model Alignment with Human Comparisons

    Mar 16, 2024Junyu Cao, Mohsen BayatiHuman Preference ModelingStatistical Learning Theory

  3. A Differentially Private Weighted Empirical Risk Minimization Procedure and its Application to Outcome Weighted Learning

    Jul 24, 2023Spencer Giddens, Yiwang Zhou, Kevin R. Krull +3Privacy-Preserving MLDifferential Privacy

  4. Reliable learning in challenging environments

    Apr 6, 2023Maria-Florina Balcan, Steve Hanneke, Rattana Pukdee +1Distribution Shift RobustnessAdversarial Robustness

  5. How many labelers do you have? A closer look at gold-standard labels

    Jun 24, 2022Chen Cheng, Hilal Asi, John DuchiModel CalibrationNoisy-Label Learning

  6. Learning Non-Vacuous Generalization Bounds from Optimization

    Jun 9, 2022Chengli Tan, Jiangshe Zhang, Junmin Liu +1Neural Network GeneralizationStatistical Learning Theory

  7. Universality of empirical risk minimization

    Feb 17, 2022Andrea Montanari, Basil SaeedStatistical Learning TheoryEmpirical Risk Minimization

  8. Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

    Jan 6, 2022Saptarshi Chakraborty, Debolina Paul, Swagatam DasRobust RegressionModel Misspecification

  9. A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning

    Sep 6, 2021Yehuda Dar, Vidya Muthukumar, Richard G. BaraniukDouble DescentNeural Network Generalization