Statistical Learning Theory

Overview

Statistical learning theory asks when a learner can infer rules that work beyond its training data, and quantifies sample complexity, prediction error, and limits on what can be learned. Classical tools include VC and Littlestone dimensions, covering numbers, Rademacher complexity, stability, PAC-Bayes bounds, and minimax analysis; these provide guarantees for empirical risk minimization, online learning, kernels, and robust or private learning. Current work also studies high-dimensional and overparameterized regression, including interpolation and double descent, neural networks and Transformers, and learning under distribution shift, dependence, partial labels, or adaptive data. An enduring tension is that statistically optimal rates may be computationally unattainable, while guarantees for modern architectures often rely on idealized assumptions and may not explain practical optimization.

Latest papers

CardsList

640 papers57 papers in the last four weeks, down 31% on the four weeks before. 0.4% of all new papers.

  1. The Statistical Fairness-Accuracy Frontier

    Aug 25, 2025Alireza Fallah, Michael I. Jordan, Annie UlichneyAlgorithmic FairnessGroup Fairness

  2. PAC-Causal Learnability: From Identification to Finite-Sample Guarantees

    Jul 25, 2025Mian Wei, Somesh Jha, David PageCausal Effect EstimationCausal Identifiability

  3. A statistical physics framework for optimal learning

    Jul 10, 2025Francesca Mignacco, Francesco MoriStatistical Physics of LearningNeural Network Training Dynamics

  4. Adaptively Truncated Signature-based Logistic Regression for Semi-parametric Functional Classification

    Jul 9, 2025Pengcheng Zeng, Siyuan JiangStatistical Learning Theory

  5. Understanding Randomization in Greedy Model Search

    Jun 18, 2025Xin Chen, Jason M. Klusowski, Yan Shuo Tan +1Feature SelectionEnsemble Learning

  6. On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification

    Jun 14, 2025Matteo Vilucchio, Lenka Zdeborová, Bruno LoureiroAdversarial AttacksAdversarial Robustness

  7. No Free Lunch: Non-Asymptotic Analysis of Prediction-Powered Inference

    May 26, 2025Pranav Mani, Peng Xu, Zachary C. Lipton +1Adaptive InferenceStatistical Learning Theory

  8. Representation Learning for Equivariant Inference with Guarantees

    May 26, 2025Daniel Ordoñez-Apraez, Vladimir Kostić, Alek Fröhlich +3Disentangled Representation LearningRepresentation Learning

  9. When majority rules, minority loses: bias amplification of gradient descent

    May 19, 2025François Bachoc, Jérôme Bolte, Ryan Boustany +1Class-Imbalanced LearningGradient Descent

  10. The Dynamics of Generalization in Deep Learning

    Apr 23, 2025Rubing Yang, Pratik ChaudhariNeural Network GeneralizationNeural Network Training Dynamics

  11. Learning in Structured Stackelberg Games

    Apr 11, 2025Maria-Florina Balcan, Kiriaki Fragkia, Keegan HarrisGame TheoryStackelberg Games

  12. Langevin Monte-Carlo Provably Learns Depth Two Neural Nets at Any Size and Data

    Mar 13, 2025Dibyakanti Kumar, Samyak Jha, Anirbit MukherjeeTwo-Layer Neural NetworksStatistical Learning Theory

  13. Near-Polynomially Competitive Active Logistic Regression

    Mar 7, 2025Yihan Zhou, Eric Price, Trung NguyenActive LearningLogistic Regression

  14. Dissecting the Impact of Model Misspecification in Data-driven Optimization

    Mar 1, 2025Adam N. Elmachtoub, Henry Lam, Haixiang Lan +1Model MisspecificationDecision-Focused Learning

  15. Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework

    Feb 25, 2025Hengzhi He, Shirong Xu, Guang ChengModel CollapseGenerative Modeling

  16. Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer

    Jan 18, 2025Haotian Lin, Matthew ReimherrSpectral RegularizationConcept Drift

  17. The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithms

    Nov 18, 2024Samir M. Perlaza, Xinying ZouInformation-Theoretic Generalization BoundsStatistical Learning Theory

  18. Discrete distributions are learnable from metastable samples

    Oct 17, 2024Abhijith Jayakumar, Andrey Y. Lokhov, Sidhant Misra +1Conditional Distribution EstimationStatistical Learning Theory

  19. On Generalisation Error Bounds for Transformers

    Oct 15, 2024Lan V. TruongTransformerStatistical Learning Theory

  20. Which Spaces can be Embedded in LpL_p-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy

    Oct 14, 2024Yiping Lu, Daozhe Lin, Qiang DuReproducing Kernel Hilbert SpacesKernel Methods

  21. Statistical Properties of Deep Neural Networks with Dependent Data

    Oct 14, 2024Chad BrownNonparametric RegressionNeural Network Approximation Theory

  22. The EM-algorithm and the Method of Moments in Softmax Mixture Models

    Sep 16, 2024Xin Bing, Florentina Bunea, Jonathan Niles-Weed +1Expectation-MaximizationDiscrete Choice Modeling

  23. Adversarial dynamical systems characterize when data-driven learning succeeds or fails

    Jul 8, 2024Matthew J. Colbrook, Igor Mezić, Alexei StepanenkoKoopman Operator LearningSpectral Methods

  24. Smoothed Analysis for Learning Concepts with Low Intrinsic Dimension

    Jul 1, 2024Gautam Chandrasekaran, Adam Klivans, Vasilis Kontonis +2Agnostic LearningMargin-Based Learning

  25. Generalization error of min-norm interpolators in transfer learning

    Jun 20, 2024Yanke Song, Kenneth Gu, Sohom Bhattacharya +1OOD GeneralizationCross-Domain Transfer Learning

  26. Gaussian-Channel Regularity and Distribution Learning with Flow Matching

    Mar 31, 2024Yuan Gao, Jian Huang, Yuling Jiao +1Flow-Based Generative ModelingStatistical Learning Theory