Statistical Learning Theory

Ask about this topic

Overview

Statistical learning theory asks when a learner can infer rules that work beyond its training data, and quantifies sample complexity, prediction error, and limits on what can be learned. Classical tools include VC and Littlestone dimensions, covering numbers, Rademacher complexity, stability, PAC-Bayes bounds, and minimax analysis; these provide guarantees for empirical risk minimization, online learning, kernels, and robust or private learning. Current work also studies high-dimensional and overparameterized regression, including interpolation and double descent, neural networks and Transformers, and learning under distribution shift, dependence, partial labels, or adaptive data. An enduring tension is that statistically optimal rates may be computationally unattainable, while guarantees for modern architectures often rely on idealized assumptions and may not explain practical optimization.

Latest papers

CardsList

584 papers79 papers in the last four weeks, up 58% on the four weeks before. 0.5% of all new papers.

  1. Provable Affine Identifiability of Nonlinear CCA under Latent Distributional Priors

    Oct 6, 2025Zhiwei Han, Stefan Matthes, Hao ShenRepresentation LearningParameter Identifiability

    fortiss GmbH, Munich, Germany · Technical university of Munich

  2. Mixture of Many Zero-Compute Experts: A High-Rate Quantization Theory Perspective

    Oct 3, 2025Yehuda DarMixture-of-Experts ModelsStatistical Learning Theory

    Faculty of Computer and Information Science, Ben-Gurion University

  3. Learning Multi-Index Models with Hyper-Kernel Ridge Regression

    Oct 2, 2025Shuo Huang, Hippolyte Labarrière, Ernesto De Vito +2Kernel Ridge RegressionKernel Regression

    Istituto Italiano di Tecnologia, Genoa, Italy · MaLGa - DIBRIS - Universit`a di Genova, Genoa, Italy · MaLGa - DIMA - Universit`a di Genova, Genoa, Italy +2

  4. Test time training enhances in-context learning of nonlinear functions

    Sep 30, 2025Kento Kuwataka, Taiji SuzukiSingle-Index ModelsTest-Time Training

    Department of Mathematical Engineering and Information Physics The University of Tokyo Hongo 7-3-1, Bunkyo-ku, Tokyo

  5. Linear Regression under Missing or Corrupted Coordinates

    Sep 23, 2025Ilias Diakonikolas, Jelena Diakonikolas, Daniel M. Kane +2Robust RegressionLearning with Missing Data

    University of Wisconsin-Madison · University of California, San Diego · University of California, Davis

  6. Effects of Structural Allocation of Geometric Task Diversity in Linear Meta-Learning Models

    Sep 22, 2025Saptati Datta, Nicolas W. Hengartner, Yulia Pimonova +2Meta-LearningFew-Shot Learning

    Department of Statistics, Texas A&M University, TX, USA · Los Alamos National Laboratory, Los Alamos, USA.

  7. A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learning

    Sep 14, 2025Jia-Qi Yang, Lei ShiStochastic ApproximationReproducing Kernel Hilbert Spaces

    School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.

  8. The Sample Complexity of Membership Inference and Privacy Auditing

    Aug 26, 2025Mahdi Haghifam, Adam Smith, Jonathan UllmanMembership Inference AttacksSample Complexity

    Khoury College of Computer Sciences, Northeastern University. · Department of Computer Science, Boston University.

  9. The Statistical Fairness-Accuracy Frontier

    Aug 25, 2025Alireza Fallah, Michael I. Jordan, Annie UlichneyAlgorithmic FairnessGroup Fairness

    Department of Computer Science and Ken Kennedy Institute, Rice University · Departments of Electrical Engineering and Computer Sciences and Statistics, University of California, Berkeley; Inria Paris · Department of Statistics, University of California, Berkeley

  10. PAC-Causal Learnability: From Identification to Finite-Sample Guarantees

    Jul 25, 2025Mian Wei, Somesh Jha, David PageCausal Effect EstimationCausal Identifiability

    Duke University · University of Wisconsin-Madison

  11. A statistical physics framework for optimal learning

    Jul 10, 2025Francesca Mignacco, Francesco MoriStatistical Physics of LearningNeural Network Training Dynamics

    1Joseph Henry Laboratories of Physics, Princeton University, Princeton, NJ 08544, USA · 2Graduate Center, City University of New York, New York, NY 10016, USA · Center of Mathematical Sciences and Applications, Harvard University, Cambridge, MA 02138, USA

  12. Adaptively Truncated Signature-based Logistic Regression for Semi-parametric Functional Classification

    Jul 9, 2025Pengcheng Zeng, Siyuan JiangStatistical Learning Theory

    Institute of Mathematical Sciences, ShanghaiTech University, Shanghai, China

  13. On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification

    Jun 14, 2025Matteo Vilucchio, Lenka Zdeborová, Bruno LoureiroAdversarial AttacksAdversarial Robustness

    Information Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Département d’Informatique, École Normale Supérieure - PSL & CNRS, France

  14. No Free Lunch: Non-Asymptotic Analysis of Prediction-Powered Inference

    May 26, 2025Pranav Mani, Peng Xu, Zachary C. Lipton +1Adaptive InferenceStatistical Learning Theory

    Abridge AI, San Francisco, CA, USA · Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

  15. Representation Learning for Equivariant Inference with Guarantees

    May 26, 2025Daniel Ordoñez-Apraez, Vladimir Kostić, Alek Fröhlich +3Disentangled Representation LearningRepresentation Learning

    CSML, Italian Institute of Technology, Genova, Italy · Universit`a di Genova, Genova, Italy · University of Novi Sad, Novi Sad, Serbia +2

  16. Accuracy vs. Accuracy: Computational Tradeoffs Between Classification Rates and Utility

    May 22, 2025Noga Amit, Omer Reingold, Guy N. RothblumAlgorithmic FairnessStatistical Learning Theory

    UC Berkeley · Stanford · Apple

  17. When majority rules, minority loses: bias amplification of gradient descent

    May 19, 2025François Bachoc, Jérôme Bolte, Ryan Boustany +1Class-Imbalanced LearningGradient Descent

    LPP · University of Lille · Institut Universitaire de France (IUF) +6

  18. The Dynamics of Generalization in Deep Learning

    Apr 23, 2025Rubing Yang, Pratik ChaudhariNeural Network GeneralizationNeural Network Training Dynamics

    Department of Applied Mathematics and Computational Sciences, University of Pennsylvania Philadelphia, PA, 19104, USA · Department of Electrical and Systems Engineering, University of Pennsylvania Philadelphia, PA, 19104, USA

  19. Learning in Structured Stackelberg Games

    Apr 11, 2025Maria-Florina Balcan, Kiriaki Fragkia, Keegan HarrisGame TheoryStackelberg Games

    Carnegie Mellon University, Pittsburgh, USA · University of California, Berkeley, USA

  20. Near-Polynomially Competitive Active Logistic Regression

    Mar 7, 2025Yihan Zhou, Eric Price, Trung NguyenActive LearningLogistic Regression

    The University of Texas at Austin

  21. Dissecting the Impact of Model Misspecification in Data-driven Optimization

    Mar 1, 2025Adam N. Elmachtoub, Henry Lam, Haixiang Lan +1Model MisspecificationDecision-Focused Learning

    Columbia University · Morgan Stanley

  22. Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework

    Feb 25, 2025Hengzhi He, Shirong Xu, Guang ChengModel CollapseGenerative Modeling

    Department of Statistics and Data Science, University of California, Los Angeles · Wang Yanan Institute for Studies in Economics, Xiamen University

  23. Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling

    Feb 21, 2025Yufan Li, Pragya SurProbability CalibrationBinary Classification

    Department of Statistics Harvard University Cambrdige, MA 02138

  24. Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer

    Jan 18, 2025Haotian Lin, Matthew ReimherrSpectral RegularizationConcept Drift

    Department of Statistics, The Pennsylvania State University.

  25. The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithms

    Nov 18, 2024Samir M. Perlaza, Xinying ZouInformation-Theoretic Generalization BoundsStatistical Learning Theory

    INRIA, Centre Inria d’Universit´e Cˆote d’Azur, 06902 Sophia Antipolis, France

  26. Discrete distributions are learnable from metastable samples

    Oct 17, 2024Abhijith Jayakumar, Andrey Y. Lokhov, Sidhant Misra +1Conditional Distribution EstimationStatistical Learning Theory

    Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA

  27. Adaptive and oblivious statistical adversaries are equivalent

    Oct 17, 2024Guy Blanc, Gregory ValiantStatistical Learning Theory

    Stanford

  28. On Generalisation Error Bounds for Transformers

    Oct 15, 2024Lan V. TruongTransformerStatistical Learning Theory

    Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), Vietnam

  29. Which Spaces can be Embedded in LpL_p-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy

    Oct 14, 2024Yiping Lu, Daozhe Lin, Qiang DuReproducing Kernel Hilbert SpacesKernel Methods

    Industrial Engineering & Management Sciences, Northwestern University · Department of Applied Physics & Applied Mathematics, Columbia University

  30. Statistical Properties of Deep Neural Networks with Dependent Data

    Oct 14, 2024Chad BrownNonparametric RegressionNeural Network Approximation Theory

    Department of Economics, University of Colorado Boulder, 80309, USA.