Statistical Learning Theory

Ask about this topic

Overview

Statistical learning theory asks when a learner can infer rules that work beyond its training data, and quantifies sample complexity, prediction error, and limits on what can be learned. Classical tools include VC and Littlestone dimensions, covering numbers, Rademacher complexity, stability, PAC-Bayes bounds, and minimax analysis; these provide guarantees for empirical risk minimization, online learning, kernels, and robust or private learning. Current work also studies high-dimensional and overparameterized regression, including interpolation and double descent, neural networks and Transformers, and learning under distribution shift, dependence, partial labels, or adaptive data. An enduring tension is that statistically optimal rates may be computationally unattainable, while guarantees for modern architectures often rely on idealized assumptions and may not explain practical optimization.

Latest papers

CardsList

584 papers79 papers in the last four weeks, up 58% on the four weeks before. 0.5% of all new papers.

  1. On the Statistical Optimality of Optimal Decision Trees

    Mar 5, 2026Zineng Xu, Subhro Ghosh, Yan Shuo TanDecision Tree LearningStatistical Learning Theory

    Department of Statistics and Data Science, National University of Singapore · Department of Mathematics, National University of Singapore

  2. Bayes with No Shame: Admissibility Geometries of Predictive Inference

    Mar 5, 2026Nicholas G. Polson, Daniel ZantedeschiBayesian Decision TheoryAnytime-Valid Inference

    Booth School of Business, University of Chicago, Chicago, IL 60637 · Muma College of Business, University of South Florida, Tampa, FL 33620

  3. Asymptotic Behavior of Multi--Task Learning: Implicit Regularization and Double Descent Effects

    Mar 5, 2026Ayed M. Alrashdi, Oussama Dhifallah, Houssem SifaouDouble DescentNeural Network Generalization

    Department of Electrical Engineering, College of Engineering, University of Ha’il, P.O. Box 2440, Ha’il, 81441, Saudi Arabia · John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA · Department of Electrical and Electronic Engineering, King’s College London – Strand, London WC2R 2LS – London, UK

  4. Oracle-efficient Hybrid Learning with Constrained Adversaries

    Mar 4, 2026Princewill Okoroafor, Robert Kleinberg, Michael P. KimOnline LearningStatistical Learning Theory

    Cornell University

  5. Beyond Cross-Validation: Adaptive Parameter Selection for Kernel-Based Gradient Descents

    Mar 3, 2026Xiaotong Liu, Yunwen Lei, Xiangyu Chang +1Kernel MethodsHyperparameter Optimization

    Center for Intelligent Decision-Making and Machine Learning, School of Management, Xi’an Jiaotong University, Xi’an, China · Department of Mathematics, The University of Hong Kong, Hongkong, China

  6. Multiplicative Oracle Inequalities for Transductive Learning via Level-Set Aggregation

    Mar 2, 2026Jian Qian, Jiachen XuTransductive LearningStatistical Learning Theory

    The University of Hong Kong · Hong Kong University of Science and Technology

  7. Relatively Smart: A New Approach for Instance-Optimal Learning

    Mar 2, 2026Shaddin Dughmi, Alireza F. PourPAC LearningSemi-Supervised Learning

    University of Southern California · University of Waterloo

  8. A short tour of operator learning theory: Convergence rates, statistical limits, and open questions

    Feb 28, 2026Simone Brugiapaglia, Nicola Rares Franco, Nicholas H. NelsenOperator LearningStatistical Learning Theory

    Concordia University · Politecnico di Milano · Cornell University and The University of Texas at Austin

  9. Physics-Aware Learnability: From Set-Theoretic Independence to Operational Constraints

    Feb 28, 2026Jeongho Bang, Kyoungho ChoStatistical Learning TheoryPAC Learning

    Institute for Convergence Research and Education in Advanced Technology, Yonsei University, Seoul 03722, Republic of Korea · Department of Quantum Information, Yonsei University, Incheon 21983, Republic of Korea · Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of Korea

  10. Active Bipartite Ranking with Smooth Posterior Distributions

    Feb 27, 2026James Cheshire, Stephan ClémençonLearning to RankStatistical Learning Theory

    Telecom Paris

  11. A Theory of Random Graph Shift in Truncated-Spectrum vRKHS

    Feb 27, 2026Zhang Wan, Tingting Mu, Samuel KaskiGraph Domain AdaptationDistribution Shift Robustness

    Department of Computer Science, University of Manchester, UK · Department of Computer Science, Aalto University, Finland · ELLIS Institute Finland

  12. Provable Subspace Identification of Nonlinear Multi-view CCA

    Feb 27, 2026Zhiwei Han, Stefan Matthes, Hao ShenRepresentation IdentifiabilityParameter Identifiability

    Chair of Data Processing, Technical University of Munich, Germany · fortiss GmbH, Munich, Germany

  13. Model Agreement via Anchoring

    Feb 26, 2026Eric Eaton, Surbhi Goel, Marcel Hussing +4Statistical Learning TheoryEnsemble Learning

    Department of Computer and Information Sciences, University of Pennsylvania · Department of Computer Science, The Johns Hopkins University

  14. Low-degree Lower bounds for clustering in moderate dimension

    Feb 26, 2026Alexandra Carpentier, Nicolas VerzelenLow-Degree Polynomial MethodClustering

    Institut für Mathematik, Universität Potsdam, Germany. · INRAE, MISTEA, Univ. Montpellier, Montpellier, France

  15. Flow Matching is Adaptive to Manifold Structures

    Feb 25, 2026Shivam Kumar, Yixin Wang, Lizhen LinFlow MatchingGenerative Modeling

    Booth School of Business, University of Chicago · Department of Statistics, University of Michigan · Department of Mathematics, University of Maryland, College Park

  16. LoBoost: Fast Model-Native Local Conformal Prediction for Gradient-Boosted Trees

    Feb 25, 2026Vagner Santos, Victor Coscrato, Luben Cabezas +2Conformal PredictionRegression Prediction Intervals

    Department of Statistics, Federal University of São Carlos, São Carlos, São Paulo, Brazil · Institute of Mathematics and Computer Science, University of São Paulo, São Carlos, São Paulo, Brazil · Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France

  17. A Researcher's Guide to Empirical Risk Minimization

    Feb 25, 2026Lars van der LaanEmpirical Risk MinimizationStatistical Learning Theory

    Department of Statistics, University of Washington

  18. Statistical Query Lower Bounds for Smoothed Agnostic Learning

    Feb 24, 2026Ilias Diakonikolas, Daniel M. KaneHalfspace LearningStatistical Learning Theory

    University of Wisconsin-Madison · University of California, San Diego

  19. Is Multi-Distribution Learning as Easy as PAC Learning: Sharp Rates with Bounded Label Noise

    Feb 24, 2026Rafael Hanashiro, Abhishek Shetty, Patrick JailletStatistical Learning TheoryLearning with Noisy Labels

    MIT

  20. Does Order Matter : Connecting The Law of Robustness to Robust Generalization

    Feb 24, 2026Mihir More, Aritra Das, Jaee Ponde +3Generalization BoundsStatistical Learning Theory

    Ashoka University, Sonepat, Haryana · Truth Audit Labs · Indian Statistical Institute, Kolkata

  21. On the Generalization Behavior of Deep Residual Networks From a Dynamical System Perspective

    Feb 24, 2026Jinshu Huang, Mingfei Sun, Chunlin WuNeural Network GeneralizationGeneralization Bounds

    School of Mathematical Sciences, Nankai University, Tianjin, China

  22. Characterizing Online and Private Learnability under Distributional Constraints via Generalized Smoothness

    Feb 24, 2026Moïse Blanchard, Abhishek Shetty, Alexander RakhlinStatistical Learning TheoryOnline Learning

    MIT

  23. Standard Transformers Achieve the Minimax Rate in Nonparametric Regression with Cs,λC^{s,λ} Targets

    Feb 24, 2026Yanming Lai, Defeng SunTransformer ExpressivityNeural Network Approximation Theory

    Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China

  24. Reliable Abstention under Adversarial Injections: Tight Lower Bounds and New Upper Bounds

    Feb 23, 2026Ezra Edelman, Surbhi GoelStatistical Learning TheoryAdversarial Robustness

    University of Pennsylvania

  25. A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning

    Feb 23, 2026Nicolas Anguita, Francesco Locatello, Andrew M. Saxe +4Neural Network GeneralizationRepresentation Learning

    Department of Engineering, University of Cambridge · Institute of Science and Technology, Austria (ISTA) · Gatsby Computational Neuroscience Unit and Sainsbury Wellcome Centre, UCL +1

  26. Smoothness Adaptivity in Constant-Depth Neural Networks: Optimal Rates via Smooth Activations

    Feb 23, 2026Yuhao Liu, Zilin Wang, Lei Wu +1Neural Network Approximation TheoryNeural Network Activation Functions

    Department of Mathematical Sciences, Tsinghua University · School of Mathematical Sciences, Peking University · Center for Machine Learning Research, Peking University +1

  27. Online Realizable Regression and Applications for ReLU Networks

    Feb 22, 2026Ilan Doron-Arad, Idan Mehalel, Elchanan MosselStatistical Learning TheoryReLU Neural Networks

    Massachusetts Institute of Technology, Cambridge, MA, USA · The Hebrew University of Jerusalem, Jerusalem, Israel

  28. (α,β)(α,β)-Stability for Boosting Vector-Valued Prediction

    Feb 21, 2026Jian Qian, Shu GeEnsemble LearningStatistical Learning Theory

    The University of Hong Kong · Independent Researcher

  29. On the Generalization and Robustness in Conditional Value-at-Risk

    Feb 20, 2026Dinesh Karthik Mulumudi, Piyushi Manupriya, Gholamali Aminian +1Conditional Value-at-RiskEmpirical Risk Minimization

    Dept. of Mathematics, IISER, Pune. · Dept. of Computer Science and Automation, Indian Institute of Sciecne, Bangalore. · The Alan Turing Institute, Greater London, England, UK.

  30. Distribution-Free Sequential Prediction with Abstentions

    Feb 20, 2026Jialin Yu, Moïse BlanchardStatistical Learning TheorySelective Prediction

    Georgia Institute of Technology