Statistical Learning Theory

Ask about this topic

Overview

Statistical learning theory asks when a learner can infer rules that work beyond its training data, and quantifies sample complexity, prediction error, and limits on what can be learned. Classical tools include VC and Littlestone dimensions, covering numbers, Rademacher complexity, stability, PAC-Bayes bounds, and minimax analysis; these provide guarantees for empirical risk minimization, online learning, kernels, and robust or private learning. Current work also studies high-dimensional and overparameterized regression, including interpolation and double descent, neural networks and Transformers, and learning under distribution shift, dependence, partial labels, or adaptive data. An enduring tension is that statistically optimal rates may be computationally unattainable, while guarantees for modern architectures often rely on idealized assumptions and may not explain practical optimization.

Latest papers

CardsList

584 papers79 papers in the last four weeks, up 58% on the four weeks before. 0.5% of all new papers.

  1. Isomorphic Functionalities between Ant Colony and Ensemble Learning: Part II-On the Strength of Weak Learnability and the Boosting Paradigm

    Mar 25, 2026Ernest Fokoué, Gregory Babbitt, Yuval LeventalEnsemble LearningEmergent Behavior in Multi-Agent Systems

    School of Mathematics and Statistics, Rochester Institute of Technology · Gosnell School of Life Sciences, Rochester Institute of Technology · Center for Imaging Science, Rochester Institute of Technology

  2. General Machine Learning: Theory for Learning Under Variable Regimes

    Mar 24, 2026Aomar OsmaniStatistical Learning TheoryAdaptive Learning

    INSA-Rouen, France

  3. A One-Inclusion Graph Approach to Multi-Group Learning

    Mar 24, 2026Noah Bergam, Samuel Deng, Daniel HsuStatistical Learning Theory

    Columbia University

  4. Generalization Bounds for Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations

    Mar 24, 2026Sebastien Andre-Sloan, Dibyakanti Kumar, Alejandro F Frangi +1Generalization BoundsPhysics-Informed ML

    Department of Computer Science, The University of Manchester

  5. Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Data

    Mar 24, 2026Anand Jerry George, Nicolas MacrisIntrinsic DimensionalityRandom Feature Methods

    École Polytechnique Fédérale de Lausanne (EPFL), Lab for Statistical Mechanics of Inference in Large Systems (SMILS), CH-1015 Lausanne, Switzerland

  6. Transformers Trained via Gradient Descent Can Provably Learn a Class of Teacher Models

    Mar 24, 2026Chenyang Zhang, Qingyue Zhao, Quanquan Gu +1TransformerStatistical Learning Theory

    The University of Hong Kong · University of California, Los Angeles

  7. The No-Clash Teaching Dimension is Bounded by VC Dimension

    Mar 24, 2026Jiahua Liu, Benchong LiSample CompressionStatistical Learning Theory

    School of Mathematics and Statistics, Xidian University.

  8. Sharper Generalization Bounds for Transformer

    Mar 23, 2026Yawen Li, Tao Hu, Zhouhui Lian +4Generalization BoundsTransformer

    School of Mathematical Sciences, Capital Normal University, 100048 Beijing, China · Wangxuan Institute of Computer Technology, Peking University, China, 100871 · Advanced Institute of Information Technology, Peking University +4

  9. High-dimensional online learning via asynchronous decomposition: Non-divergent results, dynamic regularization, and beyond

    Mar 21, 2026Shixiang Liu, Zhifan Li, Hanming Yang +1Statistical Learning TheoryStreaming Algorithms

    School of Statistics, Renmin University of China · School of Statistics and Mathematics, Zhongnan University of Economics and Law · Institute of Statistics and Big Data, Renmin University of China +1

  10. Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity

    Mar 21, 2026Zixuan Zhang, Kaixuan Huang, Tuo Zhao +2Diffusion ModelsStatistical Learning Theory

    Georgia Tech · Princeton University · Northwestern University

  11. A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models

    Mar 18, 2026Leonardo Defilippis, Florent Krzakala, Bruno Loureiro +1Statistical Learning TheoryMulti-Index Models

    Departement d’Informatique, École Normale Supérieure, PSL & CNRS · Information, Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · INRIA Paris

  12. Consistency of the kk-Nearest Neighbor Regressor under Complex Survey Designs

    Mar 18, 2026Caren Haslerk-Nearest NeighborsNonparametric Regression

  13. Self-Regularized Learning Methods

    Mar 17, 2026Max Schölpple, Liu Fanghui, Ingo SteinwartStatistical Learning TheoryImplicit Regularization

    Institute for Stochastics and Applications, University of Stuttgart · School of Mathematical Sciences, Institute of Natural Sciences and MOE-LSC, Shanghai Jiao Tong University

  14. High-Dimensional Gaussian Mean Estimation under Realizable Contamination

    Mar 17, 2026Ilias Diakonikolas, Daniel M. Kane, Thanasis PittasData ContaminationStatistical Learning Theory

    University of Wisconsin-Madison · University of California, San Diego

  15. Learnability with Partial Labels and Adaptive Nearest Neighbors

    Mar 16, 2026Nicolas A. Errandonea, Santiago Mazuelas, Jose A. Lozano +1Statistical Learning Theoryk-Nearest Neighbors

    Basque Center for Applied Mathematics, Bilbao, Spain · Ikerbasque, Basque Foundation for Science · University of the Basque Country, Spain +1

  16. Analyzing Error Sources in Global Feature Effect Estimation

    Mar 16, 2026Timo Heiß, Coco Bögel, Bernd Bischl +1Statistical Learning Theory

    LMU Munich, Munich, Germany · Munich Center for Machine Learning (MCML)

  17. Dataset Distillation Efficiently Encodes Low-Dimensional Representations from Gradient-Based Learning of Non-Linear Tasks

    Mar 16, 2026Yuri Kinoshita, Naoki Nishikawa, Taro ToyoizumiTwo-Layer Neural NetworksDataset Distillation

    The University of Tokyo, Tokyo, Japan · Laboratory for Neural Computation and Adaptation, RIKEN Center for Brain Science, Wako, Japan · RIKEN Center for Advanced Intelligence Project, Nihonbashi, Japan

  18. Learning in Function Spaces: An Unified Functional Analytic View of Supervised and Unsupervised Learning

    Mar 15, 2026K. LakshmananSupervised LearningStatistical Learning Theory

    Department of Computer Science and Engineering, Indian Insititute of Technology (BHU), Varanasi 221005.

  19. Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis

    Mar 13, 2026Chen Feng, Zhuo Zhi, Zhao Huang +5Learning with Noisy LabelsStatistical Learning Theory

    Queen’s University Belfast · University College London · University of Aberdeen +3

  20. A theory of learning data statistics in diffusion models, from easy to hard

    Mar 13, 2026Lorenzo Bardone, Claudia Merger, Sebastian GoldtDiffusion ModelsStatistical Learning Theory

    Statistical Physics of Computation Laboratory, EPFL, Lausanne, Switzerland · Theoretical and Scientific Data Science Group, SISSA, Trieste, Italy

  21. Decomposing Observational Multiplicity in Decision Trees: Leaf and Structural Regret

    Mar 12, 2026Mustafa CavusStatistical Learning TheoryDecision Tree Learning

    Eskisehir Technical University, Department of Statistics, Turkiye 26555

  22. A Geometrically-Grounded Drive for MDL-Based Optimization in Deep Learning

    Mar 12, 2026Ming Lei, Shufan Wu, Christophe BaehrDeep Learning OptimizationMinimum Description Length

    School of Aeronautics and Astronautics, Shanghai JiaoTong University, Shanghai China · Météo-France/CNRS CNRM/GAME UMR 3589 and the Mathematical Institute of Toulouse, France

  23. Upper Generalization Bounds for Neural Oscillators

    Mar 10, 2026Zifeng Huang, Konstantin M. Zuev, Yong Xia +1Generalization BoundsNeural Network Generalization

    Institute for Risk and Reliability, Leibniz University Hannover, Callinstraße 34, Hannover, 30167, Germany · Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, California, United States · Joint Research Centre for Marine Infrastructure, Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China +2

  24. From Garbage to Gold: A Data-Architectural Theory of Predictive Robustness

    Mar 8, 2026Terrence J. Lee-St. John, Jordan L. Lawson, Bartlomiej Piechowski-JozwiakStatistical Learning Theory

    Independent Researcher & Institute for Healthier Living Abu Dhabi, Abu Dhabi, UAE · Independent Researcher & Brown University, Center for Computation and Visualization, Providence, RI, USA

  25. Sparsity and Out-of-Distribution Generalization

    Mar 8, 2026Scott Aaronson, Lin Lin Lee, Jiawei LiOOD GeneralizationGeneralization Bounds

    UT Austin

  26. ShakyPrepend: A Multi-Group Learner with Improved Sample Complexity

    Mar 7, 2026Lujing Zhang, Daniel Hsu, Sivaraman BalakrishnanStatistical Learning Theory

    Department of Statistics and Data Science Carnegie Mellon University · Computer Science Department Data Science Institute Columbia University · Department of Statistics and Data Science Machine Learning Department Carnegie Mellon University

  27. Margin in Abstract Spaces

    Mar 7, 2026Yair Ashlagi, Roi Livni, Shay Moran +1Kernel MethodsStatistical Learning Theory

    School of Electrical and Computer Engineering, Tel Aviv University. · Departments of Mathematics, Computer Science, and Data and Decision Sciences, Technion, and Google Research. · Departments of Mathematics, Technion.

  28. Post-Training with Policy Gradients: Optimality and the Base Model Barrier

    Mar 7, 2026Alireza Mousavi-Hosseini, Murat A. ErdogduLanguage Model Post-TrainingPolicy Gradient

    University of Toronto, Vector Institute. · University of Toronto, Vector Institute, and Numurho.

  29. Agnostic learning in (almost) optimal time via Gaussian surface area

    Mar 6, 2026Lucas Pesenti, Lucas Slot, Manuel WiedmerAgnostic LearningStatistical Learning Theory

    ETH Zurich · University of Amsterdam

  30. Improved Scaling Laws via Weak-to-Strong Generalization in Random Feature Ridge Regression

    Mar 5, 2026Diyuan Wu, Lehan Chen, Theodor Misiakiewicz +1Weak-to-Strong GeneralizationScaling Laws

    Institute of Science and Technology Austria (ISTA) · Department of Applied and Computational Mathematics, Yale University · Department of Statistics and Data Science, Yale University