Statistical Learning Theory

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 127% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

Noise-Debiased Thermodynamic Variance for Local Learning Coefficient Probes
Jun 21, 2026Yingjia CaiSingular Learning Theory Neural Network Training Dynamics
No Reference-Free Generalization in Quantum Machine Learning
Jun 21, 2026Jeongho BangQuantum Machine Learning Statistical Learning Theory
Institute for Convergence Research and Education in Advanced Technology, Yonsei University, Seoul 03722, Republic of Korea · Department of Quantum Information, Yonsei University, Incheon 21983, Republic of Korea
Convergence Analysis of Nyström Subsampling in Covariate Shift Adaptation for Misspecified case
Jun 20, 2026Hanna Myleiko, Sergei Solodky, Vasyl SemenovUnsupervised Domain Adaptation Domain Adaptation
Institute of Mathematics NAS of Ukraine, 3 Tereschenkivska st., Kyiv, Ukraine. · Institute of Mathematics NAS of Ukraine, 3 Tereschenkivska st., Kyiv, Ukraine · University of Giessen, Department of Mathematics, Giessen, Germany. +1
Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Design
Jun 19, 2026Junren Chen, Arya MazumdarLogistic Regression Gradient Descent
Department of Mathematics, University of Maryland, College Park. · Halıcıoğlu Data Science Institute, UC San Diego.
Subsampling for supervised learning in reproducing kernel Hilbert spaces
Jun 19, 2026Eyal Vayness, Maxime SangnierImportance Sampling Adaptive Sampling
Sorbonne Universit´e, LPSM, Paris, France
Gradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation
Jun 19, 2026Jianwei LouContinual Learning Online Continual Learning
RailMind Systems, Neuss, Germany
Dead-Direction Signatures: A Cheap Spectral Reading of Singular Complexity
Jun 19, 2026Tejas Pradeep Shirodkar, P. J. NarayananSingular Learning Theory Spectral Methods
IIIT, Hyderabad
On the Oracle Complexity of Interpolation-Based Gradient Descent
Jun 18, 2026Dongmin Lee, William Lu, Anuran MakurOracle Complexity Gradient Descent
Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA · Department of Computer Science and the Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA
On Local Population-Risk Certificates
Jun 17, 2026Mingzhi SongStatistical Learning Theory Generalization Bounds
Department of Mathematics, The University of Hong Kong, Hong Kong
Smoothness-Based Derandomization of PAC-Bayes Bounds
Jun 17, 2026Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe GiguèreNeural Network Generalization PAC-Bayesian Generalization Bounds
Department of Computer Science and Software Engineering Université Laval, Québec, Canada
Kernel of Partition Paths: A Unified Representation for Tree Ensembles
Jun 17, 2026Nicolas MahlerFeature Attribution Ensemble Learning
Datapred SAS 23 rue Mirabeau 94300 Vincennes, France · Datapred SA EPFL Innovation Park – Bâtiment A 1015 Lausanne, Switzerland
Sign-Rank, Index, and List Replicability: Connections and Separations
Jun 16, 2026Ari Blondal, Hamed Hatami, Pooya Hatami +2Statistical Learning Theory
McGill University · Ohio State University
No-Free-Fairness: Fundamental Limits and Trade-offs in Learning Systems
Jun 16, 2026Khoat ThanAlgorithmic Fairness Group Fairness
Hanoi University of Science and Technology
Another Look at Log-PCA for Probability Measures: A Dynamical Formulation and Statistical Convergence
Jun 15, 2026Peng Xu, Changbo Zhu, Young-Heon Kim +1Principal Component Analysis Statistical Learning Theory
Department of Statistics University of Illinois Urbana-Champaign · Department of ACMS University of Notre Dame · Department of Mathematics University of British Columbia +1
A nonparametric two-sample test using a parametric integral probability metric
Jun 15, 2026Yuha Park, Yongdai KimTwo-Sample Testing Statistical Learning Theory
Department of Mathematics, University of Hamburg, Hamburg, Germany. · Department of Statistics, Seoul National University, Seoul, Republic of Korea.
Brownian Kernel Ladders
Jun 14, 2026Mahdi Mohammadigohari, Giuseppe Di Fatta, Giuseppe Nicosia +1Hierarchical Representation Learning Reproducing Kernel Hilbert Spaces
Faculty of Engineering, Free University of Bozen-Bolzano, Via Bruno Buozzi 1, Bolzano, 39100, Italy · Department of Biomedical and Biotechnological Sciences, University of Catania, Catania, 95123, Santa Sofia, 97, Italy · Center for Applied Optimization, Department of Industrial and Systems Engineering, University of Florida, FL 32611, Gainesville, USA
Can Neural Networks Achieve Optimal Computational-statistical Tradeoff? An Analysis on Single-Index Model
Jun 13, 2026Siyu Chen, Beining Wu, Miao Lu +2Representation Learning Single-Index Models
Department of Statistics and Data Science, Yale University · Department of Statistics, University of Chicago · Department of Management Science and Engineering, Stanford University +1
The Geometry of Saturation: Effective Rank Predicts When Labels Stop Helping in Few-Shot Classification
Jun 12, 2026Arnav GuptaEarly Stopping Few-Shot Learning
Independent Researcher, Nepal
Gradient boosting for extremes: sampling theory and application to insurance
Jun 12, 2026Stéphane Lhaut, Olivier LopezGradient-Boosted Decision Trees Heavy-Tailed Distributions
1CREST, CNRS, Ecole polytechnique, Groupe ENSAE-ENSAI, ENSAE Paris, Institut Polytechnique de Paris, Palaiseau, France
A Bregman Perspective on Classification and Regression Trees
Jun 12, 2026Mathias BourelFunctional Bregman Divergences Decision Tree Learning
IESTA, Facultad de Ciencias Económicas y de Administración, Universidad de la República, Uruguay · IRL-2030, Instituto Franco-Uruguayo de Matemática e Interacciones (IFUMI)
Learning with Simulators: No Regret in a Computationally Bounded World
Jun 11, 2026Sasha Voitovych, Abhishek Shetty, Noah Golowich +1PAC Learning Statistical Learning Theory
MIT · Microsoft Research
Limits of spectral learning under noise
Jun 11, 2026Sabin Roman, Ljupco Todorovski, Saso Dzeroski +2Noisy-Label Learning Spectral Methods
Jožef Stefan Institute, Ljubljana, Slovenia · Faculty of Mathematics and Physics, University of Ljubljana, Slovenia · Department of Chemical Engineering, Universitat Rovira i Virgili, Tarragona, Catalonia +2
A solvable model for unsupervised federated learning
Jun 11, 2026Giovanni Catania, Aurélien Decelle, Gianluca Manzan +2Statistical Physics of Learning Unsupervised Learning
Institute for Cross-disciplinary Physics and Complex Systems IFISC (CSIC-UIB), Campus Universitat Illes Balears, 07122 Palma de Mallorca, Spain. · Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid, Calle de José Gutiérrez Abascal 2, Madrid 28006, Spain. · GISC - Grupo Interdisciplinar de Sistemas Complejos 28040 Madrid, Spain. +5
Is Spurious Correlation Removal Always Learnable?
Jun 11, 2026Yibo Zhou, Bo Li, Hai-Miao Hu +3Spurious Correlation Robustness Invariant Representation Learning
Beijing Key Laboratory of Digital Media, School of Computer Science and Engineering, Beihang University, Beijing 100191, China · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China · Hangzhou Innovation Institute of Beihang University, Hangzhou 310051, China +1
Two-Layer Linear Auto-Regressive Models Estimate Latent States
Jun 10, 2026Yahya Sattar, Sunmook Choi, Leo Maynard-Zhang +3Dynamical Systems Kalman Filtering
Cornell · U Washington · Imperial College London +1
How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings?
Jun 10, 2026Julia Kostin, Kasra Jalaldoust, Elias Bareinboim +2Structural Causal Models Statistical Learning Theory
Department of Computer Science, ETH Zurich · Causal Artificial Intelligence Lab, Columbia University · Department of Statistics, Columbia University
Quantum Occam Learning: Sample-Supported Expressibility for Circuit-Based Quantum Learning
Jun 10, 2026Jeongho Bang, Kyoungho Cho, Jeongwoo JaeModel Selection Sample Complexity
Institute for Convergence Research and Education in Advanced Technology, Yonsei University, Seoul 03722, Republic of Korea · Department of Quantum Information, Yonsei University, Incheon 21983, Republic of Korea · Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of Korea +1
Robust Regression of General ReLUs with Queries
Jun 9, 2026Ilias Diakonikolas, Daniel M. Kane, Mingchen MaReLU Neural Networks Active Learning
University of Wisconsin-Madison · University of California, San Diego
Range Penalization: Theoretical Insights with Applications in Federated Learning
Jun 9, 2026Yiyuan She, Zhaojun Hu, Yifan SunStatistical Learning Theory Clustered Federated Learning
Near-Exponential Convergence Rates for kNN Classification based on Boltzmann Margin
Jun 9, 2026Luyuan Yang, Shayan Shafaei, Chao LanClassification k-Nearest Neighbors
School of Computer Science, University of Oklahoma, Norman, Oklahoma, USA

Jun 21, 2026, cs.LGJ/K move · Enter open · S save

Noise-Debiased Thermodynamic Variance for Local Learning Coefficient Probes

Yingjia Cai

Singular Learning Theory Neural Network Training Dynamics Statistical Learning Theory Local Learning

Open paper AskPDF arXiv

Local learning coefficient (LLC) probes offer a singularity-aware view of neural-network training, but mean-energy methods require a local loss baseline that is ambiguous at transient checkpoints. Thermodynamic variance avoids this input; under mini-batch evaluation, however, direct variance mixes cross-state loss fluctuations with same-state noise. We operationalize this route with the \emph{Shift-Invariant Variance Estimator} (SIVE), which estimates and subtracts the latter component using repeated evaluations. Conditional on any fixed retained path, unclipped SIVE is unbiased for noiseless path variance without requiring MCMC stationarity. The finite-scale diagnostic remains indexed by localization scale

h

---even a locally linear loss has tether-dependent variance---while interpretation as a Real Log Canonical Threshold (RLCT) requires additional stationary low-temperature conditions. Toy experiments recover calibrated finite-scale targets. At the primary localization scale, all five MNIST MLP trajectories exhibit a mid-training trough followed by a rebound in SIVE, while Raw Variance decreases from Epoch 40 to 100 in every trajectory. Across four localization scales, the joint early-drop/late-rise criterion is met in 19 of 20 trajectory--scale pairs. At Epoch 40, the estimated observation-noise correction accounts for

77.5\%

of Raw Variance. Same-state debiasing thus reveals a reproducible turning structure masked by time-varying observation noise.

Momentum

Related topics

Latest papers 395