Statistical Learning Theory

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 127% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

The Geometric Structure of Models Learning Sparse Data
May 8, 2026Thomas Walker, T. Mitchell Roddenberry, Ahmed Imtiaz Humayun +2Neural Representation Geometry Statistical Learning Theory
Rice University · Rice University† · Now at Google Research +1
A Note on Non-Negative $L_1$ -Approximating Polynomials
May 8, 2026Jane H. Lee, Anay Mehrotra, Manolis ZampetakisStatistical Learning Theory
Yale University · Stanford University
Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning
May 8, 2026Aristotelis Ballas, Christos DiouSharpness-Aware Minimization Multi-Task Learning
Department of Informatics and Telematics Harokopio University of Athens Omirou 9, Athens, Greece
The Minimax Rate of Perturbed Second-Order Calibration
May 8, 2026Kamil Ciosek, Banafsheh Rafiee, Sina Ghiassian +1Post-Hoc Calibration Minimax Estimation
Spotify
A Refined Generalization Analysis for Extreme Multi-class Supervised Contrastive Representation Learning
May 8, 2026Nong Minh Hieu, Antoine LedentClass-Imbalanced Learning Contrastive Learning
School of Computing and Information Systems, Singapore Management University.
A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks
May 8, 2026Daning Cheng, Zeyu Liu, Jun Sun +4Neural Network Generalization Residual Learning
ICT · Zhejiang Lab · JD corporation +1
Risk-Consistent Multiclass Learning from Random Label-Subset Membership Queries
May 8, 2026Jiaxu Su, Junpeng Li, Changchun Hua +1Multiclass Classification Weakly Supervised Learning
Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, China
Kernel Selection is Model Selection: A Unified Complexity-Penalized Approach for MMD Two-Sample Tests
May 7, 2026Yijin Ni, Xiaoming HuoModel Selection Two-Sample Testing
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA
A Theory of Online Learning with Autoregressive Chain-of-Thought Reasoning
May 7, 2026Ilan Doron-Arad, Idan Mehalel, Elchanan MosselCoT Reasoning Autoregressive Generation
1MIT · 2The Hebrew University
A Regime Theory of Controller Class Selection for LLM Action Decisions
May 7, 2026Zhaoyang Jiang, Zhizhong Fu, Yunsoo Kim +4Selective Prediction Language Model-Based Control
University of Glasgow · UESTC, Chengdu, China · University College London
Contrastive Identification and Generation in the Limit
May 7, 2026Xiaoyu Li, Andi Han, Jiaojiao Jiang +1Statistical Learning Theory
University of New South Wales · University of Sydney
A Fine-Grained Understanding of Uniform Convergence for Halfspaces
May 7, 2026Aryeh Kontorovich, Kasper Green LarsenStatistical Learning Theory Generalization Bounds
Ben-Gurion University · Aarhus University
A Measure-Theoretic Finite-Sample Theory for Adaptive-Data Fitted Q-Iteration
May 7, 2026Manuel Haussmann, Mustafa Mert Çelikok, Melih KandemirReinforcement Learning Q-Learning
Department of Mathematics and Computer Science University of Southern Denmark
Weak-to-Strong Generalization is Nearly Inevitable (in Linear Models)
May 7, 2026Scott Geng, Dutch Hansen, Jerry LiLogistic Regression Weak-to-Strong Generalization
University of Washington, Seattle.
When Can Voting Help, Hurt, or Change Course? Exact Structure of Binary Test-Time Aggregation
May 7, 2026Yi LiuStatistical Learning Theory Majority Voting
York University
A renormalization-group inspired lattice-based framework for piecewise generalized linear models
May 6, 2026Joshua C. ChangInterpretable ML Statistical Learning Theory
NIH Clinical Center, Rehabilitation Medicine · Mederrata Research Inc · Sound Prediction Inc
Information-theoretic Limits of Learning and Estimation
May 6, 2026Abbas El Gamal, Maxim RaginskyInformation-Theoretic Generalization Bounds Minimax Estimation
Stanford University* · UIUC**
Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
May 6, 2026Alexander Hsu, Zhaiming Shen, Wenjing Liao +1Transformer Attention In-Context Learning
Department of Mathematics, Purdue University · School of Mathematics, Georgia Institute of Technology
A Closed-Form Adaptive-Landmark Kernel for Certified Point-Cloud and Graph Classification
May 5, 2026Sushovan Majhi, Atish Mitra, Žiga Virk +1Point Cloud Classification Classification
Data Science, George Washington University, USA · Department of Mathematical Sciences, Montana Technological University, USA · Faculty of Computer and Information Science, University of Ljubljana, Slovenia +1
Realizable Bayes-Consistency for General Metric Losses
May 5, 2026Dan Tsir Cohen, Steve Hanneke, Aryeh KontorovichStatistical Learning Theory
1Ben-Gurion University of the Negev · 2Purdue University, USA.
Phase Transitions in Driven Informational Systems: A Two-Field Perspective on Learning Theory and Non-Equilibrium Chemistry
May 5, 2026Truong Xuan KhanhPhase Transitions Statistical Learning Theory
H&K Research Studio, Clevix LLC Hanoi, Vietnam
On the Spectral Structure and Objective Equivalence of Orthogonal Multilabel Fisher Discriminants
May 5, 2026Brian Keith-Norambuena, Juan Bekios-CalfaMulti-Label Classification Statistical Learning Theory
Department of Computing & Systems Engineering, Universidad Cat´olica del Norte, Antofagasta, Chile.
A Closed-Form Persistence-Landmark Pipeline for Certified Point-Cloud and Graph Classification
May 4, 2026Sushovan Majhi, Atish Mitra, Žiga Virk +1Point Cloud Classification Classification
Data Science, George Washington University, USA · Department of Mathematical Sciences, Montana Technological University, USA · Faculty of Computer and Information Science, University of Ljubljana, Institute IMFM, Slovenia +1
A Near-optimal SQ Lower Bound for Smoothed Agnostic Learning of Boolean Halfspaces
May 4, 2026Tim SinenBoolean Function Learning Statistical Learning Theory
University of Bonn, Germany
Denoising data using convex relaxations
May 4, 2026Charles Fefferman, Aalok Gangopadhyay, Matti Lassas +2Cryo-Electron Microscopy Statistical Learning Theory
Department of Mathematics, Princeton University, Princeton, NJ 08544, USA. · TCS Research · Department of Mathematics and Statistics, University of Helsinki, FI-00014 Helsinki, Finland. +2
Statistical Consistency and Generalization of Contrastive Representation Learning
May 4, 2026Yuanfan Li, Xiyuan Wei, Tianbao Yang +1Contrastive Learning Supervised Contrastive Learning
University of Sydney · 2Texas A&M University
Extrapolation in Statistical Learning with Extreme Value Theory
May 3, 2026Sebastian Engelke, Nicola Gnecco, Anne SabourinOOD Generalization Statistical Learning Theory
Research Institute for Statistics and Information Science, University of Geneva, Switzerland · Department of Mathematics, Imperial College London, United Kingdom · Université Paris Cité, Université Paris-Saclay, ENS Paris-Saclay, CNRS, SSA, INSERM, Centre Borelli, France
How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse
May 3, 2026Xiaoxuan Ma, Yixuan Yang, Song Li +1Class-Imbalanced Learning Multi-Label Classification
Department of Mathematics, Zhejiang University, Hangzhou, China.
Quantifying Multimodal Capabilities: Formal Generalization Guarantees in Pairwise Metric Learning
May 2, 2026Richeng Zhou, Xuelin Zhang, Liyuan LiuMetric Learning Statistical Learning Theory
Networked Information Aggregation for Binary Classification
May 1, 2026MohammadHossein Bateni, Zahra Hadizadeh, MohammadTaghi Hajiaghayi +2Logistic Regression Binary Classification
Google Research, New York City, NY, USA · University of California, Irvine, CA, USA · University of Maryland, College Park, MD, USA

May 8, 2026, cs.LGJ/K move · Enter open · S save

The Geometric Structure of Models Learning Sparse Data

Thomas Walker, T. Mitchell Roddenberry, Ahmed Imtiaz Humayun, Randall Balestriero+1

Rice University · Rice University† · Now at Google Research+1

Neural Representation Geometry Statistical Learning Theory Data-Efficient Learning

Open paper AskPDF arXiv

The manifold hypothesis (MH) is often used to explain how machine learning can overcome the curse of dimensionality. However, the MH is only applicable in regimes where the training data provides a sufficiently dense sample of the underlying low-dimensional data manifold, or where such a low-dimensional manifold is conceivably present. We describe the regimes where the MH is not applicable as sparse. In this paper, we demonstrate that models succeed in the sparse regime by exploiting a highly structured local geometry, a property we formalize as normal alignment. We prove that normal-aligned classifiers -- whose input-output Jacobians are rank-one and align perfectly with the training data -- minimize the training objective under norm constraints and achieve maximal local robustness under a non-zero Jacobian constraint. For continuous piecewise-affine deep networks, normal alignment manifests geometrically as centroid alignment within the network's induced power diagram partition and results from the feature-learning regime. Motivated by these theoretical insights, we introduce GrokAlign, a regularization strategy that actively induces normal alignment. We demonstrate that GrokAlign significantly accelerates the training dynamics of deep networks relevant to the grokking phenomenon. Furthermore, we apply the principle of normal alignment to Recursive Feature Machines (RFMs) to introduce Recursive Feature Alignment Machines (RFAMs). We show that RFAMs exhibit greater adversarial robustness compared to RFMs when trained on tabular data.

Momentum

Related topics

Latest papers 395