Statistical Learning Theory

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 127% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms
Jul 2, 2026Ye Tian, Mengchu Li, Marco Avella MedinaMulti-Task Learning Minimax Estimation
Department of Statistics, Pennsylvania State University · School of Mathematics, University of Birmingham · Department of Statistics, Columbia University
Aggregation with Exponential Weights is Optimal in Expectation
Jul 2, 2026Mikael Møller Høgsgaard, Patrick Rebeschini, Tobias WegelEnsemble Learning Minimax Estimation
Department of Statistics, University of Oxford · Department of Computer Science, ETH Zurich
Statistical Properties of $k$ -means Clustering for Data Missing Completely at Random
Jul 2, 2026Xin GuanClustering Unsupervised Clustering
Graduate School of Information Sciences Tohoku University Sendai, Miyagi 980-8579, Japan
Function-Counting Theory for Low-Dimensional Data Structures
Jul 1, 2026Konstantin Häberle, Helmut BölcskeiIntrinsic Dimensionality Binary Classification
ETH Zurich
Measuring Dead Directions: Decomposing and Classifying Singular Structure off Canonical Alignment
Jul 1, 2026Tejas Pradeep ShirodkarSingular Learning Theory Statistical Learning Theory
IIIT, Hyderabad
A Mechanism-Driven Theory of Phase Transitions in Active Learning
Jun 30, 2026Julia Machnio, Mads Nielsen, Mostafa Mehdipour GhaziActive Learning Statistical Learning Theory
Pioneer Centre for AI, University of Copenhagen, Copenhagen, Denmark
Accelerating Conformal Prediction via Approximate Leave-One-Out
Jun 30, 2026Jiachen Cong, Jingbo LiuConformal Prediction Statistical Learning Theory
Department of Statistics, University of Illinois Urbana-Champaign, Champaign, IL, 61820, USA
On Optimal Data Splitting for Split Conformal Prediction
Jun 30, 2026Sayan Das, Bahram Yaghooti, Todd A. Kuffner +1Conformal Prediction Statistical Learning Theory
Department of Statistics and Data Science, Washington University in St. Louis, St. Louis, MO 63130, USA · Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO 63130, USA
Explaining Machine Learning and Memorization with Statistical Mechanics
Jun 30, 2026Robin TheriaultStatistical Physics of Learning Neural Network Memorization
Faculty of Science
Fast approximation and learning of binary classification tasks in o-minimal structures using ReLU neural networks
Jun 29, 2026Clemens Kinn, Philipp PetersenReLU Neural Networks Binary Classification
Faculty of Mathematics, University of Vienna, 1090 Vienna, Austria
Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability
Jun 29, 2026Srinivasa Rao P., Vangmayi P ReddyStatistical Physics of Learning Neural Network Generalization
Curlvee Technolabs, India, Former Scientist, C-DAC · Former Scientist, C-DAC · Indian Institute of Technology Madras, India
Using Large Language Models as Low-Cost Statistical Estimators for Human-Response Data
Jun 29, 2026Haobo YangLanguage Model Calibration Statistical Learning Theory
Department of Computer Science and Engineering, SUSTech University
Data-Driven Energy-Based Learning via Gibbs Measures on Hierarchical Structures
Jun 29, 2026L. U. Abdullaev, F. Herrera, U. A. Rozikov +1Statistical Physics of Learning Phase Transitions
National University of Singapore, Singapore · Department of Computer Science and Artificial Intelligence, University of Granada, E-18071 Granada, Spain · V.I. Romanovskiy Institute of Mathematics, Uzbekistan Academy of Sciences, 9, Universitet str., 100174, Tashkent, Uzbekistan +3
The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning
Jun 29, 2026Yiting Hu, Lingjie Duan, Qian ZhangContinual Learning Statistical Learning Theory
Singapore University of Technology and Design, Singapore · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.
Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation
Jun 28, 2026Soumyadip SarkarProper Scoring Rules Multiclass Classification
Independent Researcher
Sample Complexity of Scientific Discovery: PAC Learnability of Compositional Function Trees
Jun 28, 2026Şuayp Talha Kocabay, Talha Rüzgar Akkuş, Kerem YalçınSample Complexity Statistical Learning Theory
Independent Researcher · TÜBİTAK Science High School
Generalization Analysis of Transformers in Distribution Regression
Jun 28, 2026Peilin Liu, Ding-Xuan ZhouNeural Network Generalization Transformer Attention
School of Mathematics and Statistics, University of Sydney, Sydney, NSW 2006, Australia
How Far Can Sharpness and Complexity Jointly Explain Generalization?
Jun 27, 2026Ziyu Cheng, Xitong Zhang, Longxiu Huang +1Neural Network Generalization Statistical Learning Theory
Michigan State University
Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model
Jun 26, 2026Andrea Montanari, Kangjie ZhouStatistical Learning Theory Multi-Index Models
Department of Mathematics and Department of Statistics, Stanford University · Department of Statistics, Columbia University
Surprises in Proper Positive-Only Learning
Jun 26, 2026Shai Ben-David, Farnam Mansouri, Anay Mehrotra +1Binary Classification Classification
University of Waterloo and Vector Institute · Stanford University · Yale University
How Width and Data Shape Generalization Scaling Laws in Quadratic Neural Networks
Jun 26, 2026Julius Girardin, Emanuele Troiani, Yizhou Xu +3Neural Network Generalization Scaling Laws
1Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · 2Information, Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL)
Singular Learning and Occam's Razor in Deep Monomial Networks
Jun 26, 2026Kathlén Kohn, Giovanni Luca Marchetti, Farhan Shabir +2Neural Network Optimization Singular Learning Theory
KTH Stockholm · No Affiliation · Umeå University +1
Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexity
Jun 25, 2026Haitong Liu, Deepak Narayanan Sridharan, David Steurer +1Minimax Estimation Statistical Learning Theory
ETH Zurich
Recovering Governing Equations from Solution Data: Identifiability Bounds for Linear and Nonlinear ODEs
Jun 25, 2026Yang Pan, Helmut BölcskeiParameter Identifiability Sample Complexity
Sketched Linear Contrastive Learning: Approximation, Optimization, and Statistical Scaling
Jun 25, 2026Ziyan Chen, Zhongzhu Zhou, Ding-Xuan ZhouContrastive Learning Statistical Learning Theory
The University of Sydney · Together AI
Data Augmentation: A Fourier Analysis Perspective
Jun 23, 2026Behrooz Tahmasebi, Melanie Weber, Stefanie JegelkaData Augmentation Statistical Learning Theory
Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA. · Technical University of Munich (CIT, MCML, MDSI) · MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)
Solve for the Hyperparameter, Skip the Search: Kolmogorov-Optimal Scaling Laws for Spline Regression
Jun 22, 2026Yong Yi Bay, Kathleen A. YearickModel Selection Scaling Laws
PhD, University of Illinois at Urbana-Champaign
Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions
Jun 22, 2026Sehwan Kim, Yan Sun, Faming LiangNeural Network Generalization Hierarchical Representation Learning
Ewha Womans University, Seoul 03760, Republic of Korea · New Jersey Institute of Technology, Newark, NJ 07102, USA · Purdue University, West Lafayette, IN 47907, USA
Non-asymptotic estimates of the minimal risk in statistical learning
Jun 22, 2026Liming Wu, Sen YangStatistical Learning Theory Empirical Risk Minimization
Laboratoire de Mathématiques Blaise Pascal, CNRS-UMR 6620, Université Clermont Auvergne · IASM, Harbin Institute of Technology
Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence
Jun 22, 2026Ioannis KalogridisNonparametric Regression Reproducing Kernel Hilbert Spaces
School of Mathematics and Statistics, University of Glasgow, United Kingdom

Jul 2, 2026, stat.MLJ/K move · Enter open · S save

Contaminated Multi-task Learning with Heterogeneity: Fundamental Limits and Optimal Algorithms

Ye Tian, Mengchu Li, Marco Avella Medina

Department of Statistics, Pennsylvania State University · School of Mathematics, University of Birmingham · Department of Statistics, Columbia University

Multi-Task Learning Minimax Estimation Statistical Learning Theory Robust Estimation

Open paper AskPDF arXiv

Integrating information across related tasks can improve estimation and prediction in transfer, multi-task, and federated learning, but contamination and heterogeneity make robust borrowing challenging. We study a contaminated multi-task empirical risk minimization (ERM) framework in which an

ε

fraction of

K

tasks, each with sample size

n

, may be arbitrarily contaminated while the remaining tasks are heterogeneous. Our goal is to estimate both the global minimizer of the average risk and the clean task-specific minimizers, thereby combining robustness and personalization. In the Gaussian mean model, we show that several common paradigms, including adaptive and robust regularization around a shared center, global matrix regularization, decomposition-based regularization, and score-based outlier-task detection, all suffer from a worst-case contamination error of order

ε\sqrt{d/n}

, which is suboptimal compared to the lower bound

ε/\sqrt{n}

. This identifies a dimension-dependent barrier for these approaches. We then establish minimax lower bounds for a general heterogeneous ERM setting and propose a computationally efficient filtering-based robust multi-task gradient descent method. Under local strong convexity, smoothness, and sub-Gaussian gradient assumptions, the proposed method attains high-probability upper bounds matching the minimax rates up to logarithmic factors over a broad regime. In particular, it removes the extra

\sqrt{d}

contamination dependence of many regularization-based methods and score-based outlier detection, while achieving personalization to local tasks under strong heterogeneity. Simulations and a real-data analysis demonstrate strong robustness and personalization relative to a broad range of benchmark methods.

Momentum

Related topics

Latest papers 395