Statistical Learning Theory

Follow this topic Ask about this topic

Momentum

34 papers in the last four weeks, up 127% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 395

All topics

Dataset Distillation Efficiently Encodes Low-Dimensional Representations from Gradient-Based Learning of Non-Linear Tasks
Mar 16, 2026Yuri Kinoshita, Naoki Nishikawa, Taro ToyoizumiDataset Distillation Statistical Learning Theory
The University of Tokyo, Tokyo, Japan · Laboratory for Neural Computation and Adaptation, RIKEN Center for Brain Science, Wako, Japan · RIKEN Center for Advanced Intelligence Project, Nihonbashi, Japan
Margin in Abstract Spaces
Mar 7, 2026Yair Ashlagi, Roi Livni, Shay Moran +1Kernel Methods Statistical Learning Theory
School of Electrical and Computer Engineering, Tel Aviv University. · Departments of Mathematics, Computer Science, and Data and Decision Sciences, Technion, and Google Research. · Departments of Mathematics, Technion.
Asymptotic Behavior of Multi--Task Learning: Implicit Regularization and Double Descent Effects
Mar 5, 2026Ayed M. Alrashdi, Oussama Dhifallah, Houssem SifaouDouble Descent Neural Network Generalization
Department of Electrical Engineering, College of Engineering, University of Ha’il, P.O. Box 2440, Ha’il, 81441, Saudi Arabia · John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA · Department of Electrical and Electronic Engineering, King’s College London – Strand, London WC2R 2LS – London, UK
Multiplicative Oracle Inequalities for Transductive Learning via Level-Set Aggregation
Mar 2, 2026Jian Qian, Jiachen XuTransductive Learning Statistical Learning Theory
The University of Hong Kong · Hong Kong University of Science and Technology
Relatively Smart: A New Approach for Instance-Optimal Learning
Mar 2, 2026Shaddin Dughmi, Alireza F. PourPAC Learning Semi-Supervised Learning
University of Southern California · University of Waterloo
A short tour of operator learning theory: Convergence rates, statistical limits, and open questions
Feb 28, 2026Simone Brugiapaglia, Nicola Rares Franco, Nicholas H. NelsenOperator Learning Statistical Learning Theory
Concordia University · Politecnico di Milano · Cornell University and The University of Texas at Austin
Provable Subspace Identification of Nonlinear Multi-view CCA
Feb 27, 2026Zhiwei Han, Stefan Matthes, Hao ShenRepresentation Identifiability Parameter Identifiability
Chair of Data Processing, Technical University of Munich, Germany · fortiss GmbH, Munich, Germany
A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning
Feb 23, 2026Nicolas Anguita, Francesco Locatello, Andrew M. Saxe +4Neural Network Generalization Representation Learning
Department of Engineering, University of Cambridge · Institute of Science and Technology, Austria (ISTA) · Gatsby Computational Neuroscience Unit and Sainsbury Wellcome Centre, UCL +1
Separating Oblivious and Adaptive Models of Variable Selection
Feb 18, 2026Ziyun Chen, Jerry Li, Kevin Tian +1Sparse Recovery Feature Selection
University of Washington · University of Texas at Austin
Limitations of SGD for Multi-Index Models Beyond Statistical Queries
Feb 5, 2026Daniel Barzilai, Ohad ShamirSingle-Index Models Statistical Learning Theory
Weizmann Institute of Science · University of Toronto
PCA of probability measures: Sparse and Dense sampling regimes
Feb 2, 2026Gachon Erell, Jérémie Bigot, Elsa CazellesPrincipal Component Analysis Dimensionality Reduction
Institut de Mathématiques de Bordeaux, Université de Bordeaux, CNRS (UMR 5251) · CNRS, IRIT (UMR 5505), Université de Toulouse
Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data
Feb 1, 2026Yinsong Wang, Shahin ShahrampourNeural Network Generalization Learning with Missing Data
University of Nevada Reno, Reno, NV 89557 USA · Department of Mechanical and Industrial Engineering at Northeastern University, Boston, MA 02115 USA
Deep networks learn to parse uniform-depth context-free languages from local statistics
Jan 31, 2026Jack T. Parley, Francesco Cagnetta, Matthieu WyartHierarchical Representation Learning Statistical Learning Theory
Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland · Theoretical and Scientific Data Science, SISSA, Trieste, Italy · Department of Physics and Astronomy, Johns Hopkins University, Baltimore, Maryland.
Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining
Jan 27, 2026Yunwei Ren, Yatin Dandi, Florent Krzakala +1Deep Learning Optimization Hierarchical Representation Learning
Princeton University · École Polytechnique Fédérale de Lausanne · University of California, Berkeley
Approximate full conformal prediction in an RKHS
Jan 19, 2026Davidson Lova Razafindrakoto, Alain Celisse, Jérôme LacailleConfidence Region Estimation Conformal Prediction
Laboratoire SAMM Université Paris 1 Panthéon-Sorbonne 90, rue de Tolbiac 75634 PARIS CEDEX 13, FRANCE · Safran Aircraft Engines Rond-Point René Ravaud, Réau, 77550 Moissy-Cramayel CEDEX, France
Learning with Monotone Adversarial Corruptions
Jan 5, 2026Kasper Green Larsen, Chirag Pabbaraju, Abhishek ShettyCorruption Robustness Binary Classification
Aarhus University · Stanford University · MIT
SGD-Based Knowledge Distillation with Bayesian Teachers: Theory and Guidelines
Jan 4, 2026Itai Morad, Nir Shlezinger, Yonina C. EldarStatistical Learning Theory Teacher-Student Learning
School of ECE, Ben-Gurion University, Be’er Sheva, Israel · Faculty of Math and CS, Weizmann Institute of Science, Rehovot, Israel
The Challenger: When Do New Data Sources Justify Switching Machine Learning Models?
Dec 20, 2025Vassilis Digalakis, Christophe Pérignon, Sébastien Saurin +1Sequential Decision Making Credit Scoring
Boston University · HEC Paris
Maximum Mean Discrepancy with Unequal Sample Sizes via Generalized U-Statistics
Dec 16, 2025Aaron Wei, Milad Jalali, Danica J. SutherlandTwo-Sample Testing Maximum Mean Discrepancy
University of British Columbia · Independent researcher, Vancouver, Canada · Alberta Machine Intelligence Institute
Learnability Window in Gated Recurrent Neural Networks
Dec 5, 2025Lorenzo LiviRecurrent Neural Networks Neural Network Training Dynamics
OPIT – Open Institute of Technology
Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond
Dec 4, 2025Kazuma SawayaFDR Control Feature Selection
The University of Tokyo, Bunkyo, Tokyo, Japan
A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees
Nov 28, 2025Miao Zhang, Junpeng Li, Changchun Hua +1Weakly Supervised Learning Statistical Learning Theory
Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, China
Learning and Testing Convex Functions
Nov 14, 2025Renato Ferreira Pinto, Cassandra Marcussen, Elchanan Mossel +1Statistical Learning Theory
Columbia University · Harvard University · MIT
Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies
Nov 5, 2025Gaia Grosso, Sai Sumedh R. Hindupur, Thomas Fel +3Kernel Methods Statistical Learning Theory
NSF AI Institute for Artificial Intelligence and Fundamental Interactions, Cambridge, 02139, MA · Laboratory for Nuclear Science, Massachusetts Institute of Technology, Cambridge, 02139, MA · School of Engineering and Applied Sciences, Harvard University, Allston, 02134, MA +1
Recursively Enumerably Representable Classes and Computable Versions of the Fundamental Theorem of Statistical Learning
Nov 4, 2025David Kattermann, Lothar Sebastian KrappPAC Learning Statistical Learning Theory
Institute for Symbolic Artificial Intelligence, Johannes Kepler University Linz, Austria. · Department of Mathematics and Statistics, University of Konstanz, Germany. · Institute for the Interdisciplinary Study of Language Evolution, University of Zurich, Switzerland.
Predicting kernel regression learning curves from only raw data statistics
Oct 16, 2025Dhruva Karkada, Joseph Turnbull, Yuxi Liu +1Kernel Regression Kernel Methods
UC Berkeley · Imbue
Provable Affine Identifiability of Nonlinear CCA under Latent Distributional Priors
Oct 6, 2025Zhiwei Han, Stefan Matthes, Hao ShenRepresentation Learning Parameter Identifiability
fortiss GmbH, Munich, Germany · Technical university of Munich
Learning Multi-Index Models with Hyper-Kernel Ridge Regression
Oct 2, 2025Shuo Huang, Hippolyte Labarrière, Ernesto De Vito +2Kernel Ridge Regression Kernel Regression
Istituto Italiano di Tecnologia, Genoa, Italy · MaLGa - DIBRIS - Universit`a di Genova, Genoa, Italy · MaLGa - DIMA - Universit`a di Genova, Genoa, Italy +2
Test time training enhances in-context learning of nonlinear functions
Sep 30, 2025Kento Kuwataka, Taiji SuzukiSingle-Index Models Test-Time Training
Department of Mathematical Engineering and Information Physics The University of Tokyo Hongo 7-3-1, Bunkyo-ku, Tokyo
Linear Regression under Missing or Corrupted Coordinates
Sep 23, 2025Ilias Diakonikolas, Jelena Diakonikolas, Daniel M. Kane +2Robust Regression Learning with Missing Data
University of Wisconsin-Madison · University of California, San Diego · University of California, Davis

Mar 16, 2026, cs.LGJ/K move · Enter open · S save

Dataset Distillation Efficiently Encodes Low-Dimensional Representations from Gradient-Based Learning of Non-Linear Tasks

Yuri Kinoshita, Naoki Nishikawa, Taro Toyoizumi

The University of Tokyo, Tokyo, Japan · Laboratory for Neural Computation and Adaptation, RIKEN Center for Brain Science, Wako, Japan · RIKEN Center for Advanced Intelligence Project, Nihonbashi, Japan

Dataset Distillation Statistical Learning Theory Multi-Index Models

Open paper AskPDF arXiv

Dataset distillation, a training-aware data compression technique, has recently attracted increasing attention as an effective tool for mitigating costs of optimization and data storage. However, progress remains largely empirical. Mechanisms underlying the extraction of task-relevant information from the training process and the efficient encoding of such information into synthetic data points remain elusive. In this paper, we theoretically analyze practical algorithms of dataset distillation applied to the gradient-based training of two-layer neural networks with width

L

. By focusing on a non-linear task structure called multi-index model, we prove that the low-dimensional structure of the problem is efficiently encoded into the resulting distilled data. This dataset reproduces a model with high generalization ability for a required memory complexity of

\tildeΘ$$(r^2d+L)

, where

d

and

r

are the input and intrinsic dimensions of the task. To the best of our knowledge, this is one of the first theoretical works that include a specific task structure, leverage its intrinsic dimensionality to quantify the compression rate and study dataset distillation implemented solely via gradient-based algorithms.

Momentum

Related topics

Latest papers 395