Learning Curves

Follow this topic Ask about this topic

Latest papers

14 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

On Data Scaling in Tabular ML: More Data, Better Generalization?
Oct 19, 2026Ivan Rubachev, Nikita Lazarev, Artem BabenkoTabular ML Tabular Foundation Models
Yandex · HSE University
Seer: Maximum Likelihood Regression for Learning-Speed Curves
Oct 2, 2026Carl Myers KadieLearning Curves Sample Complexity
University of Illinois at Urbana-Champaign, Graduate College, Computer Science, Urbana, Illinois
From Spectra to Joint Schedules in LLM Pre-training: 3+3(+2) Scaling-Law Regimes
Sep 30, 2026Yichen Wang, Fanghui Liu, Yudong ChenLanguage Model Pretraining Deep Learning Optimization
Department of Computer Sciences, University of Wisconsin–Madison, USA. · School of Mathematical Sciences, Institute of Natural Sciences and MOE-LSC, Shanghai Jiao Tong University, China.
Scaling Laws for Classical Machine Learning on Tabular Data: A Benchmark Study
Jul 23, 2026Kaihua DingTabular ML Scaling Laws
University of Pennsylvania
Data-Efficient Deep Learning: Empirical Guidelines for Training Set Size Estimation in Inertial Sensor Classification
Jul 10, 2026Ofir Kruzel, Itzik KlienSample Complexity Learning Curves
The Autonomous Navigation and Sensor Fusion Lab, the Hatter Department of Marine Technologies, University of Haifa, , Haifa, Israel
A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification
May 21, 2026Marcel Kühn, Yoon Thelge, Bernd RosenowNeural Network Training Dynamics Multiclass Classification
Institute for Theoretical Physics, Leipzig University · 2ScaDS.AI Dresden/Leipzig
Active Budget Allocation for Efficient Scaling Law Estimation via Surrogate-Guided Pruning
May 17, 2026Viktoria Schram, Markus Hiller, Daniel Beck +1Multi-Fidelity Optimization Scaling Laws
The School of Computing and Information Systems, the University of Melbourne, Melbourne, Australia · Royal Melbourne Institute of Technology, Melbourne, Australia · Now at Google Australia.
LEAP: Trajectory-Level Evaluation of LLMs in Iterative Scientific Design
May 14, 2026Marilyn Zhang, Tianfeng Chen, Fabián Barzuna +2LLM Evaluation AI Agent Benchmarks
On the Learning Curves of Revenue Maximization
Apr 29, 2026Steve Hanneke, Alkis Kalavasis, Shay Moran +1Sample Complexity Learning Curves
Purdue University · Yale University · Technion +1
Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions
Apr 25, 2026Weihao Lu, Qian Lin, Yingcun Xia +1Kernel Ridge Regression Benign Overfitting
Department of Statistics and Data Science, National University of Singapore. · Department of Statistics and Data Science, Tsinghua University.
Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Data
Mar 24, 2026Anand Jerry George, Nicolas MacrisIntrinsic Dimensionality Random Feature Methods
École Polytechnique Fédérale de Lausanne (EPFL), Lab for Statistical Mechanics of Inference in Large Systems (SMILS), CH-1015 Lausanne, Switzerland
Sample Size Calculations for Developing Clinical Prediction Models: Overview and pmsims R package
Feb 26, 2026Diana Shamsutdinova, Felix Zimmer, Oyebayo Ridwan Olaniran +4Learning Curves Clinical Prediction
Department of Biostatistics and Health Informatics, King’s College London · NIHR Biomedical Research Centre, Maudsley NHS Trust
The Challenger: When Do New Data Sources Justify Switching Machine Learning Models?
Dec 20, 2025Vassilis Digalakis, Christophe Pérignon, Sébastien Saurin +1Sequential Decision Making Learning Curves
Boston University · HEC Paris
Predicting kernel regression learning curves from only raw data statistics
Oct 16, 2025Dhruva Karkada, Joseph Turnbull, Yuxi Liu +1Kernel Regression Kernel Methods
UC Berkeley · Imbue

Oct 19, 2026J/K move · Enter open · S save

On Data Scaling in Tabular ML: More Data, Better Generalization?

Ivan Rubachev, Nikita Lazarev, Artem Babenko

Yandex · HSE University

Tabular ML Tabular Foundation Models Learning Curves OOD Generalization Tabular ICL

Open paper AskPDF arXiv

Recent work in tabular machine learning has emphasized the importance of diverse and realistic evaluation settings. The niche of large-scale (1M+) data, however, remains under-explored. To study training-data scaling, we assemble eight real-world tabular datasets with up to around ten million training examples, retaining the natural distribution shifts of the tasks and using random-split evaluation as a control. We find that GBDTs and MLP-based models show broadly similar scaling trends, with gains from both additional training data and algorithmic improvements. Tabular foundation models show more varied scaling behavior, with recent versions benefiting more consistently from larger contexts even under distribution shift. However, scaling training data under distribution shift introduces new challenges: in-distribution and out-of-distribution performance mostly improve together at smaller scales but can diverge at larger scales. We examine this through MLP capacity sweeps and comparisons of the validation distributions used for tuning and early stopping in MLPs and GBDTs. We also compare common practices of using validation data, contrasting its use for model selection in MLP pipelines with using it for in-context fitting in tabular foundation model pipelines. Under stronger shifts, MLPs selected using validation representative of the evaluation conditions generally maintain or improve performance as training data grows. In contrast, including such examples in context improves foundation-model performance, but does not prevent deterioration with increasing training size in our experiments.