ML Reproducibility

ML: Machine Learning

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 88

All topics

Scaling Laws for Classical Machine Learning on Tabular Data: A Benchmark Study
Jul 23, 2026Kaihua DingTabular ML Scaling Laws
University of Pennsylvania
GlucoTune: A Unified Framework for Blood Glucose Preprocessing, Forecasting, and Benchmarking in Diabetes
Jul 23, 2026Davide Marelli, Giorgia Rigamonti, Mirko Paolo Barbato +1Type 1 Diabetes Time Series Forecasting
Department of Informatics, Systems and Communication, University of Milano-Bicocca, Milano 20126, Italy
Teach it to stop, not just to click
Jul 19, 2026Barada Sahu, Shivesh PandeyAI Agent Reliability Agent Reliability
Cabal AI · Para AI
Is Model Instability just Noise to be Tolerated or a Property that can be Managed?
Jul 11, 2026Amirali Rayegan, Lunxiao Li, Tim MenziesStochastic Optimization Automated Software Engineering
North Carolina State University Raleigh, North Carolina, USA
GPUSimBench: Towards Scalable and Reliable GPU-Accelerated Simulators in Embodied AI
Jul 6, 2026Huzhenyu Zhang, Shenghai Yuan, Wenrui Yan +4GPU Acceleration Robotics Simulation
Shanghai AI Laboratory, China · MIRAI, Russia · Nanyang Technological University, Singapore +2
Grokking Is Conditional and Fragile: A Fully-Tractable, Multi-Seed Study at 12K Parameters
Jul 6, 2026Yoshiyuki OotaniPhase Transitions ML Reproducibility
Independent Researcher
Coding-agents can replicate scientific machine learning papers
Jul 2, 2026Atharva Hans, Ilias BilionisCoding Agents LLM Agent Evaluation
School of Mechanical Engineering, Purdue University, West Lafayette, IN, 47907, USA
The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation
Jun 18, 2026Nicolas Dufour, Alexei A. Efros, Patrick PérezML Reproducibility Image Generation Evaluation
Kyutai · UC Berkeley
ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues
Jun 16, 2026Shanda Li, Qiuhong Anna Wei, Jingwu Tang +5LLM Agent Evaluation AI Agent Benchmarks
School of Computer Science, Carnegie Mellon University · Datadog
The Shift Toward Open and Reproducible AI Research
Jun 15, 2026Kevin L Coakley, Thijs Snelleman, Holger Hoos +1ML Reproducibility Scientific Reproducibility
Department of Computer Science, Norwegian University of Science and Technology, Sem Sælands Vei 7-9, Trondheim, 7031, Trønderlag, Norway. · San Diego Supercomputer Center, University of California San Diego, 9500 Gilman Dr, La Jolla, 92093, California, USA. · Chair of AI Methodology, RWTH Aachen University, Theaterstrasse 35-39, Aachen, 52062, North Rhine-Westphalia, Germany. +1
Incentives and Evidence in Learned Service Orchestration
Jun 15, 2026Syed Izhan Khilji, Alireza Furutanpey, Schahram DustdarRL Control Reinforcement Learning
†Distributed Systems Group (DSG), TU Wien, Vienna, Austria · ‡Coovally.ai, Barcelona, Spain
RecourseBench: A Modular Framework for Reproducible Algorithmic Recourse Evaluation
Jun 15, 2026Zahra Khotanlou, Hashir Ahmed, Chenghao Tan +2Benchmark Design Automated Evaluation
University of Waterloo
Mojo: A Promising Tool for Scalable Financial AI Efficiency
Jun 14, 2026Henry HanQuantitative Finance ML Reproducibility
Data Science and Artificial Intelligence Innovation Laboratory, School of Engineering and Computer Science, Baylor University, USA
Snyk VulnBench JS 1.0: Can LLMs Find the Same Bugs Twice?
Jun 14, 2026Liran Tal, Johannes Kloos, Arsenii Rudich +2Software Vulnerability Detection Static Code Analysis
Snyk
BrainSurgery: Reproducible and Reliable Declarative Weight Manipulations for Model Editing and Upcycling
Jun 8, 2026Gianluca Barmina, Annemette Broch Pirchert, Andrea Blasi Núñez +2ML Reproducibility
University of Southern Denmark
Execution Realism and Reproducibility in LLM-Based Trading Systems: A Systematic Scoping Review and Evidence Audit
Jun 6, 2026Junyi Yao, Zihao Zheng, Baichuan Li +1ML Reproducibility Finance
Washington University in St. Louis, St. Louis, MO 63130 USA. · Department of Operations Research and Engineering Management, Southern Methodist University, Dallas, TX 75275 USA.
Accelerating Reproducible Research in Synthetic EHR Generation
Jun 5, 2026Jalen Jiang, Chufan Gao, Ethan Rasmussen +2Synthetic Data Generation ML Reproducibility
University of Illinois Urbana-Champaign
Jacobi-Anger Method for Deterministic Initialization in Implicit Neural Representation
Jun 4, 2026Mohammed Alsakabi, Kejia Hu, John M. Dolan +1Implicit Neural Representations Neural Network Initialization
Department of Electrical and Computer Engineering, College of Engineering · The Robotics Institute, School of Computer Science Carnegie Mellon University, Pittsburgh, PA, USA
Tight list replicability bounds via a novel sphere covering theorem
Jun 4, 2026Ari Blondal, Hamed Hatami, Pooya Hatami +2ML Reproducibility Statistical Learning Theory
McGill University · Ohio State University
Revisiting Vul-RAG: Reproducibility and Replicability of RAG-based Vulnerability Detection with Open-Weight Models
Jun 3, 2026Sabrina Kaniewski, Fabian Schmidt, Tobias HeerRetrieval-Augmented Generation LLM Evaluation
Institute for Secure Networked Systems, Esslingen University, Esslingen, Germany · Institute for Intelligent Systems, Esslingen University, Esslingen, Germany
Reproducibility is the New Copyleft: Defining AGI-oriented Reproducible Builds
Jun 2, 2026Masayuki HattaML Reproducibility AI Governance
Surugadai University, Hanno Saitama 3570046, Japan
ERICA: Quantifying Replicability of Cluster Analysis
May 29, 2026Siamak K. Sorooshyari, Manuel A. Rivas, Robert TibshiraniClustering ML Reproducibility
Department of Statistics Stanford University Stanford, CA 94305, USA · Department of Biomedical Data Science Stanford University Stanford, CA 94305, USA · Department of Statistics and Biomedical Data Science Stanford University Stanford, CA 94305, USA
Modeling Robotics Dataset Construction as an Artifact-Based Build Process
May 29, 2026Leon Pohl, Lukas Beer, George Sebastian +1Robotics Automated Software Engineering
Institute for Autonomous Driving, University of the Bundeswehr Munich, 85579 Neubiberg, Germany
A Unified and Reproducible Experimentation Framework for Speech Understanding
May 29, 2026Jing Peng, Junhao Du, Chenghao Wang +21Audio-Language Model Evaluation ML Reproducibility
1X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University · 2AISpeech Ltd, Suzhou, China · 3ETH Zürich, Switzerland +3
Croissant Tasks: A Metadata Format for Reproducible Machine Learning Evaluations
May 28, 2026Omar Benjelloun, Leonardo Martins Bianco, Isabelle Guyon +8ML Reproducibility
Google DeepMind, Paris, France · ChaLearn, CA, USA · Université Paris-Saclay, Gif-sur-Yvette, France +8
From paper to benchmark: agentic, framework-based reproduction of under-specified methods in machine health intelligence
May 27, 2026Raffael Theiler, Ludovico Comito, David Leko +3Agentic Code Generation ML Reproducibility
EPFL · University of Roma “La Sapienza”
Picid: A Modular Evaluation Infrastructure for Reproducible PHM Across Tasks and Domains
May 27, 2026Lev Telyatnikov, Raffael Theiler, Leandro Von Krannichfeldt +1Benchmark Design Predictive Maintenance
EPFL
stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation
May 20, 2026Lucas Maes, Quentin Le Lidec, Luiz Facury +9World Models ML Reproducibility
Mila & Université de Montréal · New York University · Universidade Federal de Minas Gerais +4
HyperBench: Standardizing and Scaling Synthetic Evaluation for Hyperspectral Super-Resolution
May 20, 2026Ritik Shah, Marco F. DuarteBenchmark Design Remote Sensing Image Super-Resolution
University of Massachusetts Amherst, MA 01003
A Reproducible Log-Driven AutoML Framework for Interpretable Pipeline Optimization in Healthcare Risk Prediction
May 19, 2026Rui Huang, Lican HuangClass-Imbalanced Learning Interpretable ML
School of Basic Medicine, Hangzhou Normal University, No.2318, Yuhangtang Rd, Yuhang District, Hangzhou, 311121, Zhejiang, China. · Research Department, Hangzhou Domain Zones Technology Co.Ltd.,2026 200 Zhenhua Rd, West District, Hangzhou, 310030, Zhejiang, China.

Jul 23, 2026, cs.LGJ/K move · Enter open · S save

Scaling Laws for Classical Machine Learning on Tabular Data: A Benchmark Study

Kaihua Ding

University of Pennsylvania

Tabular ML Scaling Laws ML Reproducibility Tabular Prediction

Open paper AskPDF arXiv

Prior classical-ML learning-curve work fits power laws to tree, linear, and kernel models on tabular data, but at small scale: typically one curve, one team, a handful of cells. We present a distributed classroom-scale replication: 127 graduate students each ran a fixed protocol on 3 assigned datasets, drawn from 18 tabular classification and regression datasets and 6 model families (Boosting, Random Forest, SVM, Linear/Logistic, Ridge, Lasso), yielding 11,536 training runs and 1,648 fitted power-law curves of the form error(N) = a N^(-b) + c. Three findings. (1) Power laws fit: R^2 > 0.8 on 77.7% of cells, with tree ensembles dominating at full data (Boosting 50% of datasets, RandomForest 33%; linear models underperform on classification). (2) Approximate shared exponents within a model family: for 5 of 6 families, a single family-level exponent predicts each family's cross-dataset curves nearly as well as per-dataset exponents (R^2 gap < 0.011), though AIC favors the unconstrained fit and curve collapse is partial (32-58% of points within +/-0.5 dex). We frame this as approximate predictive compressibility, not dataset-independent universality; Lasso fails outright (negative control) and Ridge is fragile under leave-one-dataset-out. (3) Replicator-implementation variance: with random_state=42 fixed, independent re-implementations of the same protocol still differ by mean CV(b) = 0.144 on the fitted exponent -- not seed variance, but the spread induced by unconstrained parts of the protocol (preprocessing, encoding, missing-value handling). We release the aggregated curves, per-cell fits, and a practical data-requirement table for N* to reach target error 0.15.

Momentum

Related topics

Latest papers 88