ML Reproducibility

ML: Machine Learning

Overview

Machine learning reproducibility asks whether reported results can be recovered from the data, code, configurations, and execution environments—and whether they hold beyond one run. Common approaches include releasing versioned artifacts and provenance, standardizing benchmark splits and metrics, controlling seeds and numerical computation, and using independent replications with multi-seed uncertainty estimates. These methods are applied across settings such as LLM inference, biomedical prediction, recommender systems, and robotics, where results can shift with hardware, preprocessing, evaluation protocols, or data choice. A central tension is that deterministic reruns and executable code do not guarantee faithful reproduction or robust conclusions, while thorough audits and repeated experiments can be costly.

Latest papers

CardsList

138 papers23 papers in the last four weeks, up 15% on the four weeks before. 0.1% of all new papers.

  1. Hawkeye: Reproducing GPU-Level Non-Determinism

    Mar 20, 2026Erez Badash, Dan Boneh, Ilan Komargodski +1ML ReproducibilityNeural Network Verification

  2. Causal Representation Learning on High-Dimensional Data: Benchmarks, Reproducibility, and Evaluation Metrics

    Mar 18, 2026Alireza Sadeghi, Wael AbdAlmageedCausal Representation LearningML Reproducibility

  3. Operationalising Artificial Intelligence Bills of Materials (AIBOMs) for Verifiable AI Provenance and Lifecycle Assurance

    Mar 17, 2026Petar Radanliev, Omar Santos, Carsten Maple +1AI AssuranceData Provenance

  4. Code Sharing In Prediction Model Research: A Scoping Review

    Mar 16, 2026Thomas Sounack, Raffaele Giancotti, Catherine A. Gao +8Scientific ReproducibilityML Reproducibility

  5. The Costs of Reproducibility in Music Separation Research: a Replication of Band-Split RNN

    Mar 10, 2026Paul Magron, Romain Serizel, Constance DouwesMusic Source SeparationAudio Source Separation

  6. SplitLight: An Exploratory Toolkit for Recommender Systems Datasets and Splits

    Feb 22, 2026Anna Volodkevich, Dmitry Anikin, Danil Gusak +3Recommender SystemsML Reproducibility

  7. WarpRec: Unifying Academic Rigor and Industrial Scale for Responsible, Reproducible, and Efficient Recommendation

    Feb 19, 2026Marco Avolio, Potito Aghilar, Sabino Roccotelli +9Recommender SystemsML Reproducibility

  8. Replicable Constrained Bandits

    Feb 16, 2026Matteo Bollini, Gianmarco Genalti, Francesco Emanuele Stradi +2Multi-Armed BanditsML Reproducibility

  9. A Random Matrix Theory Perspective on the Consistency of Diffusion Models

    Feb 2, 2026Binxu Wang, Jacob Zavatone-Veth, Cengiz PehlevanDiffusion Model SamplingML Reproducibility

  10. Standardization of Post-Publication Code Verification is Possible with the Support of the Community

    Jan 12, 2026Susana Lopez-Moreno, Sangil Kim, Eric Dolores-CuencaScientific ReproducibilityML Reproducibility

  11. Glucose-ML: A collection of longitudinal diabetes datasets for development of robust AI solutions

    Jul 18, 2025Temiloluwa Prioleau, Baiying Lu, Yanjun CuiForecasting BenchmarksBlood Glucose Forecasting

  12. Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

    Jan 18, 2025Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12Benchmark ConstructionBenchmark Design

  13. Real Faults in Deep Learning Fault Benchmarks: How Real Are They?

    Dec 20, 2024Gunel Jahangirova, Nargiz Humbatova, Jinhan Kim +2Software Engineering BenchmarksBenchmark Reproducibility

  14. Replicability is Asymptotically Free in Multi-armed Bandits

    Feb 12, 2024Junpei Komiyama, Shinji Ito, Yuichi Yoshida +1Multi-Armed BanditsML Reproducibility