ML Reproducibility

ML: Machine Learning

Overview

Machine learning reproducibility asks whether reported results can be recovered from the data, code, configurations, and execution environments—and whether they hold beyond one run. Common approaches include releasing versioned artifacts and provenance, standardizing benchmark splits and metrics, controlling seeds and numerical computation, and using independent replications with multi-seed uncertainty estimates. These methods are applied across settings such as LLM inference, biomedical prediction, recommender systems, and robotics, where results can shift with hardware, preprocessing, evaluation protocols, or data choice. A central tension is that deterministic reruns and executable code do not guarantee faithful reproduction or robust conclusions, while thorough audits and repeated experiments can be costly.

Latest papers

CardsList

138 papers23 papers in the last four weeks, up 15% on the four weeks before. 0.1% of all new papers.

  1. Measuring Deep Learning Performance - An Empirical Study of Performance Distributions Across Architectures and Tasks

    Oct 27, 2026Kevin L Coakley, Odd Erik GundersenML Reproducibility

  2. Beyond SAEs: Consistent Concept Discovery with Sparse Coding

    Oct 16, 2026Jinho Choi, Hyesu Lim, Jaegul Choo +1Sparse AutoencodersSparse Coding

  3. Equivalence testing and run-count design for stochastic machine-learning experiments

    Oct 16, 2026Dmytro Knopov, Nadiya ShvaiStatistical Hypothesis TestingStatistical Power Analysis

  4. Author-in-the-loop Agentic Reproducibility Auditing for a Machine Learning Conference

    Oct 15, 2026Yukun Dong, Linjun Zhang, Tian ZhengML ReproducibilityScientific Reproducibility

  5. Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility

    Oct 14, 2026Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu +1Sparse AutoencodersData-Efficient Learning

  6. What You Sign Is Not What You Run: The Verifiability Gap in Quantized LLM Supply Chains

    Oct 14, 2026Wei-Liang Liu, Cheng-Yen TsaiLLM SecurityML Reproducibility

  7. PRAISE: A Reproducible Workflow for Non-Programmer Scientists to Implement Published AI Models

    Oct 12, 2026Max Achterweust, Stefan Groot Nibbelink, Vincent J. de Beer +7ML ReproducibilityScientific Workflow Automation

  8. [Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning

    Oct 10, 2026Anton Nuzhdin, Andrei Gruzitski, Balázs Egyed +1VLM InterpretabilitySaliency Map Evaluation

  9. MARGoS: A Configuration-Driven Platform for Reproducible MARL Experiments in ARGoS

    Oct 9, 2026Tom Philipp Backert, Mohammad D. Soorati, Javad GhofraniML ReproducibilityMulti-Agent Reinforcement Learning

  10. Hidden Data Challenges in Clinical Machine Learning

    Oct 9, 2026Carolin Cissée, Dennis Fast, Raneen Younis +3Training Data CurationElectronic Health Records

  11. Reproducible LLM Inference Benchmarking: A Sequential Isolation Protocol for Regression Testing

    Oct 7, 2026Arnold Olympio, Juan Manuel Servera Bondroit, Wael Abdelmalek +2Benchmark ReproducibilityLLM Inference

  12. Beyond the Model: The Critical Role of Data Filtering in Clinical Machine Learning

    Oct 5, 2026Noah Subedar, Colin Campbell, Wenjing Zhang +3Training Data CurationML Reproducibility

  13. Technical Report on the Turba Fertilizer Machine Learning Stack in Morocco

    Oct 5, 2026Abdelghani Belgaid, Zakaria Mahmoud, Fahd Chibani +3Surrogate ModelingML Reproducibility

  14. Measuring and Reducing Cross-Vendor Mismatch in Language Models

    Oct 4, 2026Erland Hilman Fuadi, Chong Tian, Xiaosong Ma +1Efficient Neural Network InferenceML Reproducibility

  15. EasyClassifier: Honest, Reproducible Machine-Learning Classification for Researchers Who Do Not Program

    Oct 3, 2026Ahmad B. A. Hassanat, Ghada A. AltarawnehML ReproducibilityClassification

  16. Zephon: Elastic Determinism for Online, Stateful Foundation Model Data Loading Pipelines

    Oct 2, 2026Maximilian Böther, Josh Wills, Ties Robroek +14ML Reproducibility

  17. Model validation in machine learning: A scenario-based guide from hold-out splits to nested group cross-validation in biomedical and applied research

    Oct 1, 2026Mehmet Baygin, Sengul Dogan, Turker TuncerData LeakageCross-Validation

  18. Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation

    Sep 29, 2026Mario Sanz-Guerrero, Minh Duc Bui, Manuel Mager +1LLM EvaluationLanguage Model Generation Evaluation

  19. Making Cross-Continental Federated Learning Repeatable with FLIP: a Multi-Application Study

    Sep 28, 2026Rafael Garcia-Dias, Alexandre Triay Bagur, Chayanin Tangwiriyasakul +20HealthcareML Reproducibility

  20. Replication Failure and Trivial Baselines in Road-Level Crash Prediction

    Sep 28, 2026Maurya PatelTraffic Accident AnticipationGraph Neural Networks

  21. Identical Runs, Different Results: Benchmarking AI Coding Agents on Open-Weight Models

    Sep 27, 2026Eduardo Ariño de la Rubia, Szilard PafkaAI Coding AgentsLLM Agent Evaluation

  22. Position: Let's Strengthen Verifiability If We Can't Enforce Reproducibility

    Sep 25, 2026Samet Hicsonmez, Nermin Samet, Renaud MarletML ReproducibilityScientific Reproducibility

  23. RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?

    Sep 23, 2026Mithil Salunkhe, Haochen Ding, Samridhi Verma +1LLM Agent EvaluationAI Agent Benchmarks

  24. Reproducible AI Requires Reproducible Randomness

    Sep 22, 2026Anthony Bertrand, Tom Schmitt, Engelbert Mephu Nguifo +1ML ReproducibilityScientific Reproducibility