ML Reproducibility

ML: Machine Learning

Overview

Machine learning reproducibility asks whether reported results can be recovered from the data, code, configurations, and execution environments—and whether they hold beyond one run. Common approaches include releasing versioned artifacts and provenance, standardizing benchmark splits and metrics, controlling seeds and numerical computation, and using independent replications with multi-seed uncertainty estimates. These methods are applied across settings such as LLM inference, biomedical prediction, recommender systems, and robotics, where results can shift with hardware, preprocessing, evaluation protocols, or data choice. A central tension is that deterministic reruns and executable code do not guarantee faithful reproduction or robust conclusions, while thorough audits and repeated experiments can be costly.

Latest papers

CardsList

153 papers17 papers in the last four weeks, down 39% on the four weeks before. 0.1% of all new papers.

  1. CertiSum: Bitwise-Identical MXFP4 Training Across Platforms and Parallelism Layouts

    Nov 16, 2026Steve Rhyner, Yubo Gao, Stefan de Lasa +2ML ReproducibilityMixed-Precision Training

  2. The novelty exposed: ablation, reproducibility and transparency in 40 technical reports of open-weight large language models (LLMs)

    Nov 15, 2026Bahaeddin TürkoğluOpen-Weight Language ModelsML Reproducibility

  3. Paper-Faithful Federated Learning: Scientific Checks for Practitioner and Agent Development

    Nov 12, 2026Ali Dadras, Alp YurtseverML ReproducibilityFederated Learning

  4. Deterministic Narrative Gravity: Bridging Theory of Mind and Historical Simulation through ABM-LLM Architectures

    Nov 8, 2026Alicia Colmenero-Fernández, Lidia Ortega AlvaradoLarge Language Model-Based Social SimulationAgent-Based Modeling

  5. ReproBridge: Building Reproducibility into ML Research Artifacts

    Nov 5, 2026Yubo Wang, Rui Meng, Jaehyun Nam +2ML ReproducibilityBenchmark Reproducibility

  6. Attending to Memory Writes Reduces Seed Variability in Recurrent Length Generalization

    Nov 3, 2026Sungwoo Goo, Hwi-yeol Yun, Sangkeun JungRecurrent Neural NetworksLength Generalization

  7. Patching the Protocol: A Taxonomy of Pre-Registration Repairs in LLM-Proposer Experiments

    Nov 3, 2026Peter Chen, Ni Zhang, Siyuan ZhangML ReproducibilityScientific Reproducibility

  8. ReuScope: Auditing Data Reuse and Evaluation Sensitivity in Capsule Vision 2024

    Oct 30, 2026Eunseob Choi, Kyeonghun Kim, Hyuk-Jae Lee +1ML ReproducibilityBenchmark Reproducibility

  9. Towards Portable Specification of the Agent Harness

    Oct 29, 2026Nicolas Gertler, Claudio Novelli, Luciano FloridiLLM Agent HarnessesML Reproducibility

  10. Measuring Deep Learning Performance - An Empirical Study of Performance Distributions Across Architectures and Tasks

    Oct 27, 2026Kevin L Coakley, Odd Erik GundersenML Reproducibility

  11. Beyond SAEs: Consistent Concept Discovery with Sparse Coding

    Oct 16, 2026Jinho Choi, Hyesu Lim, Jaegul Choo +1Sparse AutoencodersSparse Coding

  12. Equivalence testing and run-count design for stochastic machine-learning experiments

    Oct 16, 2026Dmytro Knopov, Nadiya ShvaiStatistical Hypothesis TestingStatistical Power Analysis

  13. Author-in-the-loop Agentic Reproducibility Auditing for a Machine Learning Conference

    Oct 15, 2026Yukun Dong, Linjun Zhang, Tian ZhengML ReproducibilityScientific Reproducibility

  14. Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility

    Oct 14, 2026Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu +1Sparse AutoencodersData-Efficient Learning

  15. What You Sign Is Not What You Run: The Verifiability Gap in Quantized LLM Supply Chains

    Oct 14, 2026Wei-Liang Liu, Cheng-Yen TsaiLLM SecurityML Reproducibility

  16. PRAISE: A Reproducible Workflow for Non-Programmer Scientists to Implement Published AI Models

    Oct 12, 2026Max Achterweust, Stefan Groot Nibbelink, Vincent J. de Beer +7ML ReproducibilityScientific Workflow Automation

  17. [Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning

    Oct 10, 2026Anton Nuzhdin, Andrei Gruzitski, Balázs Egyed +1VLM InterpretabilitySaliency Map Evaluation

  18. MARGoS: A Configuration-Driven Platform for Reproducible MARL Experiments in ARGoS

    Oct 9, 2026Tom Philipp Backert, Mohammad D. Soorati, Javad GhofraniML ReproducibilityMulti-Agent Reinforcement Learning

  19. Hidden Data Challenges in Clinical Machine Learning

    Oct 9, 2026Carolin Cissée, Dennis Fast, Raneen Younis +3Training Data CurationElectronic Health Records

  20. Reproducible LLM Inference Benchmarking: A Sequential Isolation Protocol for Regression Testing

    Oct 7, 2026Arnold Olympio, Juan Manuel Servera Bondroit, Wael Abdelmalek +2Benchmark ReproducibilityLLM Inference