Reproducibility

Ask about this topic

Momentum

31 papers in the last four weeks, up 63% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 238

All topics
CardsList

  1. UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents

    Oct 4, 2026Dolly Sah, Tanmay Sah, Harshul Jain +1Reproducibility

    Independent Researcher

  2. AI-assisted mitotic counting improves reproducibility and efficiency across multiple tumour types

    Oct 1, 2026Simon Graham, Mostafa Jahanifar, Quoc Dang Vu +15Mitotic FiguresTumor

    Histofy Ltd, Coventry, United Kingdom · Vilnius University Faculty of Medicine and National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Klinikos, Vilnius, Lithuania · Nottingham University Hospitals NHS Trust, Nottingham, United Kingdom +5

  3. Model validation in machine learning: A scenario-based guide from hold-out splits to nested group cross-validation in biomedical and applied research

    Oct 1, 2026Mehmet Baygin, Sengul Dogan, Turker TuncerCross ValidationValidation

    Department of Computer Engineering, Faculty of Engineering and Architecture, Erzurum Technical University, Erzurum, Türkiye · Department of Digital Forensics Engineering, Technology Faculty, Firat University, Elazig, Türkiye

  4. Code That Works, Environments That Don't: Measuring Environment Reproducibility in AI-Generated Software

    Sep 30, 2026Bhanu Prakash Vangala, Tanu MalikCoding AgentsCode Generation

    Department of Electrical Engineering and Computer Science, University of Missouri–Columbia, Missouri, USA

  5. Retrieve, Reproduce, Reveal: Dissecting Retrieval-Augmented Software Vulnerability Detection

    Sep 29, 2026Sabrina Kaniewski, Tim Krämer, Julius Bächle +3Model VulnerabilitiesSoftware

    Institute for Secure Networked Systems, Esslingen University, Germany · Esslingen University, Germany · Institute for Intelligent Systems, Esslingen University, Germany +1

  6. Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation

    Sep 29, 2026Mario Sanz-Guerrero, Minh Duc Bui, Manuel Mager +1ReproducibilityPrompt Engineering

    Johannes Gutenberg University Mainz, Germany · Universidad Iberoamericana, Mexico · University of Colorado Boulder, USA

  7. How Reproducible Are Evaluation Conclusions? A Self-Audit of LLM-Inferred Prompt Structure

    Sep 24, 2026Dipankar SarkarLarge Language Model EvaluationReproducibility

    Skelf Research

  8. Can Classical Semantic-Extractive Summarization Be Evaluated in Hindi? A Replication Study

    Sep 24, 2026Showket Ahmad Khan, Mudasir Mohd, Nasrullah Sheikh +4SummarizationHindi

    Department of Computer Science, South Campus, University of Kashmir, Anantnag, India · IBM Research, San Jose, CA, USA · Manipal University Jaipur, Dehmi Kalan, Jaipur 303007, Rajasthan, India

  9. Paging the Experts: A Reproducible Characterization of Flash-Backed MoE Inference on iPhone

    Sep 24, 2026Musa ShamsMixture-Of-Expert InferenceExperts

    Independent Researcher

  10. RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?

    Sep 23, 2026Mithil Salunkhe, Haochen Ding, Samridhi Verma +1ReproducibilityMle-Bench Lite

    University of Illinois Urbana-Champaign · National Center for Supercomputing Applications

  11. Driving Epidemic Models with AI Agents: the Epydemix Agent Framework

    Sep 23, 2026Nicolò Gozzi, Ciro Cattuto, Alessandro VespignaniEpidemiological ModelsAgent-Based Model

    ISI Foundation, Turin, Italy · Network Science Institute, Northeastern University, Boston, MA, USA

  12. Exact Feedback Is Not Control: Evaluating Text-based Closed-Loop Revision in LLMs

    Sep 23, 2026Haitong Jiang, Chunlin Liu, Yile Wang +1FeedbackReproducibility

    College of Computer Science and Software Engineering, Shenzhen University

  13. EvEMTBench: An Open Benchmark for Machine Learning in Power System Protection

    Sep 23, 2026Julian Oelhaf, Georg Kordowich, Christian Bergler +3Power SystemsFault Diagnosis

    Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen 91058, Germany · Institute of Electrical Energy Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen 91058, Germany · Department of Electrical Engineering, Media and Computer Science, Ostbayerische Technische Hochschule Amberg-Weiden, Amberg 92224, Germany

  14. Time-Series Foundation Models That Understand Data Revisions

    Sep 23, 2026Taimoor AhmadTime Series Foundation ModelsReproducibility

    Superior University Lahore, Pakistan

  15. The Illinois Social Attitudes Aggregate Corpus (ISAAC): An Open Tool and Reproducible Pipeline for Analyzing Social Group Discourse at Scale

    Sep 22, 2026Babak Hemmatian, Sarah Hadjarab, Jessica Chen +1Text CorporaDemographics

    AI Innovation Institute, Stony Brook University · Department of Political Science, University of Chicago · Department of Psychology, University of Illinois Urbana–Champaign +1

  16. Reproducible AI Requires Reproducible Randomness

    Sep 22, 2026Anthony Bertrand, Tom Schmitt, Engelbert Mephu Nguifo +1ReproducibilityRandom

    UCA, LIMOS · Université Clermont Auvergne, Clermont Auvergne INP, CNRS, LIMOS, F-63000 Clermont–Ferrand, France · UCA +2

  17. Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures

    Sep 22, 2026Liam Cooper, Shinnung Jeong, Hyeran Jeon +2LLM Inference OptimizationFloating-Point

    Georgia Institute of Technology · University of California, Merced

  18. GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay

    Sep 21, 2026Yiran Wang, Xingyilang Yin, Junfu Pu +12Agentic BenchmarksVideo World Models

    ARC Lab, Tencent · GVC Lab, Great Bay University · University of Macau +3

  19. LLJ Cards: Best practices for the Use of LLMs as Judges

    Sep 21, 2026Khaoula Chehbouni, Melina Medjdoub, Florian Carichon +2Llm-As-A-JudgeJudges

    McGill University · Mila - Quebec AI Institute · Concordia University

  20. RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

    Sep 18, 2026Shuai Bai, Jiayong Deng, Sicheng Fan +30Computer-Use AgentsSynthetic Environments

    Alibaba Token Hub, Alibaba Group

  21. Reproducibility is not construct validity: LLM measurement of institutionally situated communication

    Sep 17, 2026Veronika Batzdorfer, Carlo Romano Marcello Alessandro SantagiustinaInter-Annotator AgreementLarge Language Model Evaluation

    KIT · Department of Sociology and Computational Sociology, Karlsruhe Institute of Technology (KIT), Douglasstr. 24, 76133 Karlsruhe, Germany · ALMAnaCH, médialab, Sciences Po +2

  22. TTM-Bench: A Framework for Text-to-Music System Performance Benchmarking

    Sep 16, 2026Giorgia Adorni, Michela Papandrea, Battista Rimoldi +1Text-To-MusicPerformance Metric

    Institute of Information Systems and Networking (ISIN), SUPSI, Lugano, Switzerland · t2b AG, Basel, Switzerland

  23. OPEN-1B: A Fully Auditable Training Run

    Sep 15, 2026John Donaghy, Brian Wilcox, Oğuzhan Ersoy +6ReproducibilityTraining Data

    Gensyn

  24. Evaluating Open-Weight E-Commerce Agents with Environment-Grounded Verification

    Sep 14, 2026Nimit Shah, Haitz Sáez de Ocáriz BordeAgentic CommerceAgentic Evaluations

    AION

  25. OpenAI4S: Code as Action, Science as Sessions

    Sep 14, 2026Gongbo Zhang, Hao Li, Yu Wang +15Scientific WorkflowsReproducibility

    Peking University Shenzhen Graduate School, Shenzhen, 518055, China · Tsinghua-Peking Joint Center for Life Sciences, School of Life Sciences, Tsinghua University, Beijing, China · Independent Researcher +1

  26. Membership Inference via Pairwise Likelihood Ratios

    Sep 14, 2026Shengjie Niu, Zebin Yun, Yeheng Ge +1Membership Inference AttacksMembership Inference

    The Hong Kong Polytechnic University, Hong Kong, China · Tel Aviv University, Israel

  27. Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Papers

    Sep 11, 2026Hanhua Hong, Yizhi Li, Luu Gia Huy +3Agentic BenchmarksReproducibility

    The University of Manchester · IQuest Research · Vietnam National University +2

  28. IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications

    Sep 9, 2026Yiling Ma, Yilun Zhao, Sihong Wu +2ReproducibilityResearch Automation

    Yale University · TCS Research

  29. SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code

    Sep 8, 2026Wentao Li, Yibo Wu, Yizhe Chen +5Scientific FigureImage-To-Code Generation

    Tsinghua University

  30. APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents

    Sep 7, 2026Jintian Feng, Long Chen, Xiao Yu +7Graphical User Interface AgentsDevice

    Laboratory for Artificial Intelligence and New Forms of Education, Central China Normal University · Faculty of Artificial Intelligence in Education, Central China Normal University · Agentic Labs, Acrab AI

  31. Model Retirement Creates Reproducibility Risk in Biomedical AI Publications

    Sep 7, 2026Nathan Wolfrath, Meghan Conroy, Thomas Kosten +7Biomedical TextJournal

    Bud and Sue Selig Hub for Surgical Data Science, Department of Surgery, Medical College of Wisconsin · Division of Biostatistics, Data Science Institute, Medical College of Wisconsin · Department of Computer and Electrical Engineering, Marquette University +1

  32. Last Translation Benchmark

    Sep 3, 2026Vilém Zouhar, Niyati Bafna, Mukund Choudhary +257Machine Translation QualityReproducibility

  33. Concept of a Sensor Test Environment for Dusty Agricultural Conditions

    Sep 3, 2026Peter Buckel, Johannes Hermann, Jonas Wollmann +2Precision AgricultureAgricultural Robotics

    Technical University of Munich, Chair of Agrimechatronics and Munich Institute of Robotics and Machine Intelligence (MIRMI), 85354 Freising, Bavaria, Germany. · Baden-Wuerttemberg Cooperative State University (DHBW) Ravensburg, 88045 Friedrichshafen, Baden-Wuerttemberg, Germany.

  34. A comparative study on the accuracy & repeatability of mobile robotic platforms for the delivery of precision NDE measurement

    Sep 3, 2026SeyedMohammadAmin Nabi Pour, S. Gareth Pierce, Randika Vithanage +3Mobile RobotsReproducibility

  35. LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting

    Sep 1, 2026Yufei Chen, Yiran Zhao, Xiaogang Xu +3Large Language Model ForecastingProbabilistic Forecasting

    1Shandong University · 2Nanjing University of Aeronautics and Astronautics · School of Software Technology, Zhejiang University +2

  36. MutMem-V2: Cryptographically Authorized Mutation in Persistent Agent Memory Portable Verification and Reproducible Evidence

    Sep 1, 2026Walid SaidiOn-Chain AttestationsNativemem

    Independent researcher

  37. Beyond the Clock: Measuring the Value of Adaptive Revision

    Sep 1, 2026Ayushi ChadhaSequential Decision MakingDecisions

  38. Perceptible or Not? Diagnosing Passive Fingerprints for Speech Deepfake Attribution

    Sep 1, 2026Yupei Li, Qiyang Sun, Emmanouil Benetos +2Audio Deepfake DetectionFingerprint

    1Imperial College London · 4Technical University of Munich · 2Queen Mary University of London +1

  39. Thesis Proposal: Toward a Human-Centered and Perspective-Aware Framework for Reproducible ML Evaluation and AI Alignment

    Aug 31, 2026Deepak Pandita, Christopher M. HomanArtificial Intelligence AlignmentHuman-Centered Framework

    Rochester Institute of Technology

  40. Reliable Benchmarking of Artifact Detection in Computational Pathology: A Reproducibility and Uncertainty Analysis

    Aug 31, 2026Konstantinos Moutselos, Ilias MaglogiannisWhole-Slide ImagesComputational Pathology

    Department of Digital Systems, University of Piraeus, Piraeus, Greece

  41. Aggregate Disambiguation Systems

    Aug 31, 2026José María Lago, Albert Castellana, Edgars NemšeDisambiguationMajority Voting

    GenLayer Labs Research

  42. Repeatability Characterisation and Error Budget of a Consumer Structured-Light Scanner for 3-D Wound Geometry:A Rigid-Phantom Study

    Aug 31, 2026Pushkal Kumar, Aadit Aggarwal, Karlen AleksanyanReproducibilityRigid Objects

    independent researchers in the USA

  43. MedSegBenchmarker: A Raw-Count-First Framework for Controlled 2D Medical Image Segmentation Benchmarks

    Aug 30, 2026Vanessa Borst, Lukas Horn, Daniel Grillmeyer +2Semi-Supervised Medical Image SegmentationReproducibility

    University of Würzburg, Germany

  44. Training AI Scientists to Replicate Research

    Aug 13, 2026Damon Falck, Samer Sabri, Anja Surina +8Artificial Intelligence ScientistsArtificial Intelligence Agents

    1Inherent

  45. When Can You Trust Offline Evaluation of Equal-Cost Top-k Allocation? A Controlled, Reproducible Benchmark and Practitioner's Guide

    Aug 12, 2026Binshuang LiPolicy EvaluationTop-K

    Independent Researcher

  46. Reifying Research Logic: AI-Assisted Workflow Construction and Incremental Refinement for Quantitative Syntax

    Aug 11, 2026He Wang, Jingbo Chen, Yuqiao Lai +3Agentic Workflow DesignNatural Language

    College of International Studies, National University of Defense Technology, Nanjing, Jiangsu, 210000, China.

  47. Fairness in Link Prediction Beyond Demographic Parity: A Reproducibility Study

    Aug 10, 2026Valentijn Oldenburg, Floris de Kam, Stef de Wildt +1Algorithmic FairnessDisparities

    University of Amsterdam

  48. Sci-VBench: Evaluating Knowledge- and Reasoning-Intensive Video Generation in Science Domains

    Aug 10, 2026Diandian Zhang, Tingyu Song, Lin Fu +2Scientific DiscoveryVisual Question Answering Benchmarks

    Zhejiang University · UCAS · Tongji University +1

  49. XFeat Revisited: Reproducibility and Evaluation of a Lightweight Image Matcher

    Aug 10, 2026Lazar Đoković, Aimee LinHarder Better Faster Denser Feature MatchingCapability-Speed Trade-Offs

    Faculty of Computer and Information Science University of Ljubljana

  50. Software Engineering for and with GUI Agent

    Aug 10, 2026Shengcheng Yu, Yuchen Ling, Junyang Xing +3Graphical User Interface AgentsSoftware Engineering

    State Key Laboratory for Novel Software Technology, Nanjing University, China · Technical University of Munich, Germany

  51. EnergyBridge: Benchmarking Household Energy Management, User Participation, and Grid Flexibility

    Aug 9, 2026Xudong Wu, Zeqing Wu, Jiarui Zhang +8Smart HomesParticipation

    1The University of Hong Kong · 2Tianjin University · 3Beihang University

  52. Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families

    Aug 8, 2026Alizishaan Khatri, Dun Li ChanLarge Language Model SafetyModel Activations

    Wrynx Inc · INTI International College Penang

  53. JUMP-lite: Compact, reproducible benchmarking of cell representations

    Aug 7, 2026Alán F. Muñoz, Johan Fredin Haslum, Runxi Shen +2CellsPhenotypes

    Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA

  54. When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering

    Aug 6, 2026Rasul Khanbayov, Hasan KurbanReproducibilityAgreement

    College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar

  55. Sharding Prevents LLM Oversight Failures and Adversarial Exploitation

    Aug 5, 2026Victor Akinwande, J. Zico Kolter, Aran NayebiJudgesScalable Oversight

    School of Computer Science 2Neuroscience Institute Carnegie Mellon University

  56. CheMLFlow: An Open-Source Platform for Cheminformatics and Materials Informatics Applications

    Aug 5, 2026Brendan Smith, Susana Lopez-Moreno, Eric Dolores-Cuenca +3Scientific WorkflowsAgentic Workflows

    Kernfield Labs, London, United Kingdom. · Department of Mathematics, Pusan National University, Republic of Korea. · Humanoid Olfactory Display Center, Pusan National University, Republic of Korea. +4

  57. CURATE: Leveraging LLM Agents to Compose, Catalog, and Deploy Reproducible Workflows

    Aug 4, 2026Nolan Cutler, Chia-Chen Kuo, Nanda Velugoti +2Agentic Workflow DesignCoding Agents

    School of EECS Oregon State University · School of CBEE Oregon State University

  58. Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

    Aug 4, 2026Mohsen Hariri, Weicong Chen, Nahal Shahini +11Test-Time ScalingInference-Time

    Case Western Reserve University

  59. Adversarial Fast-Moving Real-World Domains as Test Beds for Benchmarking AI Scientist Capabilities

    Aug 4, 2026William Bolton, Philip TorrScientific IdeationArtificial Intelligence Benchmarks

    Department of Engineering Science, University of Oxford, Oxford, United Kingdom.

  60. Measuring Cross-Task Behavioral Consistency in Language Model Agents

    Jul 31, 2026Amritesh Banerjee, Pranil RaichuraLanguage-Model AgentsReproducibility

    University of Massachusetts Amherst · Mantis AI Research (MIT CSAIL)