Agentic programs: an emerging form of scientific software in computational materials science

Yunsung Lim; Haekwan Jeon; Jaesun Kim; Jisu Kim; Seungwu Han

cond-mat.mtrl-sciSep 1, 2026

Agentic programs: an emerging form of scientific software in computational materials science

Authors: Yunsung Lim, Haekwan Jeon, Jaesun Kim, Jisu Kim, Seungwu Han

Organizations: Research Institute of Advanced Materials, Seoul National University, Seoul, 08826, Republic of Korea. · Department of Materials Science and Engineering, Seoul National University, Seoul, 08826, Republic of Korea. · Center for AI and Natural Sciences, Korea Institute for Advanced Study, Seoul, 02455, Republic of Korea.

Abstract

Computational materials science has traditionally delegated algorithmic tasks to computers while leaving scientific judgments to humans. We argue that recent LLM-based agent harnesses enable an emerging form of scientific software, agentic programs, that combine deterministic algorithms with bounded LLM-based judgment, task-specific verification, episodic maturation, and complete delegation in production. We illustrate this concept with DeMARS, an agentic program for constructing atomistic models from experimentally measured disordered crystal structures.

Explore similar work

Can Coding Agents Reproduce Findings in Computational Materials Science?
May 1, 2026Ziyang Huang, Yi Cao, Ali K. Shargh +15Coding Agents Reproducibility
VASP Agent: An Agentic Framework for Autonomous First-principles Calculations
Dec 22, 2025Zeyu Xia, Jinzhe Ma, Congjie Zheng +11First-Principles Calculation Coding Agents
AGAPI-Agents: An Open-Access Agentic AI Platform for Accelerated Materials Design on AtomGPT.org
Dec 12, 2025Jaehyung Lee, Justin Ely, Kent Zhang +3Data-Driven Materials Discovery Agent Framework

May 1, 2026 · cs.CLJ/K move · Enter open · S save

Can Coding Agents Reproduce Findings in Computational Materials Science?

Ziyang Huang, Yi Cao, Ali K. Shargh, Jing Luo+14

Johns Hopkins University

Large language models are increasingly deployed as autonomous coding agents and have achieved remarkably strong performance on software engineering benchmarks. However, it is unclear whether such success transfers to computational scientific workflows, where tasks require not only strong coding ability, but also the ability to navigate complex, domain-specific procedures and to interpret results in the context of scientific claims. To address this question, we present AutoMat, a benchmark for evaluating LLM-based agents' ability to reproduce claims from computational materials science. AutoMat poses three interrelated challenges: recovering underspecified computational procedures, navigating specialized toolchains, and determining whether the resulting evidence supports a claim. By working closely with subject matter experts, we curate a set of claims from real materials science papers to test whether coding agents can recover and execute the end-to-end workflow needed to support (or undermine) such claims. We then evaluate multiple representative coding agent settings across several foundation models. Our results show that current LLM-based agents obtain low overall success rates on AutoMat, with the best-performing setting achieving a success rate of only 53%. Error analysis further reveals that agents perform worst when workflows must be reconstructed from paper text alone and that they fail primarily due to incomplete procedures, methodological deviations, and execution fragility. Taken together, these findings position AutoMat as both a benchmark for computational scientific reproducibility and a tool for diagnosing the current limitations of agentic systems in AI-for-science settings.

Coding Agents Reproducibility Automated Domain-Specific Language

Open paper Ask about it PDF

Abstract

Explore similar work

Can Coding Agents Reproduce Findings in Computational Materials Science?

VASP Agent: An Agentic Framework for Autonomous First-principles Calculations

AGAPI-Agents: An Open-Access Agentic AI Platform for Accelerated Materials Design on AtomGPT.org