FUSION: a skill-based research agent for publicly obtainable nuclear-physics codes
Authors: Jin Lei
Organizations: School of Physics Science and Engineering, Tongji University, Shanghai 200092, China. · Southern Center for Nuclear-Science Theory (SCNT), Institute of Modern Physics, Chinese Academy of Sciences, Huizhou 516000, Guangdong Province, China.
Running an unfamiliar nuclear-physics code is rarely difficult because of the physics alone. One must find and build the program, learn its input conventions, and decide whether a plausible output is actually correct. A general-purpose coding agent helps with the first two tasks but may make the last one harder: it can write an input file that runs with the wrong physical convention. FUSION addresses this problem with code-specific skills. A skill obtains the code from its public source, starts from a verified input, runs and parses the calculation, records known failure modes, and must reproduce a stated benchmark to a stated tolerance before reporting a result. The current release covers twenty codes, spanning optical models and reactions, nuclear structure, fission and statistical models, astrophysics and R-matrix analysis, and heavy-ion transport. It also includes an offline, searchable collection of 61 167 pages derived from the nucl-th literature. User notes and credentials remain outside the public repository. FUSION is available under the MIT license at https://github.com/jinleiphys/FUSION; documentation is at https://vibeinscience.com. Here I describe the design, the checks behind the current release, and one complete calculation from input to comparison with measured data.
Figures & tables
Figure 1: The failure mode and the fix. (a) A general-purpose assistant writes the input from memory; the code runs and prints a number, and a wrong convention is invisible. (b) A FUSION skill starts from a verified example deck and ends with a benchmark check whose agreement is stated in significant figures.
Figure 2: The three layers. The engine and the nuclear-physics layer ship in the clone; the private layer is a mount point the platform defines but never fills. The physics codes belong to none of the three and are obtained from their authors.
Figure 3: Anatomy of a skill, using FRESCO as the example. The layout is typical rather than fixed: a scope document, verified examples, reference notes, and scripts that install, run and check, with the potential-parameter script carrying its own self-test.
Figure 4: The twenty codes with a FUSION skill, grouped by area, with the language each is written in. The selection is bounded by an openness rule, not by a ranking: a code qualifies only if it is publicly obtainable, builds from source on the target platform, and has a published paper.
Figure 5: How a skill is verified before it ships. The adversarial review loops back to the build until a round returns no new defect of the kind the previous round repaired.
Figure 6: Adversarial review of one skill (SMASH), round by round. Pink marks defects introduced by the fixes of the previous round. The review stops when a round returns without a new defect of that kind.
Figure 7: What each skill’s benchmark rests on. Green: reproduction of a stored result (the code’s own reference, a published number, or bit identity across builds). Pink: a check that does not depend on any stored output (a physics invariant, an analytic solution, measured data, or an independent code). The right column gives the recorded agreement.
Figure 8: What ships (left) and how the 703 430 in-corpus citations divide by type (right). A contrasts edge records disagreement with the cited paper and survived a focused second-pass check.
Figure 9: One page of the knowledge base, with the origin of each part. Only the metadata and the abstract are the paper’s own; the digest sections are model-written and say so in the header; the citation sections are computed from the corpus.
Figure 10: The figure the session produced. Green is the zero-free-parameter KD02 prediction computed with FRESCO through the skill; pink points are EXFOR measurements. Left: 24 MeV, enriched 90 Zr, full angular range (Wang and Rapaport, EXFOR 13160.004). Right: 55 MeV, natural Zr, forward angles (Ibaraki et al., TIARA, EXFOR 22480.005). There is no 50 MeV panel because no measurement exists at that energy.
We introduce PHITSBench, an execution-scored benchmark for the Monte Carlo Particle and Heavy Ion Transport code System (PHITS). PHITSBench comprises 282 transport-scorable tasks spanning three common workflow categories: parameter editing (Edit), syntax repair (Repair ), and complete simulation generation from natural-language descriptions (Reproduce). Each task is evaluated using a Composite Metric Score that combines execution success with agreement between generated and reference transport observables. Using PHITSBench, we evaluate five GPT-5.4-based configurations ranging from zero-shot prompting to knowledge-augmented and agentic workflows. Without domain-specific knowledge, the model performs well on editing and repair tasks (95% and 70% success, respectively) but fails to generate correct simulations from scratch (0% success on the Reproduce track). A structured, machine-readable PHITS knowledge catalog, supplied alongside the user manual, raises single-shot Reproduce-task success to 57%. Agentic execution provides a further improvement to 66-73%, but at increased computational cost. Failure analysis shows that the remaining errors are dominated by incorrect selection and configuration of physical observables rather than syntax generation. These results suggest that future progress in AI-assisted radiation-transport modeling will depend as much on machine-readable knowledge bases, curated domain-training datasets, and execution-grounded evaluation environments as on advances in foundation models themselves.
Xianglin Ji, Svetlana V. Boriskina
Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
AI agents powered by large language models exhibit strong reasoning and problem-solving capabilities, enabling them to assist scientific research tasks such as formula derivation and code generation. However, whether these agents can reliably perform end-to-end reproduction from real scientific papers remains an open question. We introduce PRBench, a benchmark of 30 expert-curated tasks spanning 11 subfields of physics. Each task requires an agent to comprehend the methodology of a published paper, implement the corresponding algorithms from scratch, and produce quantitative results matching the original publication. Agents are provided only with the task instruction and paper content, and operate in a sandboxed execution environment. All tasks are contributed by domain experts from over 20 research groups at the School of Physics, Peking University, each grounded in a real published paper and validated through end-to-end reproduction with verified ground-truth results and detailed scoring rubrics. Using an agentified assessment pipeline, we evaluate a set of coding agents on PRBench and analyze their capabilities across key dimensions of scientific reasoning and execution. The best-performing agent, OpenAI Codex powered by GPT-5.3-Codex, achieves a mean overall score of 34%. All agents exhibit a zero end-to-end callback success rate, with particularly poor performance in data accuracy and code correctness. We further identify systematic failure modes, including errors in formula implementation, inability to debug numerical simulations, and fabrication of output data. Overall, PRBench provides a rigorous benchmark for evaluating progress toward autonomous scientific research.
Scientific coding agents receive equations, boundary conditions, and output requirements in writing, then must assess the programs they revise. Rules to Tools (R2T) supplies prepared executable checks of public scientific requirements. Matched SciCode repair groups share written checks, starting programs, model, and budgets; the tool group receives a callable implementation. Across two task-ID cohorts, complete repair is 26/30 with text and 29/30 with the prepared checks. Three task IDs favor tools, one favors text, and eleven tie. The eight-ID cohort scores 13/16 versus 15/16, with a task-cluster bootstrap 95% interval of [-12.5, 43.75] percentage points for the difference. The larger shared-definition SciCode cohort ties at 13/24 per group. Five development-exposed tasks with alternate starting programs score 3/10 versus 7/10. The tool group favors tasks 17, 77, and 11; initial checks flag task 17 and report no violation for tasks 77 and 11. Task 37 favors text and has no initial reported violation. A fresh source-through-Python arm also reaches 15/16, matching the dedicated command's aggregate. In a matched PDE comparison, detailed text scores 23/24 and checks score 24/24, with 31.2% lower reported model output for checks. Agent-side output savings vary by cohort, while public CPU use rises in both task-ID cohorts. These results measure task-dependent repair outcomes and agent-side costs with prepared checks.
Jingjie Ning, Guojiang Zhao, Chen Xu +4
School of Computer Science, Carnegie Mellon University · DP Technology