SimAuthor: Harnessing Foundation Models for Persistent Scientific Simulator Authoring

Yishan Wang; Ran Piao; Mathias Funk; Aaqib Saeed

cs.LGOct 5, 2026

SimAuthor: Harnessing Foundation Models for Persistent Scientific Simulator Authoring

Authors: Yishan Wang, Ran Piao, Mathias Funk, Aaqib Saeed

Organizations: Eindhoven University of Technology

Abstract

Foundation models can generate scientific code, but authoring a scientific simulator (an executable program encoding hypotheses about how mechanisms generate observable signals) requires iterative refinement. Scientific adequacy rarely admits a unique implementation or exact test, so simulators must instead be judged against limited real observations. We study this setting as scientific simulator authoring under weak empirical feedback, where distributional comparisons between simulated and real signals guide revision, and the target is the simulator itself rather than only its generated samples. We introduce SimAuthor, a persistent authoring harness that retains and revises executable simulators, separates scalar search scores from structured discrepancy feedback, and accumulates reusable implementation mechanisms. We evaluate SimAuthor on six biomedical tasks spanning cardiac and respiratory audio, photoplethysmography (PPG), and electrocardiography (ECG). Under a fixed 100-attempt budget, SimAuthor outperforms PUCT score search on all six tasks, generally outperforms textual-strategy optimization, and achieves the highest endpoint score on five of six. The authored simulators also improve on unseen recordings, transfer to independent pretrained representations, and yield substantial out-of-distribution gains in downstream ECG classification. Finally, 111 of 138 audited revisions alter program structure and account for 86.1% of the signed score improvement. These results suggest that persistent revision can progressively convert foundation-model knowledge into better executable scientific simulators from limited empirical evidence.

Figures & tables

Figure 1: Overview. SimAuthor turns LLM prior knowledge into executable simulator code when exact verification is unavailable and feedback comes only from scarce real data.

Figure 2: The SimAuthor authoring harness. A search orchestrator selects simulator candidates based on numerical scores, while the refiner updates them using the scientific blueprint, discrepancy report, and mechanism library.

Figure 3: Best-so-far performance over 100 authoring attempts. For seeded methods, curves show the complete trajectory of the median-Best@100 run, with min–max shading across three seeds. Independent sampling follows its fixed proposal order.

	OPERA-CE			PaPaGei
Method	VSD	AS	COPD	AF
Root $\mathcal{S}^{(0)}$	$.317\,\|\,.317$	$.192\,\|\,.192$	$.152\,\|\,.152$	$.354\,\|\,.354$
Sampling	$.123\,\|\,.715$	$.106\,\|\,.612$	$.162\,\|\,.799$	$.434\,\|\,\mathbf{.830}$
Text-Opt	$\mathbf{.443}\,\|\,.784$	$.258\,\|\,.685$	$.468\,\|\,\mathbf{.847}$	$.631\,\|\,.734$
SimAuthor	$.204\,\|\,\mathbf{.789}$	$\mathbf{.306}\,\|\,\mathbf{.906}$	$\mathbf{.561}\,\|\,.752$	$\mathbf{.665}\,\|\,.737$

Table 2: Generalization beyond the authoring objective. (a) Held-out representation evaluation on audio and PPG; entries are

R_{\mathrm{sel}}\,|\,R_{\max}

. (b) ECG downstream utility with 300-example augmentation, reported as AUPRC; evaluation is in-distribution (ID) on PTB-XL and out-of-distribution (OOD) on Chapman–Shaoxing. Corresponding AUROC results in Appendix H . Bold indicates the best result.

Task	SimAuthor	$-$ Report	$-$ Library
VSD	.587	.366 ( $-.221$ )	.399 ( $-.188$ )
AS	.557	.152 ( $-.405$ )	.383 ( $-.174$ )
COPD	.672	.609 ( $-.063$ )	.662 ( $-.010$ )
AF	.738	.589 ( $-.148$ )	.719 ( $-.019$ )
LQT	.568	.460 ( $-.108$ )	.540 ( $-.028$ )
WPW	.581	.492 ( $-.089$ )	.573 ( $-.008$ )

Table 3: Focused component ablations. (a) Final candidate quality, measured by

S_{\max}

after 100 authoring attempts. (b) Search efficiency, measured by

\mathrm{AUC}_{100}

, the mean best-so-far score over the 100-attempt trajectory. All values are medians over three seeds. Parentheses report the change relative to SimAuthor for the same metric.

Panel of Figure 4 — Figure 4: Structural revision during authoring. (a,b) Abridged code changes. (c) Structural share of 138 selected-lineage revisions and their signed score gain.

	Patient-disjoint ID		External OOD
Method	VSD	AS	COPD	AF	LQT	WPW
Root $\mathcal{S}^{(0)}$	.223	.103	.088	.104	.277	.385
Sampling	.425 (+.202)	.146 (+.043)	.362 (+.275)	.117 (+.013)	.341 (+.064)	.443 (+.059)
SimAuthor	.577 (+.354)	.440 (+.338)	.507 (+.420)	.204 (+.100)	.480 (+.203)	.538 (+.153)

Table 4: Generalization to held-out recordings. VSD and AS use patient-disjoint ID splits; the remaining tasks use external OOD datasets. Scores use the same evaluator as authoring. SimAuthor reports the median across three seeds; parentheses show gains over the root. See Appendix E for dataset details.

Task	$c=.10$	$c=.25$	$c=.50$
	PUCT exploration
VSD	[.330, .362, .472]	[.448, .587 , .611]	[.488, .504, .535]
LQT	[.513, .548, .554]	[.541, .568, .594]	[.572, .596 , .637]

Table 5: Sensitivity to exploration and authoring backbone. Final

S_{\max}@100

over three seeds. Each cell lists scores in ascending order (median in the middle); bold marks the highest median for each task.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Condition	Split	Dataset	Modality	$n$
VSD	Search	ZCHSound ( Jia et al., 2024 )	Cardiac audio	60 (60)
VSD	Held-out	ZCHSound	Cardiac audio	40 (40)
AS	Search	BMD-HS ( Ali et al., 2026 )	Cardiac audio	37 (5)
AS	Held-out	BMD-HS	Cardiac audio	23 (3)
COPD	Search	ICBHI 2017 ( Rocha et al., 2017 )	Resp. audio	65 (25)
COPD	Held-out	KAUH + TR ( Fraiwan et al., 2021 ; Altan et al., 2017 )	Resp. audio	99 (45)

Appendix

Table 6: Reference data for the six-condition benchmark. Search recordings are available during authoring; held-out recordings are used only for evaluation. Counts are recordings/windows, with unique participants in parentheses where available.

Task	Method	Seed 1	Seed 2	Seed 3	Std. dev.
VSD	PUCT Score Search	.365	.356	.337	.014
	Text-Opt	.416	.512	.346	.083
	SimAuthor	.587	.448	.611	.088
AS	PUCT Score Search	.254	.247	.227	.014
	Text-Opt	.370	.412	.504	.069
	SimAuthor	.429	.559	.557	.074

Appendix

Table 7: Seed-level main authoring results. Final best-so-far score

S_{\max}@100

for all three random seeds. For each method–task pair, the main paper reports the run with the median final score; Figure 3 shows the complete trajectory of the corresponding representative run. The final column gives the sample standard deviation across seeds. Root

\mathcal{S}^{(0)}

and Independent Sampling are fixed references reported in Table 1 .

Configuration	VSD	AS	COPD	AF	LQT	WPW
SimAuthor	.587/.448/.611	.429/.559/.557	.672/.521/.689	.748/.738/.738	.541/.568/.594	.581/.543/.653
$-$ Report	.417/.366/.270	.152/.452/.152	.603/.609/.648	.638/.548/.589	.448/.460/.463	.491/.497/.492
$-$ Library	.399/.409/.283	.383/.447/.287	.662/.704/.540	.719/.725/.703	.627/.540/.488	.573/.555/.623

Appendix

Table 8: Seed-level focused ablations. Each cell reports Seed 1 / Seed 2 / Seed 3. (a) Final best-so-far score

S_{\max}@100

. (b) Normalized

\mathrm{AUC}_{100}

of the best-so-far trajectory. LQT and WPW use the final ECGFounder representation-space evaluator.

Method	VSD	AS	COPD	AF	LQT	WPW
Root $\mathcal{S}^{(0)}$	.2229	.1027	.0875	.1044	.2771	.3848
Sampling	.4246	.1460	.3624	.1173	.3408	.4433
SimAuthor seed 1	.5771	.4264	.6109	.2040	.4926	.4965
SimAuthor seed 2	.4093	.4402	.4095	.2428	.4245	.5379
SimAuthor seed 3	.5786	.4737	.5072	.1813	.4804	.6155
SimAuthor median	.5771	.4402	.5072	.2040	.4804	.5379

Appendix

Table 9: Seed-level held-out evaluation. Each SimAuthor row evaluates the search-score-best simulator from one independent authoring seed; held-out recordings are never used for node or seed selection. Root and best-of-100 sampling are fixed references. The final row reports the column-wise median across the three SimAuthor runs and gives the values reported in Table 4 .

Method	Calls	Input (k)	Output (k)	Total (k)
Independent Sampling	100.0	$46\pm 1$	$185\pm 22$	231
PUCT Score Search	100.0	$314\pm 65$	$270\pm 68$	584
SimAuthor	$106.4\pm 1.4$	$634\pm 68$	$244\pm 71$	878

Appendix

Table 10: Compute per 100-attempt run. Mean

\pm

s.d. across conditions and completed seeds. Token counts are estimated from character counts and exclude image tokens.

	VSD	AS	COPD	AF	LQT	WPW
Best-of-100	.477	.184	.391	.492	.418	.456
Best-of-380	.479	.298	.475	.530	.505	.458
index of best	118	371	254	324	313	338
SimAuthor	.587	.557	.672	.738	.568	.581

Appendix

Table 11: Independent sampling at a token-matched budget. Best-of-

N

over an approximately 380-program independent pool, compared with SimAuthor at its standard 100-attempt budget. “Index of best” gives the position of the highest-scoring program within the sampling sequence.

	LQT		WPW
Training	ID	OOD	ID	OOD
Real	$.965$	$.797$	$.951$	$.926$
+ Conv.	$.968$	$.821$	$.953$	$.926$
+ SimAuthor	$\mathbf{.974}$	$\mathbf{.889}$	$\mathbf{.973}$	$\mathbf{.965}$

Appendix

Table 12: AUROC for downstream ECG classification. All conditions use frozen ECGFounder representations and the same class-weighted logistic-regression probe. “+ Conv.” and “+ SimAuthor” augment the real training set with 300 positive examples. The SimAuthor program is selected using only its representation-space authoring score across three independent authoring runs; downstream performance is never used for selection. ID denotes patient-disjoint PTB-XL and OOD denotes Chapman–Shaoxing. Bold indicates the best result in each column.

Explore similar work

Jun 3, 2026cs.AI

Simulate, Reason, Decide: Scientific Reasoning with LLMs for Simulation-Driven Decision Making

Scientific simulators are increasingly being integrated into LLM-driven systems for high-stakes simulation-driven decision-making. However, existing frameworks primarily use LLMs to generate, calibrate, or execute simulators, treating them as black-box interfaces rather than as structured mechanistic systems that can be reasoned about. As a result, current approaches lack the ability to identify, represent, and reason about the assumptions and mechanisms underlying simulator behavior, limiting transparency, auditability, and decision justification. We introduce MechSim, a mechanism-grounded neuro-symbolic reasoning framework for executable scientific simulators. Unlike prior neuro-symbolic approaches that primarily reason over static symbolic structures, MechSim enables LLM agents to reason about the mechanisms, assumptions, and execution behavior of scientific simulators. Our framework represents simulators through a shared structured schema capturing assumptions, variables, mechanism dependencies, and execution traces. On top of this representation, LLM agents operate as constrained reasoning engines that generate structured, evidence-grounded explanations linking simulator outcomes to their underlying mechanisms. We evaluate our approach across multiple high-stakes domains and show that it improves mechanism-level explanation quality, simulator analysis, and downstream decision-making reliability.

Yuhan Yang, Ruipu Li, Alexander Rodríguez

Computer Science and Engineering University of Michigan

Neuro-Symbolic Framework Decisions

Ask PDF

Jun 8, 2026cs.AI

Auto-Configuring Scientific Simulators with Lightweight Coding-Agent Adapters

Configuring an advanced scientific simulator, translating a modeling goal into a valid, runnable input deck, is a persistent bottleneck that costs domain scientists hours to days. Input decks are executable interfaces: simulator-specific vocabulary, cross-file references, schema constraints, and validation rules must align before a simulation can run. We show that this bottleneck can be substantially reduced with a lightweight adapter around an off-the-shelf coding agent, rather than a bespoke simulator agent. Coding agents already navigate files, edit code, run commands, and repair outputs; what they lack is the simulator's executable contract, and rebuilding the agent loop risks discarding harness-calibrated tool-use and self-correction behavior. We introduce SIGA, a coding-agent adapter that supplies this contract through retrieval, procedural memory, agent-callable validation, and validation-gated termination while leaving the model and loop frozen. Because this contract is small and external, SIGA also supports adapter self-evolution: prior trajectories can rewrite the adapter contents without modifying the underlying agent. On GEOS, a multiphysics subsurface simulator, SIGA's main gain is reliability: on harder held-out tasks it improves TreeSim from 0.720 to 0.789 and reduces across-run standard deviation by about 16x by preventing empty or invalid decks. In a human calibration, SIGA reaches in about five minutes the deck quality a domain expert reached in about three hours. Transfers to OpenFOAM and LAMMPS show the recipe is portable but interface-dependent: completion gates help when structural completeness is the bottleneck, while memory and retrieval help when value correctness is.

Matthew Ho, Brian Liu, Jixuan Chen +2

University of California, San Diego

Coding Agents Lightweight Semantic Memory

Ask PDF

Apr 19, 2026cs.AI

SOCIA-EVO: Automated Simulator Construction via Dual-Anchored Bi-Level Optimization

Automated simulator construction requires distributional fidelity, distinguishing it from generic code generation. We identify two failure modes in long-horizon LLM agents: contextual drift and optimization instability arising from conflating structural and parametric errors. We propose SOCIA-EVO, a dual-anchored evolutionary framework. SOCIA-EVO introduces: (1) a static blueprint to enforce empirical constraints; (2) a bi-level optimization to decouple structural refinement from parameter calibration; and (3) a self-curating Strategy Playbook that manages remedial hypotheses via Bayesian-weighted retrieval. By falsifying ineffective strategies through execution feedback, SOCIA-EVO achieves robust convergence, generating simulators that are statistically consistent with observational data. The code and data of SOCIA-EVO are available here: https://github.com/cruiseresearchgroup/SOCIA/tree/evo.

Yuncheng Hua, Sion Weatherhead, Mehdi Jafari +2

School of Computer Science and Engineering, University of New South Wales, Australia

Alphaevolve Simulation-Based Reinforcement Learning Code Generation Distributional Fidelity Bilevel Optimization

Ask PDF

	OPERA-CE			PaPaGei
Method	VSD	AS	COPD	AF
Root $\mathcal{S}^{(0)}$	$.317\,\|\,.317$	$.192\,\|\,.192$	$.152\,\|\,.152$	$.354\,\|\,.354$
Sampling	$.123\,\|\,.715$	$.106\,\|\,.612$	$.162\,\|\,.799$	$.434\,\|\,\mathbf{.830}$
Text-Opt	$\mathbf{.443}\,\|\,.784$	$.258\,\|\,.685$	$.468\,\|\,\mathbf{.847}$	$.631\,\|\,.734$
SimAuthor	$.204\,\|\,\mathbf{.789}$	$\mathbf{.306}\,\|\,\mathbf{.906}$	$\mathbf{.561}\,\|\,.752$	$\mathbf{.665}\,\|\,.737$