Reliability by Architecture: Agentic Workflows for Validated Multi-Physics Simulation from Natural Language
Authors: Søren Taverniers, Eric Darve, Hamdi A. Tchelepi
Organizations: Department of Mechanical Engineering, Stanford University, Stanford, CA 94305, USA · Institute for Computational and Mathematical Engineering, Stanford University, Stanford, CA 94305, USA · Department of Energy Science & Engineering, Stanford University, Stanford, CA 94305, USA
Preparing, executing, and validating strongly coupled multi-physics simulations require expert time. Agentic workflows use large language models (LLMs) to coordinate them from natural-language descriptions, but reliability requires checks of physical consistency and fidelity to the problem description. Our agentic harness implements explicit task allocation: LLMs interpret user requests and simulator diagnostics, generate input components not covered by predefined construction rules, and propose repairs, while deterministic routines construct supported components and check inputs, edits, and results. We evaluate CO2-storage problems using the multi-physics simulator GEOS, from compositional flow to flow-poromechanics with frictional fault contact. In 80 attempts on two flow problems, grounding raises the proportion of structurally valid inputs from 5/20 to 20/20, but acceptance reaches 20/20 only with complete rule-based input preparation. With identical inputs, local repairs correct three seeded defects while preserving valid inputs. Supporting tests identify simulations passing physics checks despite differing from the problem description or reference results.
Figures & tables
Figure 1: Agentic harness coordination and study acceptance. (a) The dotted boundary encloses LangGraph nodes. Gray double-headed arrows summarize coordination and shared-state exchange. Deck preparation includes pre-solve admissibility and input-fidelity checks. Postprocessing includes physics checks and applicable input- and output-fidelity checks. Fidelity checks compare inputs and outputs with the user request. The escalation checkpoint manages pausing and resumption for expert review outside the agentic harness. Red dashed arrows show optional repair after failed postprocessing checks through the orchestrator, including physics and fidelity failures, and runtime-error repair from execution to deck preparation. (b) Study acceptance requires satisfying all listed conditions, including any additional fidelity checks performed outside the LangGraph workflow as part of the study. Panel (b) shows assessment logic and outcomes, not additional LangGraph nodes ( Sections 2.2.3 , 2.4.1 and 2.4.2 ).
Stage
LLM contribution
Deterministic routines
Retrieve and interpret
Use repository browsing and search tools to find reference decks when embedding retrieval is unavailable or returns weak matches
Retrieve decks by embedding similarity; rank candidates against the requested physics; assemble current-schema context
Mesh and deck construction
Choose geometry strategy and tools; write components not synthesized by deterministic code
Size the mesh; map regions and materials; convert units; correct links between deck objects; construct input components from requested parameters using predefined rules
Physical setup
Extract requested parameters
Calculate initial states and controls; make referenced input files available to the simulator; check admissibility and input fidelity
Runtime repair
Diagnose the failure from the supplied log excerpts and propose a local edit when no prescribed correction applies
Select error messages and surrounding log lines for LLM diagnosis; apply specified corrections; check schema compliance and preservation of required, previously valid deck content; enforce retry limits
Execution and postprocessing
Optionally generate a narrative summary of the simulation results and the outcomes of the post-solve checks
Invoke GEOS; extract execution information and numerical quantities to check completion and apply physics, input-fidelity, and output-fidelity checks
Table 1: Allocation of responsibilities between LLMs and deterministic routines in the agentic harness surrounding GEOS. The simulator numerically solves the governing equations.
Check
Question
Example
Schema validation
Does the deck use the accepted input structure?
Required elements and allowed attributes
Pre-solve admissibility
Are values and component relationships physically and numerically admissible?
Does the computed behavior satisfy the applicable physics checks?
Increase in stored CO 2 matches the injected mass
Output fidelity
Do saved results satisfy the requested output requirements and agree with any specified reference?
CO 2 inventory after injection stops and requested simulation duration; baseline-trajectory agreement
Table 2: Questions answered by the validation stages. Input- and output-fidelity checks assess agreement with the request or a specified reference, rather than physical consistency.
Figure 2: Simulation domains of the four scenarios simulated in GEOS, as X – Z cross-sections (not to a common scale; all cases place a storage aquifer between seals, Table 7 ). Three principal evaluation designs reuse these scenarios, as summarized in Table 3 . (a) The flow cases share a layered aquifer between caprock and base seals. T1 starts with an emplaced mobile free-gas CO 2 plume whose upward migration is checked, whereas T2 starts with 100% brine and uses a bottom-hole-pressure-controlled injector/producer pair. The depicted initial plume belongs to T1 and the wells to T2. (b) The coupled poromechanics case (T3) adds two steeply dipping ( ±75∘ ) continuum fault zones that compartmentalize the domain, with source-driven injection. (c) The SPE11C-inspired contact case (T4) [ 16 ] , on a shallower section with a larger width-to-depth ratio, treats three steep faults as discrete frictional surfaces, using Lagrange multipliers to enforce contact constraints and a friction coefficient of μ=0.6 . Injection uses two volumetric SourceFlux regions; the vertical axis is exaggerated for legibility. Colors: sand = storage reservoir/aquifer, blue-gray = seal, gray = fault, orange = wells or sources, blue = emplaced CO 2 .
Evaluation
Inputs and comparison
Question
Generated-deck flow ablation
Plume (T1) and two-well flow (T2); four configurations, 10 attempts per case and configuration
Which setup components increase accepted yield?
Contact capability demonstration
Natural-language description of flow–poromechanics with frictional contact (T4)
Can the workflow construct, execute, and validate this coupled setup?
Paired supplied-deck repair
Two-well flow (T2) and continuum flow–poromechanics (T3); identical valid or defective decks with repair disabled and enabled
Can a local correction restore execution while preserving the required setup?
Table 3: The three principal evaluations. Physical scenarios are reused to answer different questions; generated and supplied inputs are kept distinct.
Question
Test
Evidence sought
How closely does a coupled solver configuration reproduce an analytical solution?
Single-phase Terzaghi consolidation, with separate mesh and timestep refinement
Pressure, displacement, and settlement errors from direct GEOS runs, without agents
Can a mass-balanced result violate the requested schedule?
Injection followed by monitoring
Input schedule and saved inventory compared with the request
What can output checks miss?
Seven input defects, with pre-solve checks bypassed
Differences between output detection, input admissibility, and fidelity of material assignment
Do the implemented check functions detect their intended violations?
Direct physics-gate check-library test
Detection and false alarms without simulation-file extraction
How closely do extracted inventories agree with published reference results?
SPE11B reference comparison
CO 2 inventory histories compared with the official geos1 submission
Does passing physics checks imply agreement with a reference trajectory?
Continuum numerical-parameter test
Physics verdicts compared with baseline-trajectory differences
Table 4: Supporting validation tests and the questions they examine. The tests distinguish physics checks, fidelity to the request, and workflow responses to failures.
Configuration
Schema-valid
Accepted
Combined accepted
(both cases)
Plume
Two wells
yield [95% CI]
Ungrounded generation
5/20
0/10
0/10
0/20 [0–16.1%]
+ grounding
20/20
0/10
0/10
0/20 [0–16.1%]
+ normalization
20/20
0/10
0/10
0/20 [0–16.1%]
+ synthesis
20/20
10/10
10/10
20/20 [83.9–100%]
Table 5: Flow-ablation outcomes, with 10 attempts per benchmark and configuration. Rows successively add components. Schema validity is assessed after generation and its allowed internal retries; acceptance additionally requires simulation completion, the physics gate, and input- and output-fidelity checks. Intervals are two-sided 95% Wilson intervals.
Figure 3: Effects of grounding, synthesis, and normalization. Each ‘without’ or ‘with’ setting includes 20 runs (10 per flow benchmark); error bars are two-sided 95% Wilson intervals. Panels (a) and (b) report the generated-deck flow ablation. Panel (c) reports a separate comparison that reruns the same 20 decks saved from that ablation. Panel (a) measures schema validity; panels (b) and (c) measure final acceptance. The enabled components are shown below each panel, and runtime repair is disabled.
Input
No repair
Repair enabled
Accepted edit or outcome
Valid two-well flow deck
pass
pass
No repair edit
Broken well-control reference
Non-pass
pass
One attribute
Missing relative-permeability table
Non-pass
pass
One table added
Valid continuum flow–poromechanics deck
pass
pass
No repair edit
Broken solid-solver reference
Non-pass
pass
One attribute
Table 6: Matched supplied-deck repair tests. Each pair starts from identical inputs. pass requires both the physics-gate checks and all required input- and output-fidelity checks to pass.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Case
Domain (km)
Added physics & structures
Drive
Horizon
T1 flow-only
10×10×5
compositional CO 2 –brine, isothermal 50 ∘ C; no wells/faults
emplaced free-gas plume ( Sg≈0.6 )
365 d
T2 two-well flow
10×10×5
CO 2 –brine, isothermal 50 ∘ C; initially 100% brine; injector & producer under bottom-hole-pressure (BHP) control
well-pair sweep
365 d
T3 +poromech. & faults
10×10×5
poromechanics; two continuum fault zones; regional-constant materials
SourceFlux : 500 million kg over 730 d, then 182.5 d monitoring
912.5 d
T4 +contact
8.4×5×1.2
frictional Lagrange contact on three faults ( μ=0.6 , cohesion 2 MPa), fixed fracture storage
two SourceFlux regions, 30 d injection + 15 d monitoring
45 d
SPE11B
8.4×0.01×1.2
thermal compositional flow; layered facies; no wells/faults
two SourceFlux sources, each 0.035 kg/s: years 0–50 and 25–50
1000 yr
Appendix
Table 7: Physical specifications of the four GEOS scenarios and the SPE11B reference comparison. The scenarios are reused across evaluation designs as mapped in Table 3 . SPE11B is an external reference comparison ( Section 4.4 ). “Drive” is the mechanism that mobilizes CO 2 ; horizons are the simulated physical times. All domains use ≲ 100,000 cells (T4 uses 22,280 tetrahedra).
Cells
Maximum timestep (s)
Pressure (%)
Displacement (%)
Settlement (%)
40
0.25
1.052
3.315
2.673
80
0.25
0.993
3.089
2.387
80
0.0625
0.334
1.070
0.820
Appendix
Table 8: Analytical consolidation errors, maximized over the four sampled times, not over the complete trajectory. Both flow methods give the values shown to the reported precision.
Constitutive (Biot bounds, PVT temperature range, fault slip)
3
3
Coupling (poroelastic uplift sign, injectivity)
2
2
Numerical (solver convergence, stress finiteness)
2
2
Total
15
15
Appendix
Table 9: Direct check-library test: introduced violations and detections by category. The unchanged baseline raises no alarm among its 14 applicable checks.
Seeded defect
Direct solve + post-solve physics gate
Negative Young’s modulus
aborts at initialization
Poisson ratio 0.95 (inadmissible)
aborts at initialization
Solver references an undefined region
aborts at initialization
Grain modulus below drained modulus
aborts at initialization
Reference porosity 1.8
execution completes; gate fail
Caprock at storage-zone permeability (no longer a seal)
execution completes; gate pass —output matches the baseline run
Appendix
Table 10: Seven input defects tested by direct execution and the post-solve physics gate, with pre-solve checks bypassed. A gate pass here does not imply workflow acceptance.
Figure 4: SPE11B reference comparison. (a) Layered facies, reporting boxes A and B, and the two CO 2 injection locations in the SPE11B domain, shown as an 8.4km×1.2km cross-section. (b) Mobile CO 2 in box A from the rerun and official geos1 submission. Table 11 also reports mobile CO 2 in box B, immobile and dissolved CO 2 in both boxes, and CO 2 in the seal within box A.
Quantity
Box A
Box B
Mobile CO 2
0.016 (1.00)
0.901 (1.34)
Immobile CO 2
0.504 (1.10)
0.700 (1.40)
Dissolved CO 2
1.113 (2.18)
1.298 (2.39)
CO 2 in seal
1.677 (3.24)
—
Mean rel. L2 (7 QoIs)
0.887
Appendix
Table 11: SPE11B inventory comparison against the official geos1 submission. Entries are relative trajectory L2 errors; parentheses give the rerun value at 1000 years divided by the reference value at the same time. The mean covers the seven reported quantities.
Outcome
Count
Rate
95% Wilson CI
Target contradiction identified
12/12
100%
75.8 – 100%
Valid original approved unchanged
1/12
8.3%
1.5 – 35.4%
Defect correction accepted after checks
7/12
58.3%
—
Valid original routed to human review
11/12
91.7%
—
Appendix
Table 12: Held-out evaluation of the optional critic on 12 matched pairs of valid and deliberately defective inputs. Sensitivity is target-specific defect detection; baseline specificity is the proportion of valid originals approved unchanged. Intervals are two-sided 95% Wilson intervals.
Stage
Category
Representative checks
Pre-solve
Admissibility
Permeability and elastic-modulus bounds; relative-permeability endpoint consistency; injector BHP above formation pressure and below the estimated fracture pressure
Post-solve
Conservation
Global CO 2 inventory balance against estimated injection and production; CO 2 mass monotonicity during injection
Field bounds
Positive absolute pressure and fluid density; saturation and component-fraction bounds and sums; porosity and Biot bounds; PVT temperature range and thermal consistency
Containment
Free gas in low-permeability seals, or missing seal contrast in a multi-region case containing free gas
Coupling response
Injection-zone pressure response considered with mass delivery; sign of process-induced vertical displacement
Numerical checks
Run completion and convergence diagnostics; finite stress and fault displacement jumps (legacy fallback: fault-region displacement)
Appendix
Table 13: Representative pre-solve admissibility and post-solve physics-gate checks. Applicability depends on the problem profile. Input- and output-fidelity checks assess agreement with the request or a specified reference.
LLM agents are increasingly used for code generation, but physics-based simulation poses a deeper challenge: natural-language descriptions of simulation models are inherently underspecified, and different admissible resolutions of implicit choices produce physically valid but scientifically distinct configurations. Without explicit detection and resolution of these ambiguities, neither the correctness of the result nor its reproducibility from the original description can be assured. This paper investigates agentic scientific simulation, where model construction is organized as an execution-grounded interpret-act-validate loop and the simulator serves as the authoritative arbiter of physical validity rather than merely a runtime. We present JutulGPT, a reference implementation built on the fully differentiable Julia-based reservoir simulator JutulDarcy. The agent combines structured retrieval of documentation and examples with code synthesis, static analysis, execution, and systematic interpretation of solver diagnostics. Underspecified modelling choices are detected explicitly and resolved either autonomously (with logged assumptions) or through targeted user queries. The results demonstrate that agent-mediated model construction can be grounded in simulator validation, while also revealing a structural limitation: choices resolved tacitly through simulator defaults are invisible to the assumption log and to any downstream representation. A secondary experiment with autonomous reconstruction of a reference model from progressively abstract textual descriptions shows that reconstruction variability exposes latent degrees of freedom in simulation descriptions and provides a practical methodology for auditing reproducibility. All code, prompts, and agent logs are publicly available.
Knut-Andreas Lie, Olav Møyner, Elling Svee +1
SINTEF Digital, Oslo, Norway · Department of Mathematical Sciences, NTNU, Trondheim, Norway
Large language model (LLM) agents are increasingly entrusted with natural-language workflow instructions (e.g., retail-payment policies) that specify not only what outcome to achieve, but also which steps, branches, and tool interactions are permitted. When these instructions are supplied as prompt context, however, the model retains control over both procedure selection and step execution. As interactions accumulate, an agent can skip required steps, take unsupported branches, or execute a valid step with unsupported arguments or effects--a failure mode we call workflow misalignment. In this work, we propose COVENANT, a compiler-and-interpreter architecture for workflow-aligned agent execution. Our key insight is to treat workflow instructions as source programs rather than prompts. COVENANT converts the instructions into a workflow abstract syntax tree (WAST) and lowers it to a workflow control-flow graph (WCFG). At runtime, a controller interprets the WCFG one node at a time, checks each proposal against requirements extracted from the instructions before committing controller state or advancing the graph, and returns diagnostic feedback for repair. To evaluate COVENANT, we use 120 cases from three existing benchmarks, spanning seven workflow scenarios. Compared with state-of-the-art LLM agents, COVENANT improves benchmark success from 50.00% to 83.33% and reduces the workflow-misalignment failure rate from 42.50% to 15.83% (62.75% relative). These results show that COVENANT substantially mitigates workflow misalignment, moving LLM-agent alignment beyond isolated prompt following toward reliable execution of complex and multi-step workflows.
Equipping Large Language Models (LLMs) to execute reliable multi-step workflows has become a central challenge in artificial intelligence. Despite recent advances in LLMs' agentic capabilities, most agent systems still lack formal methods for specifying, verifying, and debugging their workflow and execution trajectories. This challenge mirrors a long-standing problem in mathematics, where the ambiguity of natural languages (NLs) motivates the development of formal languages (FLs). Inspired by this paradigm, we propose Lean4Agent, to the best of our knowledge, the first framework that uses Lean4, a dependent-type FL to model and verify agent behavior. Lean4Agent launches FormalAgentLib, an extensible Lean4 library for formally modeling and verifying agent workflows' semantic consistency under explicit assumptions, and enabling localization of execution-time failures revealed by trajectories. Building on FormalAgentLib, we further develop LeanEvolve, which applies results in FormalAgentLib to revise workflows to enhance its capability. Extensive experiments on a hard problem subset of SWE-Bench-Verified and a subset of ELAIP-Bench across 5 leading LLMs indicate that the verification-passing workflows outperform the failing ones by an average of 11.94%, and LeanEvolve further improves SWE performance by 7.47% on average. Furthermore, Lean4Agent establishes a foundation for a new field of using expressive dependent-type FL to formally model and verify agent behavior.
Ruida Wang, Jerry Huang, Pengcheng Wang +3
University of Illinois Urbana-Champaign · Independent researcher