Reliability by Architecture: Agentic Workflows for Validated Multi-Physics Simulation from Natural Language
Organizations: Department of Mechanical Engineering, Stanford University, Stanford, CA 94305, USA · Institute for Computational and Mathematical Engineering, Stanford University, Stanford, CA 94305, USA · Department of Energy Science & Engineering, Stanford University, Stanford, CA 94305, USA
Abstract
Preparing, executing, and validating strongly coupled multi-physics simulations require expert time. Agentic workflows use large language models (LLMs) to coordinate them from natural-language descriptions, but reliability requires checks of physical consistency and fidelity to the problem description. Our agentic harness implements explicit task allocation: LLMs interpret user requests and simulator diagnostics, generate input components not covered by predefined construction rules, and propose repairs, while deterministic routines construct supported components and check inputs, edits, and results. We evaluate CO2-storage problems using the multi-physics simulator GEOS, from compositional flow to flow-poromechanics with frictional fault contact. In 80 attempts on two flow problems, grounding raises the proportion of structurally valid inputs from 5/20 to 20/20, but acceptance reaches 20/20 only with complete rule-based input preparation. With identical inputs, local repairs correct three seeded defects while preserving valid inputs. Supporting tests identify simulations passing physics checks despite differing from the problem description or reference results.
Figures & tables
| Stage | LLM contribution | Deterministic routines |
|---|---|---|
| Retrieve and interpret | Use repository browsing and search tools to find reference decks when embedding retrieval is unavailable or returns weak matches | Retrieve decks by embedding similarity; rank candidates against the requested physics; assemble current-schema context |
| Mesh and deck construction | Choose geometry strategy and tools; write components not synthesized by deterministic code | Size the mesh; map regions and materials; convert units; correct links between deck objects; construct input components from requested parameters using predefined rules |
| Physical setup | Extract requested parameters | Calculate initial states and controls; make referenced input files available to the simulator; check admissibility and input fidelity |
| Runtime repair | Diagnose the failure from the supplied log excerpts and propose a local edit when no prescribed correction applies | Select error messages and surrounding log lines for LLM diagnosis; apply specified corrections; check schema compliance and preservation of required, previously valid deck content; enforce retry limits |
| Execution and postprocessing | Optionally generate a narrative summary of the simulation results and the outcomes of the post-solve checks | Invoke GEOS; extract execution information and numerical quantities to check completion and apply physics, input-fidelity, and output-fidelity checks |
| Check | Question | Example |
|---|---|---|
| Schema validation | Does the deck use the accepted input structure? | Required elements and allowed attributes |
| Pre-solve admissibility | Are values and component relationships physically and numerically admissible? | Positive solid density; compatible solver-region assignments |
| Input fidelity | Does the assembled setup agree with the request? | Requested injection rate and stopping time |
| Post-solve physics gate | Does the computed behavior satisfy the applicable physics checks? | Increase in stored CO 2 matches the injected mass |
| Output fidelity | Do saved results satisfy the requested output requirements and agree with any specified reference? | CO 2 inventory after injection stops and requested simulation duration; baseline-trajectory agreement |
| Evaluation | Inputs and comparison | Question |
|---|---|---|
| Generated-deck flow ablation | Plume (T1) and two-well flow (T2); four configurations, 10 attempts per case and configuration | Which setup components increase accepted yield? |
| Contact capability demonstration | Natural-language description of flow–poromechanics with frictional contact (T4) | Can the workflow construct, execute, and validate this coupled setup? |
| Paired supplied-deck repair | Two-well flow (T2) and continuum flow–poromechanics (T3); identical valid or defective decks with repair disabled and enabled | Can a local correction restore execution while preserving the required setup? |
| Question | Test | Evidence sought |
|---|---|---|
| How closely does a coupled solver configuration reproduce an analytical solution? | Single-phase Terzaghi consolidation, with separate mesh and timestep refinement | Pressure, displacement, and settlement errors from direct GEOS runs, without agents |
| Can a mass-balanced result violate the requested schedule? | Injection followed by monitoring | Input schedule and saved inventory compared with the request |
| What can output checks miss? | Seven input defects, with pre-solve checks bypassed | Differences between output detection, input admissibility, and fidelity of material assignment |
| Do the implemented check functions detect their intended violations? | Direct physics-gate check-library test | Detection and false alarms without simulation-file extraction |
| How closely do extracted inventories agree with published reference results? | SPE11B reference comparison | CO 2 inventory histories compared with the official geos1 submission |
| Does passing physics checks imply agreement with a reference trajectory? | Continuum numerical-parameter test | Physics verdicts compared with baseline-trajectory differences |
| Configuration | Schema-valid | Accepted | Combined accepted | |
|---|---|---|---|---|
| (both cases) | Plume | Two wells | yield [95% CI] | |
| Ungrounded generation | [0–16.1%] | |||
| grounding | [0–16.1%] | |||
| normalization | [0–16.1%] | |||
| synthesis | [83.9–100%] | |||
| Input | No repair | Repair enabled | Accepted edit or outcome |
|---|---|---|---|
| Valid two-well flow deck | pass | pass | No repair edit |
| Broken well-control reference | Non-pass | pass | One attribute |
| Missing relative-permeability table | Non-pass | pass | One table added |
| Valid continuum flow–poromechanics deck | pass | pass | No repair edit |
| Broken solid-solver reference | Non-pass | pass | One attribute |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Case | Domain (km) | Added physics & structures | Drive | Horizon |
|---|---|---|---|---|
| T1 flow-only | compositional CO 2 –brine, isothermal 50 ∘ C; no wells/faults | emplaced free-gas plume ( ) | 365 d | |
| T2 two-well flow | CO 2 –brine, isothermal 50 ∘ C; initially 100% brine; injector & producer under bottom-hole-pressure (BHP) control | well-pair sweep | 365 d | |
| T3 +poromech. & faults | poromechanics; two continuum fault zones; regional-constant materials | SourceFlux : million kg over 730 d, then 182.5 d monitoring | 912.5 d | |
| T4 +contact | frictional Lagrange contact on three faults ( , cohesion 2 MPa), fixed fracture storage | two SourceFlux regions, 30 d injection + 15 d monitoring | 45 d | |
| SPE11B | thermal compositional flow; layered facies; no wells/faults | two SourceFlux sources, each 0.035 kg/s: years 0–50 and 25–50 | 1000 yr |
| Cells | Maximum timestep (s) | Pressure (%) | Displacement (%) | Settlement (%) |
|---|---|---|---|---|
| 40 | 0.25 | 1.052 | 3.315 | 2.673 |
| 80 | 0.25 | 0.993 | 3.089 | 2.387 |
| 80 | 0.0625 | 0.334 | 1.070 | 0.820 |
| Check category | Seeded errors | Caught |
|---|---|---|
| Admissibility (saturation, pressure, composition, porosity, density) | 6 | 6 |
| Conservation (CO 2 mass balance, monotonicity) | 2 | 2 |
| Constitutive (Biot bounds, PVT temperature range, fault slip) | 3 | 3 |
| Coupling (poroelastic uplift sign, injectivity) | 2 | 2 |
| Numerical (solver convergence, stress finiteness) | 2 | 2 |
| Total | 15 | 15 |
| Seeded defect | Direct solve post-solve physics gate |
|---|---|
| Negative Young’s modulus | aborts at initialization |
| Poisson ratio (inadmissible) | aborts at initialization |
| Solver references an undefined region | aborts at initialization |
| Grain modulus below drained modulus | aborts at initialization |
| Reference porosity | execution completes; gate fail |
| Caprock at storage-zone permeability (no longer a seal) | execution completes; gate pass —output matches the baseline run |
| Quantity | Box A | Box B |
|---|---|---|
| Mobile CO 2 | 0.016 (1.00) | 0.901 (1.34) |
| Immobile CO 2 | 0.504 (1.10) | 0.700 (1.40) |
| Dissolved CO 2 | 1.113 (2.18) | 1.298 (2.39) |
| CO 2 in seal | 1.677 (3.24) | — |
| Mean rel. (7 QoIs) | 0.887 | |
| Outcome | Count | Rate | 95% Wilson CI |
|---|---|---|---|
| Target contradiction identified | – | ||
| Valid original approved unchanged | – | ||
| Defect correction accepted after checks | — | ||
| Valid original routed to human review | — |
| Stage | Category | Representative checks |
|---|---|---|
| Pre-solve | Admissibility | Permeability and elastic-modulus bounds; relative-permeability endpoint consistency; injector BHP above formation pressure and below the estimated fracture pressure |
| Post-solve | Conservation | Global CO 2 inventory balance against estimated injection and production; CO 2 mass monotonicity during injection |
| Field bounds | Positive absolute pressure and fluid density; saturation and component-fraction bounds and sums; porosity and Biot bounds; PVT temperature range and thermal consistency | |
| Containment | Free gas in low-permeability seals, or missing seal contrast in a multi-region case containing free gas | |
| Coupling response | Injection-zone pressure response considered with mass delivery; sign of process-induced vertical displacement | |
| Numerical checks | Run completion and convergence diagnostics; finite stress and fault displacement jumps (legacy fallback: fault-region displacement) |
Agentic Workflows
Agentic workflows coordinate AI agents, tools, and humans across multi-step tasks, emphasizing reliable execution, efficiency, and oversight.
See all 233 papers