Human-guided physics-constrained AI agents construct an auditable model of soil-plug evolution

Jie Shi; Yimin Lu; Zhongkun Ouyang

cs.CESep 20, 2026

Human-guided physics-constrained AI agents construct an auditable model of soil-plug evolution

Authors: Jie Shi, Yimin Lu, Zhongkun Ouyang

Organizations: Institute of Ocean Engineering, Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China · Department of Civil, Environmental, and Construction Engineering, Texas Tech University, USA

Abstract

Engineering predictions require physical mechanisms to be translated consistently into equations, discretization, code, and validation, yet errors can propagate despite local checks. Artificial-intelligence (AI) agents automate scientific tasks, but coordinating and independently auditing the theory-to-solver process under physical constraints and human oversight remains unresolved. We introduce a human-in-the-loop, physics-constrained multi-agent workflow where human experts define admissible physics and modeling boundaries, while agents retrieve evidence, derive equations, implement solvers, and audit the theory-to-code chain. Applied to soil-plug evolution during suction-caisson installation, the workflow generated and audited 6 formulations in 2.9 h of agent execution once physical knowledge and inputs were prepared. Among these formulations, adding seepage-driven soil void-ratio evolution to the geometric baseline reduced mean absolute final-heave error from 58.4% to 9.0% across 14 profiles; the selected model further incorporated near-wall dilation and achieved mean absolute percentage errors of 12.4% across 9 final-state cases and 4.2% at the endpoints of 5 process histories. Beyond predictive performance, blinded replay recovered all 9 target problems, while an independent audit uncovered 5 implementation problems after 36 predefined checks had passed. Overall, this work extends multi-agent AI beyond task automation toward human-governed engineering solvers.

Figures & tables

Figure 1: The theory-to-solver bottleneck and its demonstration problem. (a) Model development in physics-intensive engineering traverses interdependent stages from problem definition and mechanism selection to governing equations, constitutive closure, discretization, code implementation, verification, and benchmark evaluation. Representative applications illustrate the associated expertise, time, and error-traceability challenges. (b) Soil-plug evolution during suction-caisson installation in sand provides the demonstration problem. The hidden response couples large deformation and a moving boundary, suction-induced seepage, effective-stress and void-ratio evolution, and soil–caisson interface behavior.

Version	Diagnostic question	Workflow action	Screening outcome
V1	Can geometric displacement caused by skirt penetration alone explain plug heave?	Established a geometric baseline from solid-volume conservation and suction-caisson design relations [ 5 ] .	Produced the largest final-heave and process final-point errors among the 6 versions.
V2	How does suction-driven seepage change effective stress and void ratio?	Added source-supported relations linking seepage, effective stress, and void ratio to update the soil state [ 32 , 33 , 34 ] .	Both errors decreased only slightly relative to V1.
V3	How should lateral effective stress and soil rebound evolve during confined unloading?	Updated lateral and mean effective stresses using relations for earth pressure at rest, stress history, and granular-soil elasticity [ 35 , 36 , 37 ] .	Both errors decreased only modestly relative to V2.
V4	Does stress-path evolution produce additional void-ratio change in dense sand?	Introduced a stress-path-dependent void-ratio update based on state-parameter and state-dependent dilatancy relations [ 34 , 38 , 39 ] .	Produced the first substantial reduction in both screening errors relative to V3.
V5	Can a more compact void-ratio update improve both final-state and process-level predictions?	Tested an alternative void-ratio update under the same parameter-source rules.	Reduced the process final-point error but increased the final-heave error.
V6	Can the admitted mechanisms be assembled to keep both final-state and process errors low?	Combined the audited seepage and void-ratio relations with near-wall soil dilation and consistent final-state updating [ 40 , 15 , 6 ] .	Retained a low final-heave error and achieved the lowest process final-point error; V6 was selected.

Table 1: Workflow-guided development and historical screening of candidate soil-plug formulations.

Figure 2: Historical screening of candidate soil-plug formulations. All 6 formulations were evaluated using the same benchmark cases and parameter-source rules. (a) Final-heave mean absolute percentage error (MAPE) across 9 final-state cases. (b) Process final-point MAPE across 5 measured heave–depth histories. (c) Joint evolution of the two errors across successive formulations; the inset enlarges the differences among V4–V6. Lower values indicate closer agreement with the benchmark observations.

Version	Derivation (min)	Implementation (min)	Independent audit (min)	Total (min)	Tokens (million)	API equivalent cost (US$)
V1	4.1	8.5	6.7	19.3	2.96	2.03
V2	10.8	17.6	7.4	35.7	5.14	3.82
V3	6.0	8.6	7.3	21.9	8.79	5.57
V4	5.8	21.2	10.5	37.5	15.44	8.70
V5	6.3	7.5	7.4	21.2	18.01	9.72
V6	9.4	19.1	7.3	35.8	22.72	12.39

Table 2: Agent execution time, token use, and estimated API-equivalent cost for the V1–V6 model-development.

Figure 3: Retrospective leave-one-study-out assessment of candidate-selection stability. The 6 candidate formulations were fixed before the analysis. In each fold, one study was excluded from candidate scoring and selection, and the selected formulation was subsequently evaluated against the held-out study. (a) Scores of V1–V6 under the primary endpoint-to-curve weighting of 75:25; boxes denote the selected lowest-scoring formulations. (b) Formulations selected using endpoint-to-curve error weights of 75:25, 50:50, and 25:75. (c) Held-out endpoint MAPE and curve normalized root-mean-square error (NRMSE) of the selected formulation.

Figure 4: Physical structure and coordinate definitions of the selected soil-plug formulation. The fixed coordinate $\mathchar 29049$ is measured downward from the initial seabed, and the local coordinate $\mathchar 29048$ is measured downward from the current plug surface. At penetration depth $\mathchar 29050$ , the plug length is $\mathchar 29004_{\mathrm{\mathchar 29040\mathchar 29036\mathchar 29045\mathchar 29031}}\mathchar 12349\mathchar 29050\mathchar 8235\mathchar 29032_{\mathrm{\mathchar 29032\mathchar 29029\mathchar 29025\delimiter 69640972\mathchar 29029}}$ . The initial material coordinate $\mathchar 28944_{\mathchar 28720}$ tracks the soil entering the internal control volume. $\mathchar 29005_{\mathchar 28999}$ , $\mathchar 29005_{\mathchar 29011}$ , and $\mathchar 29005_{\mathchar 28996}$ denote skirt-penetration-induced geometric displacement, seepage-driven soil void-ratio evolution, and near-wall soil dilation, respectively. Dashed blue curves schematically indicate the seepage paths.

Audit task	Observed result	Supported conclusion
Blinded replay of known issues	All 9 known issues were recovered and localized; no issues outside the target set, and no human assistance was provided during the scored replay.	The retained development records enabled an isolated audit role to recover and localize known issues under blinded conditions.
Derivation-to-code probe audit	All 36 predefined checks passed; the separate audit reported 5 additional findings across 4 of the 5 probes, with no finding in one probe.	The role-separated audit identified both an omitted predefined rule and implementation issues not covered by the predefined checks.
Final code verification	All 37 code tests passed, comprising 33 module-level tests and 4 regression tests; all 5 independently calculated reference checks also passed.	Within the checked scope, the final implementation followed the equations and calculation sequence described in the manuscript.

Table 3: Results of workflow audit and final code verification.

Figure 5: Staged benchmark evaluation of V6 plug-heave predictions. (a) Evidence map linking the available source data, benchmark problem, and supported claim at each evaluation level. (b) Formula reduction and volume closure for 7 cases using source-reported changes in void ratio or volumetric strain. The diagonal line represents equality between predicted and measured final heave, and the shaded region denotes the predefined ±10% comparison band. (c) Final state prediction for 9 benchmark cases. Black diamonds are measured final heave, blue points represent central predictions, and horizontal bands show the sensitivity range obtained by varying

\mathchar 29029_{\mathchar 28672}\delimiter 68408078\mathchar 29029_{\mathchar 29037\mathchar 29025\mathchar 29048}

from 0.9 to 1.1. Percentages report the signed errors of the central predictions. (d) Predicted and measured plug-heave–depth histories for 5 cases, together with the corresponding sensitivity bands and final-point errors. The bands in panels (c) and (d) represent prescribed parameter sensitivity rather than probabilistic confidence intervals.

Figure 6: Overview of the human-guided physics-constrained multi-agent workflow. Blue and orange dashed arrows indicate feedback on physical assumptions and implementation issues, respectively.

Section	Content
Supplementary Note 1	Model formulation and numerical solution procedure.
Supplementary Note 2	Design and findings of the two workflow audits.
Supplementary Note 3	Verification of the final solver through module-level, regression, and analytical reference tests.
Supplementary Note 4	Benchmark inputs, data sources, and comparison results.
Supplementary Data 1	Numerical data underlying the benchmark summaries and Fig. 5 of the main text.
Supplementary Data 2	Sources and uses of the global and case-specific model inputs.

Table 10

Symbol or mark	Definition	Unit or type
Geometry and coordinates
$\mathchar 28996_{\mathchar 29033}\mathchar 24891\mathchar 28996_{\mathchar 29039}\mathchar 24891\mathchar 29042_{\mathchar 29033}$	$\mathchar 28996_{\mathchar 29033}$ : inner caisson diameter; $\mathchar 28996_{\mathchar 29039}$ : outer caisson diameter; $\mathchar 29042_{\mathchar 29033}$ : inner caisson radius.	m
$\mathchar 28993_{\mathchar 29033}\mathchar 24891\mathchar 28939_{\mathchar 28993}$	$\mathchar 28993_{\mathchar 29033}$ : internal cross-sectional area; $\mathchar 28939_{\mathchar 28993}$ : outer-to-inner area ratio, $\delimiter 67273472\mathchar 28996_{\mathchar 29039}\delimiter 68408078\mathchar 28996_{\mathchar 29033}\delimiter 84054785^{\mathchar 28722}$ .	$\mathchar 28993_{\mathchar 29033}$ : m 2 ; $\mathchar 28939_{\mathchar 28993}$ : dimensionless
$\mathchar 29050\mathchar 24891\mathchar 29050_{\mathchar 29038}\mathchar 24891\mathchar 29050_{\mathchar 29038\mathchar 8235\mathchar 28721}$	$\mathchar 29050$ : penetration depth; $\mathchar 29050_{\mathchar 29038}$ : depth at the start of the current step; $\mathchar 29050_{\mathchar 29038\mathchar 8235\mathchar 28721}$ : depth at the end of the current step.	m
$\mathchar 28673\mathchar 29050\mathchar 24891\dot{\mathchar 29050}\mathchar 24891\mathchar 28673\mathchar 29044$	$\mathchar 28673\mathchar 29050$ : penetration increment; $\dot{\mathchar 29050}$ : penetration rate; $\mathchar 28673\mathchar 29044$ : elapsed time for one step.	$\mathchar 28673\mathchar 29050$ : m; $\dot{\mathchar 29050}$ : m s -1 ; $\mathchar 28673\mathchar 29044$ : s
$\mathchar 29032_{\mathrm{\mathchar 29032\mathchar 29029\mathchar 29025\delimiter 69640972\mathchar 29029}}\mathchar 24891\mathchar 29004_{\mathrm{\mathchar 29040\mathchar 29036\mathchar 29045\mathchar 29031}}\mathchar 24891\mathchar 29000$	$\mathchar 29032_{\mathrm{\mathchar 29032\mathchar 29029\mathchar 29025\delimiter 69640972\mathchar 29029}}$ : net plug heave; $\mathchar 29004_{\mathrm{\mathchar 29040\mathchar 29036\mathchar 29045\mathchar 29031}}$ : total plug length; $\mathchar 29000$ : domain height measured from the plug free surface, $\mathchar 29000\mathchar 12349\mathchar 29004_{\mathrm{\mathchar 29040\mathchar 29036\mathchar 29045\mathchar 29031}}\mathchar 12349\mathchar 29050\mathchar 8235\mathchar 29032_{\mathrm{\mathchar 29032\mathchar 29029\mathchar 29025\delimiter 69640972\mathchar 29029}}$ .	m

Table 11

Supplementary Fig. 1. Stepwise numerical solution and state transfer of the soil-plug model. Trial state denotes fields updated during the $\mathchar 29005_{\mathchar 28999}$ – $\mathchar 29005_{\mathchar 29011}$ iteration; converged state denotes the accepted solution before the $\mathchar 29005_{\mathchar 28996}$ correction. Committed state carries the accepted state and history into the next penetration step. Output state contains the post- $\mathchar 29005_{\mathchar 28996}$ refreshed fields stored for reporting and verification.

Check	Metric	Value
Component combination	Mean absolute final-heave error with $\mathchar 29005_{\mathchar 28999}$ only	58.42%
Component combination	Mean absolute final-heave error with $\mathchar 29005_{\mathchar 28999}$ and $\mathchar 29005_{\mathchar 29011}$	9.01%
Component combination	Mean absolute final-heave error with $\mathchar 29005_{\mathchar 28999}$ , $\mathchar 29005_{\mathchar 29011}$ , and $\mathchar 29005_{\mathchar 28996}$	9.48%
Relative $\mathchar 29005_{\mathchar 28996}$ heave increment	Mean absolute ratio of the $\mathchar 29005_{\mathchar 28996}$ heave increment to final plug heave, all profiles	0.956%
Relative $\mathchar 29005_{\mathchar 28996}$ heave increment	Mean absolute ratio of the $\mathchar 29005_{\mathchar 28996}$ heave increment to final plug heave, final state prediction	0.097%
Relative $\mathchar 29005_{\mathchar 28996}$ heave increment	Mean absolute ratio of the $\mathchar 29005_{\mathchar 28996}$ heave increment to final plug heave, continuous process prediction	2.503%

Table 13

Audit task	Materials shown to the audit role	Information not shown	Question evaluated
Blinded audit replay (A1–A9)	The relevant physical relation, calculation rule, required input and output of the code module, uncorrected code or output, and available check records	The issue label, answer key, prior discussion, correction history, and other project files	Can the audit role find and locate a known issue without being given its answer?
Derivation-to-code probes (P1–P5)	The supplied equations and state-update rules, generated code, and results of the fixed implementation checks	The target problem and answer	Can the audit role identify omissions or inconsistencies not covered by the predeclared checks?

Table 14

Scored item	What was checked	Criterion for a pass
Finding and location	Whether the report found the target problem and named the affected equation, calculation step, or code location	The target problem and its location are both identified correctly
Consequence and correction	Physical or numerical consequence, proposed correction, and follow-up check	The correction addresses the problem and the follow-up check can verify it
False positives and human assistance	Any unrelated finding and any help given during replay	The target problem is found without an incorrect additional finding or an extra hint
Independent probe finding	The equation or rule that the code failed to follow, its code location, and the supporting check result	The report identifies a specific omission or disagreement from the supplied equations, code, and check results

Table 15

Case	Target problem	What the audit role had to check
A1	Correction for near-wall soil dilation absent from the solver call chain	Whether the solver calls the $\mathchar 29005_{\mathchar 28996}$ correction
A2	Degree-valued $\mathchar 28960_{\mathchar 29028}$ passed directly to a trigonometric function expecting radians	Whether the angle unit matches the function input
A3	Committed local increment not limited by the remaining dilation capacity	Whether the applied increment remains within the available capacity
A4	The reported mean effective stress and deviatoric stress retained their trial values and were not recalculated from the final stresses	Whether the reported mean effective stress and deviatoric stress are recalculated from the final stresses
A5	Trial iterations overwrite committed history before convergence is accepted	Whether history is saved only after convergence
A6	Reference confinement uses vertical trial stress instead of lateral trial stress	Which stress is used as the reference confinement

Table 16

Probe	Calculation checked	Fixed checks passed	What the independent audit found
P1	Recalculation of final outputs in converged, edge, and nonconverged branches	10/10	One branch omitted the final-output recalculation; an additional check was added
P2	Active nodes and the piecewise $\mathchar 28961$ calculation in $\mathchar 29005_{\mathchar 28996}$	11/11	One code branch could fail, and the supplied instructions did not state clearly whether inactive grid nodes remained in the input
P3	Saving $\mathchar 29005_{\mathchar 29011}$ history only after convergence is accepted	5/5	The code stopped checking trials as soon as it found one that met the convergence criterion. However, the recorded count included all supplied trials, including those that had not been checked.
P4	Applying $\mathchar 29005_{\mathchar 29011}$ rebound before calculating the remaining capacity	5/5	The code and equation treated $\mathchar 28673\mathchar 29029_{\mathrm{\mathchar 29042\mathchar 29029\mathchar 29026}}$ differently in the numerator
P5	Recalculating hydraulic and stress outputs from the final geometry	5/5	All final-refresh checks were repeated and no additional problem was found

Table 17

Test group	Number of tests	Main operations checked	Acceptance criterion and observed result	Outcome
Hydraulic closure and moving-boundary stress	10	Pressure-difference recovery, tip gradient, free-surface stress, suction-controlled boundaries, and pump-flow separation	Integral and boundary residuals remain within the adopted tolerance; pump flow does not update plug heave or void ratio	Passed
Bolton relationship and $\mathchar 29005_{\mathchar 28996}$ closure	9	Node activation, reference confinement, dilatancy, mobilized displacement, capacity truncation, and final clipping	The correction is non-negative, capacity-limited, and confined to active nodes	Passed
$\mathchar 29005_{\mathchar 29011}$ path update and projection constraints	6	Mean effective stress, lateral projection, critical-state variables, rebound, state-gap release, and clipping	Updated variables match analytical or independently evaluated values	Passed
State commitment and final output refresh	8	Separation of trial and accepted states, saving after convergence, final $\mathchar 29005_{\mathchar 28996}$ update, output recalculation, and saved iteration and check results	Histories are saved only from accepted states, and final outputs are recalculated from the final geometry and stresses	Passed

Table 18

ID	Check	Quantity or comparison	Criterion and observed result	Outcome
R-01	Pressure-difference closure	$\mathchar 28941_{\mathchar 29047}\mathchar 4946\nolimits_{\mathchar 28720}^{\mathchar 29000}\mathchar 29033\delimiter 67273472\mathchar 29048\mathchar 24891\mathchar 29050\delimiter 84054785\,\mathchar 29028\mathchar 29048\mathchar 8704\mathchar 28673\mathchar 29045\delimiter 67273472\mathchar 29050\delimiter 84054785$	Absolute residual remains within the adopted tolerance	Passed
R-02	Moving free-surface stress	$\mathchar 28955^{\mathchar 560}_{\delimiter 69640972}\delimiter 67273472\mathchar 28720\mathchar 24891\mathchar 29050\delimiter 84054785$	Zero within numerical tolerance	Passed
R-03	Deep-limit approximation	Finite-domain seepage kernel and its exponential deep-limit form	Relative difference remains within tolerance	Passed
R-04	Critical-state consistency	$\mathchar 29029_{\mathchar 29027\mathchar 29043}$ and $\mathchar 28681\mathchar 12349\mathchar 29029\mathchar 8704\mathchar 29029_{\mathchar 29027\mathchar 29043}$	Matches independent scalar evaluation	Passed
R-05	$\mathchar 29005_{\mathchar 28996}$ capacity truncation	$\mathchar 28673\bar{\mathchar 28962}_{\delimiter 69640972}^{\mathrm{\mathchar 28996}}\mathchar 12820\mathchar 28673\bar{\mathchar 28962}_{\delimiter 69640972\mathchar 24891\mathrm{\mathchar 29042\mathchar 29029\mathchar 29037}}^{\mathrm{\mathchar 28996}}$	No node exceeds the remaining capacity	Passed
B-01	Input-sequence consistency	Penetration depths, suction-pressure history, geometry, and soil inputs	Input arrays and parameter assignments remain unchanged	Passed

Table 19

Case group	External-boundary basis	Adopted rule	Source class
Kim and Kim (2020) [ 16 ]	900-mm-diameter centrifuge container at 25g and 50g	$\mathchar 29010_{\mathchar 29026}\mathchar 12349\mathchar 28720\mathchar 314\mathchar 28724\mathchar 28725\mathchar 29006_{\mathchar 29031}$ m	derived
Tran and Randolph (2008) [ 14 ]	390-mm by 650-mm strongbox at 100g	$\mathchar 29010_{\mathchar 29026}\mathchar 12349\mathchar 28720\mathchar 314\mathchar 28721\mathchar 28729\mathchar 28725\mathchar 8706\mathchar 28721\mathchar 28720\mathchar 28720\mathchar 12349\mathchar 28721\mathchar 28729\mathchar 314\mathchar 28725$ m	derived
Tjelta (1995) [ 41 ]	Full-scale field bucket without a test-box boundary	$\mathchar 29010_{\mathchar 29029}^{\mathchar 8707}\mathchar 12349\mathchar 29010_{\mathchar 29029\mathchar 24891\mathchar 561}$ ; no finite $\mathchar 29010_{\mathchar 29026}$	direct
Wang et al. (2019) [ 44 ]	1.50-m by 0.96-m by 0.90-m model box	$\mathchar 29010_{\mathchar 29026}\mathchar 12349\mathchar 28720\mathchar 314\mathchar 28724\mathchar 28728$ m	derived
Zhao et al. (2025) [ 43 ]	1.0-m by 0.5-m by 0.6-m model tank	$\mathchar 29010_{\mathchar 29026}\mathchar 12349\mathchar 28720\mathchar 314\mathchar 28722\mathchar 28725$ m	derived

Table 20

Setting	Adopted value or rule	Source basis	Use
$\mathchar 29029_{\mathchar 28672}\delimiter 68408078\mathchar 29029_{\max}$	1.0 central; 0.9–1.1 band	Declared sensitivity rule	Prediction sensitivity band
$\mathchar 28949_{\mathchar 29027}$	0.019	Same-form silica-sand CSL prior	State update
$\mathchar 28948_{\mathchar 29027}$	0.7	Same-form silica-sand CSL prior	State update
$\mathchar 29040_{\mathchar 29025}$	100 kPa	Reference pressure paired with the CSL prior	State update
$\mathchar 28948_{\mathchar 29043}$	0.006	Fixed global coefficient	$\mathchar 29005_{\mathchar 29011}$ update
$\mathchar 28951$	0.30	Granular-soil literature range	Lateral-stress update

Table 21

Study	Benchmark comparison	Data obtained from the study	Use in the comparison
Koteras and Ibsen (2019) [ 42 ]	Volume closure, six tests	Geometry, penetration, final heave, and before/after void ratio are direct; area ratio is derived.	Closure uses the reported void-ratio change.
Ragni et al. (2020) [ 15 ]	Formula reduction, one test	Prototype geometry, penetration and heave ratios, geometric decomposition, and PIV strain are direct.	The retained test uses the reported self-weight boundary condition.
Kim and Kim (2020) [ 16 ]	Final state prediction, four profiles	Geometry, final heave, suction–depth curves, and most soil properties are direct; missing state inputs follow cited analog or common rules.	Digitized normalized curves are converted at prototype scale; $\mathchar 29035_{\mathchar 29042}\delimiter 68408078\mathchar 29035_{\delimiter 69640972}\mathchar 12349\mathchar 28721\mathchar 314\mathchar 28723$ .
Tran and Randolph (2008) [ 14 ]	Final state prediction, five profiles	Geometry, suction curves, and most soil properties are direct; initial state is derived where required.	Final heave is calculated as $\mathchar 29032_{\mathrm{\mathchar 29032\mathchar 29029\mathchar 29025\delimiter 69640972\mathchar 29029}}\mathchar 12349\mathchar 29004\mathchar 8704\mathchar 29050_{\mathrm{\mathchar 29029\mathchar 29038\mathchar 29028}}$ , where $\mathchar 29004$ is the caisson length and $\mathchar 29050_{\mathrm{\mathchar 29029\mathchar 29038\mathchar 29028}}$ is the final penetration depth digitized from the published curve.
Tjelta (1995) [ 41 ]	Full-scale process prediction, A3	Field suction and heave are direct; soil-state inputs combine site/design information and cited regional analogs.	The solver uses $\mathchar 28958_{\mathrm{\mathchar 29037\mathchar 29039\mathchar 29028\mathchar 29029\mathchar 29036}}\mathchar 12349\mathchar 28723\mathchar 28728\mathchar 314\mathchar 28725^{\mathchar 8718}$ ; no finite outer boundary is imposed.
Wang et al. (2019) [ 44 ]	Laboratory process prediction	Geometry, process curves, and the principal soil properties are direct or derived from reported values.	Case inputs use the common hydraulic-ratio and prediction sensitivity band rules in S4.2.

Table 22

Case	Test	Measured heave (mm)	Predicted heave (mm)	Error (%)	Source basis
Koteras and Ibsen (2019)	T1	45.00	49.53	10.07	Reported void-ratio closure.
Koteras and Ibsen (2019)	T2	47.00	48.57	3.33	Reported void-ratio closure.
Koteras and Ibsen (2019)	T3	44.00	47.60	8.19	Reported void-ratio closure.
Koteras and Ibsen (2019)	T4	42.00	54.40	29.54	Reported void-ratio closure.
Koteras and Ibsen (2019)	T5	49.00	55.40	13.06	Reported void-ratio closure.
Koteras and Ibsen (2019)	T9	57.00	58.45	2.55	Reported void-ratio closure.

Table 23

Case	Profile	Measured final heave (cm)	Central final heave (cm)	Band min (cm)	Band max (cm)	Central error (%)
Kim and Kim (2020)	T1-1 S 25g	66.00	48.26	37.00	58.31	-26.87
Kim and Kim (2020)	T1-2 S 50g	53.00	52.30	39.47	64.53	-1.32
Kim and Kim (2020)	T3-1 S 50g	58.00	50.74	38.28	63.13	-12.52
Kim and Kim (2020)	T4-3 S 25g	70.00	59.47	46.04	73.80	-15.05
Tran and Randolph (2008)	TRAN-T1	50.85	60.53	48.07	70.46	19.04
Tran and Randolph (2008)	TRAN-T2	45.04	41.63	34.20	47.79	-7.56

Table 24

Case	Profile	Measured final point (cm)	Central final point (cm)	Band min (cm)	Band max (cm)	Final-point error (%)
Tjelta (1995)	TJELTA-A3	15.43	16.70	14.45	18.98	8.23
Wang et al. (2019)	full process	4.79	4.96	4.51	5.41	3.45
Zhao et al. (2025)	ZHAO-T1	3.68	3.73	3.40	4.06	1.29
Zhao et al. (2025)	ZHAO-T2	3.48	3.67	3.37	3.98	5.49
Zhao et al. (2025)	ZHAO-T3	3.28	3.37	3.10	3.63	2.74

Table 25

Case	Profile	Curve MAPE (%)	Curve RMSE (cm)	Curve NRMSE (%)
Tjelta (1995)	TJELTA-A3	95.49	3.15	20.42
Wang et al. (2019)	full process	19.24	0.52	10.80
Zhao et al. (2025)	ZHAO-T1	36.73	0.16	4.22
Zhao et al. (2025)	ZHAO-T2	119.25	0.14	3.92
Zhao et al. (2025)	ZHAO-T3	13.88	0.14	4.25

Table 26

AI Agents for Scientific Discovery

AI agents for scientific discovery use language models, tools, and experiments to generate, test, and refine scientific knowledge.

See all 293 papers

Physics Is All You Need? A Case Study in Physicist-Supervised AI Development of Scientific Software
May 28, 2026Nhat-Minh NguyenAI for Science AI Coding Agents
Kavli IPMU (WPI), UTIAS, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8583, Japan · Center for Data-Driven Discovery, Kavli IPMU (WPI), UTIAS, The University of Tokyo, Kashiwa, Chiba 277-8583, Japan · Institute For Interdisciplinary Research in Science and Education, ICISE, Quy Nhon, 55121, Vietnam
Physics-Audited Agentic Discovery in Scientific Machine Learning
Jul 8, 2026Diab W. Abueidda, Bilal Ahmed, Panos Pantidis +1Neural Surrogate Modeling PDE Surrogate Modeling
Civil and Urban Engineering Department, New York University Abu Dhabi, United Arab Emirates · National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, United States of America
From Perception to Autonomous Computational Modeling: A Multi-Agent Approach
Apr 8, 2026Daniel N. WilkeLLM Agents Multi-Agent LLM Systems
School of Mechanical, Industrial and Aeronautical Engineering, University of the Witwatersrand, Johannesburg, South Africa

May 28, 2026J/K move · Enter open · S save

Physics Is All You Need? A Case Study in Physicist-Supervised AI Development of Scientific Software

Nhat-Minh Nguyen

Kavli IPMU (WPI), UTIAS, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8583, Japan · Center for Data-Driven Discovery, Kavli IPMU (WPI), UTIAS, The University of Tokyo, Kashiwa, Chiba 277-8583, Japan · Institute For Interdisciplinary Research in Science and Education, ICISE, Quy Nhon, 55121, Vietnam

AI for Science AI Coding Agents Human Oversight of AI LLM Agent Reliability

Open paper AskPDF arXiv

Are AI agents tools, co-authors, or researchers? We present a quantified case study (

N=1

): a physicist supervising an AI coding agent (Claude Code, Sonnet and Opus models) over 12 work days and 57 sessions to build CLAX-PT, a differentiable one-loop perturbation theory module in JAX. We documented and classified 15 supervision events by intervention level. The agent resolved ten autonomously by iterating against oracle tests. Two more by the physicist's domain knowledge. The three it could not -- all evaded oracle detection -- share a common property: the agent treated symptom reduction as root-cause resolution. It spent 33 of the 57 sessions adjusting coefficients within a code architecture that could not represent the target physics, and could not re-evaluate its CLASS-PT branch choice even when prompted to reconsider; only an injected physics concept (anisotropic BAO damping) triggered the redesign. Separately, the agent committed a calibrated correction that passed all oracle tests but corresponded to no quantity in the theory, predicting wrong values at any other cosmology. The fudge factor was caught and replaced within the same session. Three supervision practices proved critical for catching what oracle tests missed: testing at diverse parameter points beyond the fiducial calibration; shared changelogs that surfaced stalled exploration across sessions; and an explicit rule against unphysical numerical patches. In this case, supervision design, not model capability, determined whether the agent's output was trustworthy. Closing the gap would require agents that propose architectural alternatives rather than optimize within a given structure, and distinguish predictive adequacy from explanatory correctness -- capabilities not exhibited here, not obviously addressed by scaling alone. [Abridged.]

Abstract

AI Agents for Scientific Discovery

Explore similar work

Physics Is All You Need? A Case Study in Physicist-Supervised AI Development of Scientific Software

Physics-Audited Agentic Discovery in Scientific Machine Learning

From Perception to Autonomous Computational Modeling: A Multi-Agent Approach