From Perception to Autonomous Computational Modeling: A Multi-Agent Approach

Daniel N. Wilke

cs.CEApr 8, 2026

From Perception to Autonomous Computational Modeling: A Multi-Agent Approach

Authors: Daniel N. Wilke

Organizations: School of Mechanical, Industrial and Aeronautical Engineering, University of the Witwatersrand, Johannesburg, South Africa

Abstract

We present a solver-agnostic framework in which coordinated large language model (LLM) agents autonomously execute the complete computational mechanics workflow, from perceptual data of an engineering component through geometry extraction, material inference, discretisation, solver execution, uncertainty quantification, and code-compliant assessment, to an engineering report with actionable recommendations. Agents are formalised as conditioned operators on a shared context space with quality gates that introduce conditional iteration between pipeline layers. We introduce a mathematical framework for extracting engineering information from perceptual data under uncertainty using interval bounds, probability densities, and fuzzy membership functions, and introduce task-dependent conservatism to resolve the ambiguity of what `conservative' means when different limit states are governed by opposing parameter trends. The framework is demonstrated through a finite element analysis pipeline applied to a photograph of a steel L-bracket, producing a 171,504-node tetrahedral mesh, seven analyses across three boundary condition hypotheses, and a code-compliant assessment revealing structural failure with a quantified redesign. All results are presented as generated in the first autonomous iteration without manual correction, reinforcing that a professional engineer must review and sign off on any such analysis.

Figures & tables

Figure 1: Three-layer architecture with orchestrator and iterative feedback. The input (perceptual data: image, video, audio, or text, plus user metadata) feeds into three processing layers: the perception layer extracts structured data from the perceptual input; the analysis layer handles discretisation, solving, and verification; the assessment layer interprets results against design codes and emits actionable recommendations and proposed design modifications as outputs. An overarching orchestrator (right) coordinates the entire process, maintaining a dynamic checklist of tasks, dispatching agents, monitoring quality gate outcomes, and routing feedback between layers. The agent evolver (top right) processes engineer corrections after each completed analysis, updating agent definitions and shared memory for future runs. Dashed arrows indicate iterative feedback: assessment findings can trigger re-analysis with modified assumptions, and analysis-layer quality gates can request updated perception-layer outputs. The orchestrator decides which agent to invoke next based on the current state of the shared context, the pending checklist items, and any gate failures. The architecture is solver-agnostic (FEA, computational fluid dynamics (CFD), DEM, SPH, MPM, lattice Boltzmann method (LBM)).

Physics	Fields	Scalar eqns	PDE type	ICs?	State?	Constitutive + additional inputs
Solid mechanics
Static elastic	$\mathbf{u}$	$d$	Elliptic	No	No	$E,\nu$ (or $\mathbb{C}$ ); BCs ( $\bar{\mathbf{u}},\bar{\mathbf{t}}$ ); $\mathbf{b}$
Dynamic elastic	$\mathbf{u}$ , $\dot{\mathbf{u}}$	$d$	Hyperbolic	Yes	No †	$E,\nu,\rho$ ; BCs; $\mathbf{u}_{0},\dot{\mathbf{u}}_{0}$
Elastoplastic	$\mathbf{u}$ , $\boldsymbol{\alpha}$	$d{+}n_{\alpha}$	Hyp. + evol.	Yes	Yes	$E,\nu$ , yield $f$ , hardening $H$ ; $\boldsymbol{\alpha}_{0}$
Hyperelastic (large def.)	$\mathbf{u}$ , $\mathbf{F}$	$d$	Elliptic (NL)	No	No	$W(\mathbf{F})$ strain energy; BCs; $\mathbf{b}$
Fluid mechanics

Table 1: Continuum physics: fields, equations, and required inputs. This table is not exhaustive; many other physics models (piezoelectric, magnetohydrodynamic, phase-field fracture, etc.) follow the same pattern. For each row, agents must determine all listed quantities before a solver can be invoked.

Parameter	Extracted	Plaus. band	True	Within band?
Arm length (mm)	125	[106.3, 143.8]	150	No ( $-17\%$ )
Width (mm)	25	[21.3, 28.8]	25	Yes
Thickness (mm)	2.5	[2.13, 2.88]	3.0	No ( $-17\%$ )
$R_{\mathrm{inner}}$ (mm)	3.0	[2.55, 3.45]	3.0	Yes
$R_{\mathrm{outer}}$ (mm)	5.5	[4.68, 6.33]	6.0	Yes
Hole diameter (mm)	5.0	[4.25, 5.75]	6.0	No ( $-17\%$ )

Table 2: Comparison of autonomously extracted geometry (from photograph) against ground-truth physical measurements, together with the derived bracket self-weight. The “Plaus. band” column is the first-iteration self-reported plausibility band produced by the geometry agent from image evidence alone; it is an agent-generated estimate of its own confidence, not a calibrated confidence interval, and it is not guaranteed to bracket the true value. Five of the eleven geometric parameters and both derived mass quantities fall outside this self-reported band. Rather than being a failure of the uncertainty-quantification framework, these exceedances are the primary demonstration of why the framework places a professional engineer in the loop at the end of the pipeline ( Section 7 ): ground truth is not recoverable from a single photograph, so every such row flags a parameter for which engineer-supplied measurement is the definitive reference. The residuals reported here become the feedback signal

\delta r

that updates the agent via Equation 7 . The agent correctly identifies the bar width (25 mm) and inner-bend radius (3 mm) but under-estimates arm length by 17% and thickness by 17%. The self-weight underestimate compounds the length and thickness errors but does not materially affect the analysis because the self-weight

G

is negligible compared to the applied design load (

G\approx

1.2\text{,}\mathrm{N}

vs $P=$200\text{\,}\mathrm{N}

Panel of Figure 2 — Figure 2: From photograph to finite element mesh. (a) Input image with visual cues used for geometry extraction and material identification. (b) Resulting tetrahedral mesh with graded refinement.

Figure 3: Autonomous mesh feature audit. The discretisation reviewer agent scans the generated mesh to confirm hole count, identifies each hole with an index, and reports diameters, positions, and countersink presence. The inner-bend region carries the finest element size (0.5 mm), with secondary refinement at hole/countersink features (0.8 mm), selected automatically by the discretisation generator agent.

Run	BC	Load	Variant	$\sigma_{\max}$ (MPa)	$\delta_{\max}$ (mm)	Cons. for
R1	Nominal	LC1 (UDL)	Baseline	1651	10.77	Baseline
R2	Stiff	LC1 (UDL)	Baseline	799	6.59	-
R3	Flexible	LC1 (UDL)	Baseline	2024	11.15	Stiffness
R4	Nominal	LC2 (Tip)	Baseline	4153	33.75	Strength
R5	Nominal	LC1 (UDL)	300 N	2477	16.15	Scaling
R6	Nominal	LC1 (UDL)	$E{=}190$	1651	11.34	$E$ -sens.

Table 3: FEA simulation matrix (

t=2.5

mm baseline with countersinks). Peak VM stress is at the BC singularity; the engineering bending stress at the bend is

480\text{\,}\mathrm{MPa}

for LC1. R7 is the coarse-mesh convergence check.

Figure 4: Von Mises stress across all three BC variants (LC1, 200 N UDL). Left column: full bracket view; right column: zoom into the inner-bend region at the two nearest holes. Nominal (top): peak $\sigma_{\mathrm{VM}}=$1651\text{\,}\mathrm{MPa}$$ at BC singularity. Flexible (middle): peak $\sigma_{\mathrm{VM}}=$2024\text{\,}\mathrm{MPa}$$ , highest stress. Stiff (bottom): peak $\sigma_{\mathrm{VM}}=$799\text{\,}\mathrm{MPa}$$ , lowest stress due to back-face clamping distributing load. The inner-bend is autonomously identified as the critical hot-spot; the engineering bending stress at the bend cross-section ( $480\text{\,}\mathrm{MPa}$ ) is consistent with the hand calculation. — Figure 4: Von Mises stress across all three BC variants (LC1, 200 N UDL). Left column: full bracket view; right column: zoom into the inner-bend region at the two nearest holes. Nominal (top): peak $\sigma_{\mathrm{VM}}=$ 1651\text{,}\mathrm{MPa} $at BC singularity. Flexible (middle): peak $\sigma_{\mathrm{VM}}=$2024\text{\,}\mathrm{MPa}$ , highest stress. Stiff (bottom): peak $\sigma_{\mathrm{VM}}=$ 799\text{,}\mathrm{MPa}$$ , lowest stress due to back-face clamping distributing load. The inner-bend is autonomously identified as the critical hot-spot; the engineering bending stress at the bend cross-section ( $480\text{\,}\mathrm{MPa}$ ) is consistent with the hand calculation.

Figure 5: Deformed shapes (front view, warp $\times 0.5$ , common colour scale) for all three BC variants under LC1 (200 N UDL). Nominal: $\delta_{\max}=$10.77\text{\,}\mathrm{mm}$$ ; flexible: $\delta_{\max}=$11.15\text{\,}\mathrm{mm}$$ (stiffness-conservative); stiff: $\delta_{\max}=$6.59\text{\,}\mathrm{mm}$$ . The flexible variant produces the largest deflection, confirming it is conservative for stiffness under load-controlled conditions ( Section 5 ). — Figure 5: Deformed shapes (front view, warp $\times 0.5$ , common colour scale) for all three BC variants under LC1 (200 N UDL). Nominal: $\delta_{\max}=$ 10.77\text{,}\mathrm{mm} $; flexible: $\delta_{\max}=$11.15\text{\,}\mathrm{mm}$ (stiffness-conservative); stiff: $\delta_{\max}=$ 6.59\text{,}\mathrm{mm}$$ . The flexible variant produces the largest deflection, confirming it is conservative for stiffness under load-controlled conditions ( Section 5 ).

Figure 6: FEA simulation matrix results across all seven runs. Top: peak von Mises stress with yield (

\sigma_{y}=

200\text{,}\mathrm{MPa}

), allowable ( $\sigma_{y}/1.5=$133\text{\,}\mathrm{MPa}

), and analytical bounds [300 to 1000] MPa shown as references. Bottom: maximum displacement with the

L/50=

2.5\text{,}\mathrm{mm}$$ serviceability limit. R4 (concentrated tip load, red) produces the worst case for both stress and deflection; R2 (stiff BC, green) produces the lowest stress. All runs exceed both yield and deflection limits. Colour coding: blue = baseline, green = best case, red = worst case, grey = sensitivity variants.

Baseline ( $t=2.5$ mm), nominal BC, 200 N UDL:
Check	Criterion	Value	Limit	Verdict
Strength (ULS)	$\sigma_{\mathrm{bend}}<\sigma_{y}/1.5$	480 MPa	133 MPa	FAIL (FoS = 0.42)
Stiffness (SLS)	$\delta<L/50$	10.77 mm	2.5 mm	FAIL
Bearing	$\sigma_{b}<2.5f_{u}t/\gamma$	5.4 MPa	1417 MPa	PASS
Redesign ( $t=5.0$ mm), nominal BC, 200 N UDL:
Strength (ULS)	$\sigma_{\mathrm{bend}}<\sigma_{y}/1.5$	156.6 MPa	133 MPa	COND. (FoS y = 1.28)

Table 4: Code-compliance assessment. Baseline (

t=2.5

mm) fails all strength/stiffness checks. Maximum safe load:

46\text{\,}\mathrm{N}

(4.7 kg), governed by stiffness. Notation:

\sigma_{\mathrm{bend}}

= engineering bending stress at the bend,

\sigma_{y}

= yield strength (

200\text{\,}\mathrm{MPa}

\sigma_{b}

= bearing stress,

f_{u}

= ultimate tensile strength (

340\text{\,}\mathrm{MPa}

t

= thickness,

\gamma=1.5

= safety factor,

L

= arm length,

\delta

= tip deflection, FoS = factor of safety, ULS = ultimate limit state, SLS = serviceability limit state.

Figure 7: Autonomous redesign outcome: baseline ( $t=2.5$ mm, top) versus redesign RD2 ( $t=5.0$ mm, bottom) for the nominal BC variant. Left: bracket geometry; centre: inner-bend mesh detail; right: mesh and result metrics including scaling ratios versus beam-theory predictions.

Metric	Value	Metric	Value
Wall-clock time	22 min 5 s	Total tokens	92,245
Agents invoked	9	Tool calls	36
FEA runs (baseline)	7	FEA runs (redesign)	3
Gate iterations (G1 to G4)	1 each	Files generated	$\sim$ 46
Estimated cost (Opus 4.6)	$1.01	Estimated cost (Sonnet 4)	$0.61

Table 5: Pipeline execution statistics. All nine agents complete their checklist items on the first attempt (no gate re-iterations required). The FEA solver agent consumes the most tokens due to the seven-run simulation matrix. Cost estimates use Claude Opus 4.6 API pricing (

5/M input,

25/M output) with the observed 70/30 input/output token split.

Figure 8: Two-tier self-improving feedback loop. Prompt-level refinement (olive dashed): engineer corrections are distilled into abstract patterns updating agent definitions and shared memory, lightweight, interpretable, and reversible. Model-level refinement (red dashed): when the same correction recurs across multiple analyses, the feedback operator triggers supervised retraining (fine-tuning or preference optimisation) of the LLM executing the agents, so that persistent weaknesses are addressed at the model level. Each analysis performed under engineering supervision makes future analyses more reliable.

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 681 papers

Apr 10, 2026

Automating Structural Analysis Across Multiple Software Platforms Using Large Language Models

Recent advances in large language models (LLMs) have shown the promise to significantly accelerate the workflow by automating structural modeling and analysis. However, existing studies primarily focus on enabling LLMs to operate a single structural analysis software platform. In practice, structural engineers often rely on multiple finite element analysis (FEA) tools, such as ETABS, SAP2000, and OpenSees, depending on project needs, user preferences, and company constraints. This limitation restricts the practical deployment of LLM-assisted engineering workflows. To address this gap, this study develops LLMs capable of automating frame structural analysis across multiple software platforms. The LLMs adopt a two-stage multi-agent architecture. In Stage 1, a cohort of agents collaboratively interpret user input and perform structured reasoning to infer geometric, material, boundary, and load information required for finite element modeling. The outputs of these agents are compiled into a unified JSON representation. In Stage 2, code translation agents operate in parallel to convert the JSON file into executable scripts across multiple structural analysis platforms. Each agent is prompted with the syntax rules and modeling workflows of its target software. The LLMs are evaluated using 20 representative frame problems across three widely used platforms: ETABS, SAP2000, and OpenSees. Results from ten repeated trials demonstrate consistently reliable performance, achieving accuracy exceeding 90% across all cases.

Ziheng Geng, Jiachen Liu, Ian Franklin +3

Department of Civil and Architectural Engineering, University of Miami, Coral Gables, FL 33146, USA · HBC Engineering Company, Miami, FL 33178, USA · College of Civil Engineering, Hunan University, Changsha, 410082, China +2

Code Generation Structural Engineering Multi-Agent LLMs

Ask PDF

May 27, 2026

VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis

Finite Element Analysis (FEA) serves as the cornerstone of modern engineering design. However, its workflow is inherently complex and relies heavily on domain expertise. Although recent efforts have integrated Large Language Models (LLMs) into FEA, existing approaches face limitations in handling multimodal inputs and executing complex tasks. To address these limitations, we propose VFEAgent, an end-to-end multi-agent system designed to automate FEA modeling and simulation directly from input images and problem descriptions. Our methodology integrates two core components: (1) a multimodal vision-language multi-agent pipeline that employs ReAct-driven reasoning to extract structured FEA specifications from heterogeneous inputs and (2) a verification-first code synthesis framework, incorporating robust self-debugging and fallback mechanisms to ensure executability and physical validity. We systematically evaluated the system across various engineering mechanics scenarios. The results demonstrate that VFEAgent achieves a high success rate in generating complete and physically valid simulations, outperforming LLM-based baseline methods in reliability and correctness. These findings validate the feasibility of automating the complete FEA workflow, highlighting the framework's potential to liberate engineers from tedious manual analysis.

Jiachen Zhang, Junyi Lao, Chenghao Liu +5

Peking University · China Agricultural University · 1Peking University +1

Program Synthesis Finite Element Method Multimodal Agents Multi-Agent LLMs

Ask PDF

May 28, 2026

A Multi-AI-agent Framework Enabling End-to-end Finite Element Analysis for Solid Mechanics Problems

Finite element analysis (FEA) is the most important numerical approach for solid mechanics. Challenges of FEA include a steep learning curve for entry-level users and potential false simulations due to incorrect definitions of key simulation components, such as boundary conditions, load cases, and solution variables. Years of engineering experience are usually necessary for real-world problem-solving. To address these issues, we present AbaqusAgent, a multi-agent framework grounded in large language models (LLMs) for solid mechanics analyses. AbaqusAgent is developed to facilitate analysis case generation and execution using Abaqus, one of the most widely used FEA packages, by turning users' natural-language instructions into executed FEA analyses and result visualization. AbaqusAgent is composed of six agents, including interpreter, architect, input writer, runner, reviewer, and visualizer agents, encompassing all the essential pre-processing and post-processing steps of standard FEA analyses. A wide variety of 50 solid mechanics problems have been successfully validated, achieving an overall success rate of 86%. Beyond improving the efficiency of FEA for solid mechanics problems and lowering the barrier to computational mechanics education, AbaqusAgent advances the human-simulation interaction paradigm and enables integration with AI-empowered optimization and material characterization workflows. The code is available at https://github.com/LIRAM-LIN/AbaqusAgent

Titu Ranjan Sarker, Muhammed Jawaad Zulqernine, Ling Yue +3

University of Texas at Arlington, Arlington, TX 76010, USA · Rensselaer Polytechnic Institute, Troy, NY 12180, USA · Institute for Predictive Performance Methodologies, Fort Worth, TX 76118, USA

Multi-Agent LLM Systems Structural Mechanics LLM Agent Orchestration Tool-Using Agents Finite Element Method Scientific Workflow Automation

Ask PDF

Abstract

LLM Agents

Explore similar work

Automating Structural Analysis Across Multiple Software Platforms Using Large Language Models

VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis

A Multi-AI-agent Framework Enabling End-to-end Finite Element Analysis for Solid Mechanics Problems