DUCTILE: Agentic LLM Orchestration of Engineering Analysis in Product Development Practice

Alejandro Pradas-Gomez; Arindam Brahma; Ola Isaksson

cs.SEMar 10, 2026

DUCTILE: Agentic LLM Orchestration of Engineering Analysis in Product Development Practice

Authors: Alejandro Pradas-Gomez, Arindam Brahma, Ola Isaksson

Organizations: Product Development Division, Department of Mechanical Engineering, Chalmers University of Technology, Gothenburg, Sweden · GKN Aerospace Engine Systems, Trollhättan, Sweden

Abstract

Engineering analysis automation in product development relies on rigid interfaces between tools, data formats and documented processes. When these interfaces change, as they routinely do as the product evolves in the engineering ecosystem, the automation support breaks. This paper presents a DUCTILE (Delegated, User-supervised Coordination of Tool- and document-Integrated LLM-Enabled) agentic orchestration, an approach for developing, executing and evaluating LLM-based agentic automation support of engineering analysis tasks. The approach separates adaptive orchestration, performed by the LLM agent, from deterministic execution, performed by verified engineering tools. The agent interprets documented design practices, inspects input data and adapts the processing path, while the engineer supervises and exercises final judgment. DUCTILE is demonstrated on an industrial structural analysis task at an aerospace manufacturer, where the agent handled input deviations in format, units, naming conventions and methodology that would break traditional scripted pipelines. Evaluation against expert-defined acceptance criteria and deployment with practicing engineers confirm that the approach produces correct, methodologically compliant results across 10 repeated independent runs. The paper discusses the paradigm shift and the practical consequences of adopting agentic automation, including unintended effects on the nature of engineering work when removing mundane tasks and creating an exhausting supervisory role.

Figures & tables

Req.	Requirement name	Description	Supporting literature
R1	Inspectability	Agentic application (prompts, plans, tool calls, intermediate artifacts) must be directly inspectable by engineers/auditors; no hidden state.	[ 17 , 18 ]
R2	Reproducibility	Model, prompt, tool versions and inputs must be pinned and logged; runs are reproducible or cause - attributable despite stochastic LLM steps.	[ 18 , 17 ]
R3	Deterministic execution boundary	Safety/engineering computations run in verified, deterministic tools; the LLM performs orchestration and code generation only.	[ 18 , 51 ]
R4	Traceability & auditability	Every artifact is trace - linked to the generating step (arguments, stdout/stderr, checksums, timestamps) to support do/check/approve reviews.	[ 17 , 18 ]
R5	Data governance	Deployment mode (on - premises/API) and data flows comply with confidentiality/export/privacy rules and are justified.	[ 17 , 18 ]
R6	Human oversight & accountability	Engineers review plans and sign off on outputs; autonomy is limited to orchestration, not engineering judgment.	[ 17 ]

Table 1: Requirements for DUCTILE and supporting literature

Figure 1: Schematic of the LLM inference loop. The model takes the full context window as input, produces a probability distribution over the next token, samples one token according to a sampling strategy, appends it to the context window, and repeats until a stop token (<EOS>) is generated. The resulting sequence of tokens forms the response.

Panel of Figure 2 — Figure 2: Comparison of LLM responses with and without thinking mode enabled. The thinking scratchpad (dashed box) allows the model to self-critique and reconsider the autoregressive initial response before committing to a final answer.

Panel of Figure 3 — Figure 3: Comparison of LLM responses with and without tool calling. Without tools, the model relies on parametric knowledge from training weights, returning a room-temperature value. With tool calling, the model queries an external materials database for temperature-dependent data, grounding its response on verified external information.

Figure 4: Architecture of an agentic application. The inference engine processes user messages through the LLM within a context window, generating either final answers or tool calls. Tool execution is handled internally or through external interfaces that connect to external resources including file systems, knowledge bases, tool repositories and system interfaces.

DUCTILE recommended values
	S-Basis	B-Basis	A-Basis
Pass probability $p$	$\geq 0.50$	$\geq 0.90$	$\geq 0.99$
Required $k$	5	29	299
Development	$k=3$
Deployment	$k\geq 10$

Table 2: Minimum consecutive passes

k

for the pass^k metric at 95% confidence, derived from the Clopper–Pearson bound

p_{\mathrm{lower}}=\alpha^{1/k}

with

\alpha=0.05

Figure 5: Top: Views of the component in the physical domain. Bottom: Activity View of the TRS Strength evaluation. Only a subset of the inputs, documents and tools are shown for visual clarity. Highlighted in orange is the load processing step where the use case focuses.

Deviation	What changed	Impact on automation
File format	YAML instead of JSON	Requires new parser
Unit system	Imperial instead of SI	Silent error
Node naming	right/left instead of port/starboard	FEM run fails downstream
Method change	1.04 factor on Fx forces	Silent error

Table 3: Deviations in the new analysis scenario. The first three are input format changes; the fourth is an OEM-communicated correction specified in the task description.

Panel of Figure 6 — Figure 6: Engineer-interpreted activity process flowchart of the natural language and unstructured description contained in the loads design practice document. NOTE: The agent must infer a similar workflow by reading the document; this flowchart is not available in the agent’s context.

Panel of Figure 7 — Figure 7: Summary of Engineer 1’s interaction. Green bubbles: engineer messages. Orange bubbles: agent responses. Dashed boxes: agent reasoning. Yellow boxes: tool calls. Dotted boxes: engineer actions outside the conversation. An expanded transcript is available in the supplemental material.

LLM Agent Orchestration

Large language model agent orchestration coordinates models, tools, and workflows to complete complex tasks reliably and efficiently.

See all 458 papers

Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows
Oct 2, 2026Merve Astekin, Yan Naing Tun, Arda Goknil +5Software Engineering Agents LLM Agent Evaluation
SINTEF, Oslo, Norway · Singapore Management University, Singapore, Singapore · Ozyegin University, Istanbul, Turkiye
From Perception to Autonomous Computational Modeling: A Multi-Agent Approach
Apr 8, 2026Daniel N. WilkeLLM Agents Multi-Agent LLM Systems
School of Mechanical, Industrial and Aeronautical Engineering, University of the Witwatersrand, Johannesburg, South Africa
EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design
May 19, 2026Gioele Molinari, Florian Felten, Soheyl Massoudi +1Multi-Agent LLM Systems LLM Agent Evaluation
IDEAL Chair of Artificial Intelligence in Engineering Design ETH Zurich Zurich, Switzerland · IDEAL, ETH Zurich, Zurich, Switzerland · Autom8.build, Brussels, Belgium

Oct 2, 2026J/K move · Enter open · S save

Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows

Merve Astekin, Yan Naing Tun, Arda Goknil, Erik Johannes Husom+4

SINTEF, Oslo, Norway · Singapore Management University, Singapore, Singapore · Ozyegin University, Istanbul, Turkiye

Software Engineering Agents LLM Agent Evaluation Multi-Agent LLM Systems LLM Inference Efficiency Energy-Efficient ML

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used in software engineering, including agentic systems that coordinate multiple agents, but impose higher computational and environmental costs. In this paper, we present a comprehensive empirical study of agentic LLM systems across five software engineering tasks: code generation, technical debt identification, code vulnerability detection, log parsing, and log analysis. For each task, we compare LLM configurations that range from a non-agentic single-query baseline to multi-agent workflows, using six open-weight LLMs, two prompt strategies, and three hardware platforms. We assess each configuration in terms of accuracy, inference latency, and energy consumption. Our results reveal substantial trade-offs between agentic complexity and energy efficiency: multi-agent designs consume on average 6.36

\times

as much energy and run 6.07

\times

as long as the non-agentic baseline, with worst-case slowdowns of up to 160

\times

for individual task--hardware pairs. Accuracy gains from additional agents are limited and task-specific: multi-agent improves average vulnerability-detection accuracy, but lightweight non-agentic and single-agent configurations still dominate the Pareto front, accounting for 59 of 66 Pareto-optimal configurations. Model and prompt choice act as task-specific levers whose effective direction varies between tasks rather than as global defaults. We translate these findings into design guidelines for sustainable, task-aware LLM-based development tools.

Abstract

LLM Agent Orchestration

Explore similar work

Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows

From Perception to Autonomous Computational Modeling: A Multi-Agent Approach

EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design