Agentic Risk-Aware Set-Based Engineering Design

Large Language Models (LLMs) can generate Computer-Aided Design (CAD), yet lack physical comprehension required for reliable engineering design. Instead of attempting to implicitly learn physical laws from data, we propose a Hybrid Agentic-Physical Architecture that embeds validated knowledge-based engineering tools directly into the decision making loop of autonomous AI agents. In this framework, engineering design is formulated as a closed-loop, sequential decision making process guided by explicit physical verification. Based on a load case, dedicated agents iteratively plan, generate, evaluate, and revise engineering designs using knowledge-based tools as a feedback signal. We introduce a benchmark dataset and metrics for assessing functional validity in generative CAD. Our system generates more complex and physically verified designs, with a 4.2 increase in structural complexity and improving compile rate by 3.5% compared to similar agentic methods. The codebase, prompts and dataset will be made publicly available to support reproducibility and future research.

Large Language Model (LLM) agents are increasingly applied to engineering design tasks, yet existing evaluation frameworks do not adequately address multi-agent systems that combine simulation, retrieval, and manufacturing preparation. We introduce a benchmark suite with three evaluation dimensions: (1) a workflow benchmark with seven prompt styles targeting distinct cognitive demands-including direct tool use, semantic disambiguation, conditional branching, and working-memory tasks; (2) a Retrieval-Augmented Generation (RAG) benchmark with gated scoring isolating retrieval contributions to parameter selection; and (3) an High Performance Computing (HPC) benchmark evaluating end-to-end ML training orchestration on a SLURM cluster. Alongside the benchmark we present EngiAI, a Multi-Agent System (MAS) reference implementation built on LangGraph that operationalizes the benchmark by coordinating seven specialized agents through a supervisor architecture, unifying topology optimization, document retrieval, HPC job orchestration, and 3D printer control. Across four LLM backends and two EngiBench problems, proprietary models achieve 96-97% average task completion on Beams2D, while open-source 4B-parameter models reach 55-78%, with clear generational improvement. Conditional branching proves most challenging, with task completion dropping to 20-53% for the conditional style on Photonics2D. RAG gating confirms near-perfect retrieval-augmented scores (about 1.0) versus near-zero without retrieval, validating the evaluation design. On HPC orchestration, one model completes all pipeline steps in 100% of runs while another drops to 50%, revealing that multi-step instruction following degrades over long-running workflows.

The design of reinforced concrete (RC) highway barriers is a safety-critical engineering task that requires strict compliance with regulatory provisions such as the AASHTO LRFD Bridge Design Specifications. Current engineering practice relies largely on manual, iterative, and experience-driven procedures to satisfy complex material, geometric, and mechanical constraints. Although standalone large language models (LLMs) show strong capabilities in knowledge representation and text generation, their direct use in structural engineering design is limited by hallucination, numerical reasoning errors, and insufficient integration with physics-based analysis. To address these limitations, this study proposes a "generation-validation-modification" closed-loop framework for automated RC barrier design based on the multi-agent orchestration capability of AutoGen. The framework integrates specialized agents for parameter generation, mechanics-based calculation, target-interval evaluation, deviation diagnosis, and rule-based design modification. The multi-agent framework (MAF) was evaluated on three barrier testing levels (TL-3, TL-4, and TL-5) using sixty RC barrier designs with different geometric configurations. Three DeepSeek models with different parameter scales, DS-8B, DS-32B, and DS-671B, were investigated. All designs were evaluated according to Section 13 of the AASHTO LRFD Bridge Design Specifications, 10th Edition (2024). Results show that MAF-DS-8B achieves a target-interval compliance rate of 98.3%, while the best-performing standalone LLM, DS-32B, achieves 11.7%. These results demonstrate that multi-agent architectures can improve the reliability, interpretability, and accessibility of AI-assisted engineering design for practical applications.