Image-Based Structural Analysis Using Computer Vision and LLMs: PhotoBeamSolver

Bridge structures are regularly inspected for structural damage such as cracks and corrosion in order to ensure public safety and reduce maintenance costs. Much research has been done on automating this process using computer vision methods, which are often evaluated and compared using metrics such as intersection over union, mean average precision, etc. However, predicting the actual effectiveness of an inspection method within the field of structural engineering from these metrics remains challenging. To enable the systematic use of these increasingly popular methods in engineering practice, evaluating the performance of these methods in a way that is compatible with standard engineering approaches is therefore an urgent necessity. We present a new statistical evaluation framework to allow the comparison of computer vision methods with conventional visual inspection for crack detection in steel bridges. The framework is based on probability of detection curves and can account for the influence of image resolution. We apply this evaluation method to the real-world ``Cracks in Steel Bridges'' dataset, which contains annotated images of cracks in bridge structures. The quantification of the probability of detection and its uncertainty enables a practical assessment of the effect of automated methods for damage detection in structural reliability analyses. In turn, this enables the wide-spread use of automated (AI-based) damage detection in safety critical applications. This evaluation method provides evidence that the proposed computer vision approach approach is robust for the crack detection task and can have a high added value as an addition to conventional visual inspection methods.

Converting structural framing plans into editable finite-element model drafts is labor-intensive and susceptible to transcription errors. Existing building-component recognition systems generally depend on task-specific neural detectors, whereas language-model agents in structural engineering typically operate on text or model data rather than on drawings. To the authors' knowledge, this work is the first to apply an agentic vision-language layer to structural-component detection and model drafting from framing-plan PDFs without task-specific detector training or fine-tuning. A deterministic stage extracts geometric primitives, estimates scale by dimension-ratio consensus, recognizes five entity classes using an explicit drafting grammar, and assembles an editable layout. The agentic stage constrains typed corrections through deterministic candidates, operation-specific admission tests, change-level review, and fail-closed transactions. Evaluation used an author-generated benchmark of 100 plans, divided equally between a development half used for all rule revisions and a seed-disjoint held-out half generated after the rules were frozen and evaluated once. All scores are end-to-end results for the complete framework on the held-out half. Scale estimates were within 0.1% of the generator reference for every drawing. Recall and precision were 0.922/0.997 for columns, 0.886/0.990 for beams, 1.000/1.000 for walls, 1.000/1.000 for braces, and 1.000/0.964 for openings. A controlled study repeated two corruptions three times on three development drawings. Calibration passed all nine trials, whereas member repair satisfied every strict end-state criterion in five of nine trials. Because both benchmark halves share a generator, the evaluation does not address independently drafted plans, raster input, analytical connectivity, or solver validation.

Recent advances in large language models (LLMs) have shown the promise to significantly accelerate the workflow by automating structural modeling and analysis. However, existing studies primarily focus on enabling LLMs to operate a single structural analysis software platform. In practice, structural engineers often rely on multiple finite element analysis (FEA) tools, such as ETABS, SAP2000, and OpenSees, depending on project needs, user preferences, and company constraints. This limitation restricts the practical deployment of LLM-assisted engineering workflows. To address this gap, this study develops LLMs capable of automating frame structural analysis across multiple software platforms. The LLMs adopt a two-stage multi-agent architecture. In Stage 1, a cohort of agents collaboratively interpret user input and perform structured reasoning to infer geometric, material, boundary, and load information required for finite element modeling. The outputs of these agents are compiled into a unified JSON representation. In Stage 2, code translation agents operate in parallel to convert the JSON file into executable scripts across multiple structural analysis platforms. Each agent is prompted with the syntax rules and modeling workflows of its target software. The LLMs are evaluated using 20 representative frame problems across three widely used platforms: ETABS, SAP2000, and OpenSees. Results from ten repeated trials demonstrate consistently reliable performance, achieving accuracy exceeding 90% across all cases.