Sketch2Simulation: Automating Flowsheet Generation via Multi Agent Large Language Models
Organizations: Sargent Centre for Process Systems Engineering, Imperial College London, London, SW7 2AZ, United Kingdom · Department of Chemical Engineering, Brigham Young University, Provo, Utah 84602, United States
Abstract
Converting process sketches into executable simulation models remains a major bottleneck in process systems engineering, requiring substantial manual effort and simulator-specific expertise. Recent advances in generative AI have improved both engineering-diagram interpretation and LLM-assisted flowsheet generation, but these remain largely disconnected: diagram-understanding methods often stop at extracted graphs, while text-to-simulation workflows assume structured inputs rather than raw visual artifacts. To bridge this gap, we present an end-to-end multi-agent large language model system that converts process diagrams directly into executable Aspen HYSYS flowsheets. The framework decomposes the task into three coordinated layers: diagram parsing and interpretation, simulation model synthesis, and multi-level validation. Specialized agents handle visual interpretation, graph-based intermediate representation construction, code generation for the HYSYS COM interface, execution, and structural verification. We evaluate the framework on four chemical engineering case studies of increasing complexity, from a simple desalting process to an industrial aromatic production flowsheet with multiple recycle loops. The system produces executable HYSYS models in all cases, achieving complete structural fidelity on the two simpler cases and strong performance on the more complex ones, with connection consistency above 0.93 and stream consistency above 0.96. These results demonstrate a viable end-to-end sketch-to-simulation workflow while highlighting remaining challenges in dense recycle structures, implicit diagram semantics, and simulator-interface constraints.
Figures & tables
| Mechanism | Purpose | Stage |
|---|---|---|
| Description validation (A1.1) | Assess alignment between the visual input and the generated description | Post-descriptor (A1) |
| Schema and prompt safeguards | Enforce structured outputs, object constraints, and internal consistency | Interpretation and synthesis agents |
| Execution validation and fixing | Validate executability and apply targeted runtime corrections | Execution (B4) |
| Agent | Model / logic core | Agent type | Environment |
|---|---|---|---|
| Descriptor (A1) | Gemini 3 Flash | Multimodal LLM | Cloud |
| Validation (A1.1) | Gemini 3 Flash | Multimodal LLM | Cloud |
| Extractor (A2) | Gemini 3 Flash | Multimodal LLM | Cloud |
| Normalization (A3) | Rule-based logic | Rule-based agent | Local |
| Basis (B1) | Qwen2.5-Coder-7B | Code-oriented LLM | Local |
| Instantiation (B2) | Qwen2.5-Coder-7B | Code-oriented LLM | Local |
| Category | Component | Specification |
|---|---|---|
| Hardware environment | CPU | Intel Xeon Gold 6442Y (2.60 GHz) |
| Memory | 256 GB RAM | |
| GPU | NVIDIA RTX A4000 (16 GB VRAM) | |
| Operating system | Windows 11 (64-bit) | |
| Compute platform | CUDA 12.8 | |
| Software stack | Programming language | Python 3.11.13 |
| Case | Process | Unit operations | Stream density | Recycle loops | Diagram characteristics |
|---|---|---|---|---|---|
| 1 | Desalting | Low | Sparse | None | Missing labels, implicit mixing |
| 2 | Merox Sweetening | Moderate | Moderate | One | Compact layout, ambiguous connectivity |
| 3 | Atmospheric Distillation | Moderate | Moderate | One | Non-standard symbols, partially labeled units |
| 4 | Aromatic Production | High | Dense | Multiple | Industrial-scale flowsheet with complex interconnections |
| Metric | Evaluated element | Purpose |
|---|---|---|
| Unit Consistency (UC) | Unit operations | Evaluates the correctness of extracted unit operations |
| Stream Consistency (SC) | Material streams | Evaluates the correctness of extracted material flows |
| Connection Consistency (CC) | Directed unit-to-unit connectivity | Evaluates the correctness of process topology |
| Material Consistency (MC) | Feed and process material components | Evaluates the correctness of extracted material and component identities |
| Configuration | Target | Purpose |
|---|---|---|
| C0 – Full Workflow | All | Baseline configuration |
| C1 – Remove Descriptor | A1 | Evaluate the contribution of visual-text grounding |
| C2 – Remove Normalization | A3 | Evaluate the contribution of structural refinement |
| C3 – Merge Coding Agents | B1–B3 | Evaluate the contribution of modular code decomposition |
| C4 – Disable RAG | B1 | Evaluate the contribution of retrieval-based material mapping |
| Parameter | Value | Purpose |
|---|---|---|
| Temperature | 0.0 | Removes probabilistic sampling |
| Top-k | 1 | Selects only the highest-probability token |
| Top-p | 1.0 | Disables variability from nucleus sampling |
| Seed | 42 | Ensures consistent behavior across executions |
Program Synthesis
Program synthesis automatically constructs executable programs from examples, specifications, demonstrations, or natural-language descriptions.
See all 179 papers