cs.IRMar 23, 2026
SaveGraphRAG for Engineering Diagrams: ChatP&ID Enables LLM Interaction with P&IDs
Organizations: Process Intelligence Research Group, Department of Chemical Engineering, Delft University of Technology
Abstract
Large Language Models (LLMs) combined with Retrieval-Augmented Generation (RAG) and knowledge graphs offer new opportunities for interacting with engineering diagrams such as Piping and Instrumentation Diagrams (P&IDs). However, directly processing raw images or smart P&ID files with LLMs is often costly, inefficient, and prone to hallucinations. This work introduces ChatP&ID, an agentic framework that enables grounded and cost-effective natural-language interaction with P&IDs using Graph Retrieval-Augmented Generation (GraphRAG), a paradigm we refer to as GraphRAG for engineering diagrams. Smart P&IDs encoded in the DEXPI standard are transformed into structured knowledge graphs, which serve as the basis for graph-based retrieval and reasoning by LLM agents. This approach enables reliable querying of engineering diagrams while significantly reducing computational cost. Benchmarking across commercial LLM APIs (OpenAI, Anthropic) demonstrates that graph-based representations improve accuracy by 18% over raw image inputs and reduce token costs by 85% compared to directly ingesting smart P&ID files. While small open-source models still struggle to interpret knowledge graph formats and structured engineering data, integrating them with VectorRAG and PathRAG improves response accuracy by up to 40%. Notably, GPT-5-mini combined with ContextRAG achieves 91% accuracy at a cost of only $0.004 per task. The resulting ChatP&ID interface enables intuitive natural-language interaction with complex engineering diagrams and lays the groundwork for AI-assisted process engineering tasks such as Hazard and Operability Studies (HAZOP) and multi-agent analysis.
Figures & tables
Figure 1: Agentic workflow illustrating how LLMs interact with external tools. The input prompt (2) provides intent, context, and structured information to the model (1). When needed, the model invokes tools (2a) to obtain additional information or perform tasks, and the tool’s output serves as context that improves the final response.
Figure 2: ChatP&ID Workflow. The ChatP&ID takes a flowsheet knowledge graph as its input. The knowledge graph can be generated by digitizing a PDF or image ( Theisen et al., 2023 ) and exporting it to a DEXPI-conformant smart P&ID. The DEXPI P&ID then transformed into a knowledge graph using pyDEXPI ( Goldstein et al., 2025 ) . Using the pyDEXPI object, each flowsheet element is mapped one-to-one to a node in the knowledge graph. From this knowledge graph, multiple abstraction layers are created, from a complete graph to process-level and conceptual-level. GraphRAG methods: ContextRAG, VectorRAG, PathRAG, or CypherRAG, are implemented as tools to retrieve relevant information from a knowledge graph. The LLMs are integrated into the workflow as an agent that determines the appropriate retrieval method to invoke. Finally, a context-aware response is generated to answer user queries.
Figure 3: ChatP&ID agentic workflow integrating graph database and graph-based retrieval method.
Figure 4: Illustration of the path exploration algorithm. To answer the query “how to control process stream temperature,” VectorRAG is first applied to the global embedding space to identify a suitable starting node. The Heat Exchanger (HE) is selected as the starting point, as it is the semantically closest node to the query. From this node, two candidate paths exist: (i) a direct connection to the Globe Valve, or (ii) a path through the Temperature Indicating Controller (TIC). VectorRAG is then applied to the local embedding of neighboring nodes to resolve this choice, selecting the TIC. The exploration proceeds through the Actuator and Globe Valve, and terminates at the Off-Page Connector (OPC) for Cooling Water Return (CWR). The resulting path is returned to the LLM to support the generation of the final answer.
Figure 5: Sample of DEXPI P&ID used in this case study ( Theißen and Wiedau, 2021 ) .
| Criterion | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| Relatedness | Not related at all | Mostly not related | Partly related | Mostly related | Completely related |
| Completeness | Incomplete, key details missing | Incomplete, insufficient details | Complete, insufficient details | Complete, with most details | Complete and detailed answer |
| Correctness | Completely incorrect | Mostly incorrect | Partly correct | Mostly correct | Completely correct |
| Coherence | Unable to draw conclusions from relevant knowledge | Barely able to draw conclusions from established knowledge | Can draw conclusions with some difficulty | Can independently draw conclusions based on established knowledge | Can independently draw correct conclusions based on state-of-the-art knowledge |
Table 1: Rubric criteria for scientific answer evaluation for LLM (adapted from Schulze Balhorn et al., 2024 ).
Figure 6: Evaluation of RAG strategies across task types: (left) Heatmap of response accuracy scores, (right) Relationship between average LLM-as-judge scores and semantic similarity. Small random noise (±0.05) is added to LLM scores for visibility.
Figure 7: Comparison of semantic similarity and LLM-as-judge evaluations for a graph-query task retrieving the set pressure of a safety valve: (i) GPT-5 with ContextRAG, (ii) Claude Opus 4.1 with ContextRAG, and (iii) Claude Haiku 3.5 using read image.
Figure 8: Performance overview of ChatP&ID. The scatter plot groups results by four tasks: graph query, path exploration, knowledge inference, and graph summarization. The y-axis represents the response accuracy, calculated by averaging all LLM-as-judge metrics. Data points are color-coded by tool configuration and shaped by the LLM model host (OpenAI, Anthropic, Ollama). The size of each marker indicates the token cost per QA pair, with larger markers corresponding to higher costs. Horizontal bars with annotations display the mean accuracy score and mean cost for each tool configuration. The average of the accuracy score and cost excludes the Ollama-hosted model, as not all tasks can be solved by the model.
| Representation | Graph Query (single query) | Graph Query (multi-query) | Graph Summarization | Knowledge Inference | Path Exploration | Average |
| GPT-5 | ||||||
| Graph Abstractions | ||||||
| - Conceptual Level | 0.91 / $0.011 | 1.00 / $0.033 | 1.00 / $0.064 | 0.98 / $0.055 | 0.92 / $0.027 | 0.94 / $0.027 |
| - Process Level | 0.93 / $0.035 | 0.96 / $0.060 | 1.00 / $0.096 | 0.98 / $0.076 | 0.87 / $0.063 | 0.93 / $0.055 |
| - Complete Level | 0.95 / $0.086 | 0.75 / $0.126 | 1.00 / $0.168 | 0.98 / $0.126 | 0.82 / $0.132 | 0.90 / $0.113 |
| Flowsheet Image | 0.79 / $0.002 | 0.61 / $0.022 | 0.93 / $0.027 | 0.87 / $0.020 | 0.67 / $0.008 | 0.76 / $0.001 |
Table 2: Response accuracy and token cost per task across different knowledge graph abstractions and flowsheet representations. Tools ContextRAG is used to query from knowledge graphs, image read for flowsheet image, and file read for Proteus File. GPT-5 and GPT-4o-mini results are for comparison. The response accuracy is the average of LLM-as-judge score: completeness, coherence, correctness, and relatedness.
| Model | ContextRAG | VectorRAG | PathRAG | CypherRAG | Multimodal Context | Proteus Context |
| Anthropic (response accuracy / cost per task) | ||||||
| Claude-3-5-Haiku | 0.82 / $0.008 | 0.71 / $0.003 | 0.72 / $0.002 | 0.54 / $0.001 | 0.59 / $0.002 | 0.76 / $0.123 |
| Claude-3-7-Sonnet | 0.88 / $0.032 | 0.71 / $0.015 | 0.72 / $0.009 | 0.61 / $0.005 | 0.62 / $0.007 | 0.81 / $0.464 |
| Claude-Sonnet-4 | 0.87 / $0.033 | 0.71 / $0.014 | 0.72 / $0.009 | 0.69 / $0.005 | 0.68 / $0.009 | 0.81 / $0.465 |
| Claude-Opus-4-1 | 0.85 / $0.162 | 0.69 / $0.068 | 0.72 / $0.045 | 0.68 / $0.024 | 0.66 / $0.038 | 0.78 / $2.077 |
| OpenAI (response accuracy/ cost per task) | ||||||
Table 3: LLMs performance across GraphRAG tools, multimodal context, and Proteus context. Various models from Anthropic, OpenAI, and Ollama are tested. Anthropic and OpenAI are compared by the response accuracy and token cost per task. While the Ollama model is measured by response accuracy and the time needed to complete the task. The Ollama was run in a Mac mini M4 with 24GB RAM. The flowsheet is represented as a conceptual graph for GraphRAG, a flowsheet image for Multimodal Context, and a Proteus XML file for Proteus Context. The response accuracy is the average of LLM-as-judge score: completeness, coherence, correctness, and relatedness.
| Tool | Graph Query (single) | Graph Query (multi) | Graph Summarization | Knowledge Inference | Path Exploration | Average |
| response accuracy | ||||||
| ContextRAG | 0.89 | 0.95 | 1.00 | 0.98 | 0.88 | 0.91 |
| VectorRAG | 0.90 | 0.79 | 0.78 | 0.87 | 0.68 | 0.82 |
| PathRAG | 0.87 | 0.65 | 0.88 | 0.88 | 0.79 | 0.83 |
| CypherRAG | 0.88 | 0.81 | 0.93 | 0.86 | 0.84 | 0.86 |
| Multimodal Context | 0.79 | 0.85 | 0.95 | 0.91 | 0.80 | 0.83 |
Table 4: Tool performance across tasks: response accuracy, cost per task ($), and execution time (s). The LLM examined is GPT-5-mini, the graph abstraction at the conceptual level is used to test GraphRAG, the image read tool is used to read Multimodal Context, and file read for Proteus Context. The response accuracy is the average of LLM-as-judge score: completeness, coherence, correctness, and relatedness.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Task Type | Question | Reference Answer |
|---|---|---|
| Graph Query (Single) | What is the cylinder length of tank T4750? | 4.0 m |
| Graph Query (Single) | What is the design heat transfer area of heat exchanger H1007? | 46.8 m 2 |
| Graph Query (Single) | What is the design pressure of pump P4711? | 10.0 m (pressure head) |
| Graph Query (Single) | What is the design shaft power of pump P4711? | 60.0 kW |
| Graph Query (Single) | What is the design volume flow rate of pump P4712? | 420.0 m 3 /h |
| Graph Query (Single) | What is the nominal diameter of valve 66KL21? | DN 80 |
Table 1: Dataset of Questions and Reference Answers for Graph Tasks
| Representation | Graph Query (single) | Graph Query (multi) | Graph Summarization | Knowledge Inference | Path Exploration | Average |
|---|---|---|---|---|---|---|
| OpenAI: GPT-5 | ||||||
| Graph Abstractions | ||||||
| - Conceptual Level | 0.91 / $0.011 | 1.00 / $0.033 | 1.00 / $0.064 | 0.98 / $0.055 | 0.92 / $0.027 | 0.94 / $0.027 |
| - Process Level | 0.93 / $0.035 | 0.96 / $0.060 | 1.00 / $0.096 | 0.98 / $0.076 | 0.87 / $0.063 | 0.93 / $0.055 |
| - Complete Level | 0.95 / $0.086 | 0.75 / $0.126 | 1.00 / $0.168 | 0.98 / $0.126 | 0.82 / $0.132 | 0.90 / $0.113 |
| Flowsheet Image | 0.79 / $0.002 | 0.61 / $0.022 | 0.93 / $0.027 | 0.87 / $0.020 | 0.67 / $0.008 | 0.76 / $0.001 |
Table 2: Response accuracy and token cost per task across different knowledge graph abstractions and flowsheet representations. Tools: ContextRAG is used to query knowledge graphs; image read for flowsheet images; and file read for Proteus File. Comprehensive results for Anthropic and OpenAI models. The response accuracy is the average of LLM-as-judge score: completeness, coherence, correctness, and relatedness.
| Tool | Graph Query (single) | Graph Query (multi) | Graph Sum. | Know. Inf. | Path Expl. | Average |
|---|---|---|---|---|---|---|
| Anthropic: Claude 3.5 Haiku | ||||||
| ContextRAG | 0.93 / $0.007 / 6.1s | 0.81 / $0.010 / 14.3s | 0.65 / $0.010 / 19.7s | 0.73 / $0.009 / 16.4s | 0.72 / $0.008 / 8.1s | 0.82 / $0.008 / 9.8s |
| VectorRAG | 0.85 / $0.002 / 5.7s | 0.66 / $0.003 / 10.8s | 0.53 / $0.004 / 16.2s | 0.64 / $0.004 / 16.9s | 0.58 / $0.003 / 10.7s | 0.71 / $0.003 / 9.9s |
| PathRAG | 0.84 / $0.002 / 55.6s | 0.64 / $0.002 / 139.1s | 0.58 / $0.004 / 103.3s | 0.67 / $0.004 / 110.1s | 0.62 / $0.002 / 108.1s | 0.72 / $0.002 / 89.3s |
| CypherRAG | 0.51 / $0.000 / 2.0s | 0.63 / $0.001 / 4.3s | 0.43 / $0.001 / 3.5s | 0.54 / $0.002 / 8.0s | 0.57 / $0.001 / 3.0s | 0.54 / $0.001 / 3.5s |
| Multimodal Context | 0.58 / $0.001 / 7.1s | 0.49 / $0.002 / 16.9s | 0.58 / $0.003 / 23.2s | 0.68 / $0.003 / 21.0s | 0.60 / $0.002 / 12.2s | 0.59 / $0.002 / 12.5s |
Table 3: Tool performance across tasks: response accuracy, cost per task ($), and execution time (s). Comprehensive results for Anthropic, OpenAI, and Ollama models. The response accuracy is the average of LLM-as-judge score: completeness, coherence, correctness, and relatedness.
More in this topic
Graph Retrieval-Augmented Generation
Graph Retrieval-Augmented Generation connects language models to knowledge graphs to retrieve relational evidence for grounded, multi-hop answers.
See all 232 papers