Large manufacturing companies face challenges in information retrieval due to data silos maintained by different departments, leading to inconsistencies and misalignment across databases. This paper presents an experience in integrating and retrieving qualification data for electronic components used in satellite board design. Due to data silos, designers cannot immediately determine the qualification status of individual components. However, this process is critical during the planning phase, when assembly drawings are issued before production, to optimize new qualifications and avoid redundant efforts. To address this, we propose a pipeline that uses Virtual Knowledge Graphs for a unified view over heterogeneous data sources and LLMs to enhance retrieval and reduce manual effort in data cleansing. The retrieval of qualifications is then performed through an Ontology-based Data Access approach for structured queries and a vector search mechanism for retrieving qualifications based on similar textual properties. We perform a comparative cost-benefit analysis, demonstrating that the proposed pipeline also outperforms approaches relying solely on LLMs, such as Retrieval-Augmented Generation (RAG), in terms of long-term efficiency.
Figures & tables
Part Number (PN)
Package Code
Subpackage Code
Manufacturer
Family
Pitch
P1111111
FP1
a1
ABC
Hybrid
1.27
P2222222
C2
x2
XYZ
Capacitor
2.2
P3333333
R1
a3
ABC
Resistor
1.92
Table 1: Examples of records from the PLM-DB and QC.
Figure 1: Overview of the proposed LLM-enhanced semantic integration pipeline. The approach integrates two heterogeneous data silos (PLM-DB and QC) by combining LLMs and VKGs. LLMs assist in semi-automated data cleaning, including normalization and extraction of key fields, while knowledge engineering is performed once to create a semantic layer over the data. Direct and by-similarity qualifications are retrieved via exact symbolic queries over the VKG, whereas alternative qualifications are retrieved through vector search and refined using rules based on the component type and finally validated by a domain expert.
Figure 2: Overview of the pipeline for semi-automated extraction of Part Numbers (PN) from textual descriptions using LLMs. Textual entries, potentially containing Part Numbers, are first processed by an LLM that automatically extracts candidate PNs. These candidates are subsequently cross-checked against the PLM-DB to validate correctness. Entries that cannot be verified through this step are flagged and sent to human domain experts for manual review, including those for which a candidate PN could not be extracted (indicated as NA). Successfully validated PNs are then added to QC.
Figure 3: High-level overview of the VKG semantic model for qualification retrieval. It shows the two main entities ( PLMDB_COMPONENT and QUALIFICATION_CARD ) and the core join attributes (PN, manufacturer, package and subpackage) used for direct and by-similarity matching. Additional domain-specific properties are omitted for readability.
Model
Qual. Type
Precision
Recall
F1-score
IoU
GPT-OSS-120B
Direct
0.947
0.896
0.921
0.959
Similarity
0.969
0.907
0.937
0.907
Alternative
0.516
0.773
0.619
0.596
Overall
0.866
0.886
0.876
0.846
Mistral 3 24B
Direct
0.055
0.654
0.101
0.287
Similarity
0.152
0.306
0.203
0.143
Table 2: Performance of the RAG-based approach with different LLMs for direct, by-similarity, and alternative qualifications.
Figure 4: Cumulative effort in person-days required for each approach as a function of the number of electronic components, considering system setup time and human-in-the-loop validation. The top figure shows the absolute effort of the three pipelines, while the bottom figure shows the relative effort compared to the manual (AS-IS) baseline. The VKG+LLM pipeline requires higher setup time but becomes significantly more efficient at scale, reducing effort by over 70% beyond 5000 components.
Hardware datasheets contain engineering specifications essential for component selection, system integration, and hardware compatibility verification. However, manual inspection is tedious and error-prone because datasheets are lengthy and heterogeneous, with dense information fragmented across tables, figures, and text. Although large language models (LLMs) can automate this process, naïve one-shot processing omits relevant specifications even when the document fits within the model's context window. We present an automated LLM-driven pipeline that transforms hardware datasheets into general-purpose hierarchical hardware knowledge graphs through chunk-based property extraction, hierarchical routing, and iterative refinement. The framework uses LLMs for information extraction and semantic reasoning while preserving source provenance and applying graph updates through validated instructions and deterministic local execution. This staged design reduces per-query context and maintains traceable intermediate representations. The resulting knowledge graphs provide semantically organized, source-grounded hardware properties for downstream retrieval and reasoning. We evaluate the framework on 20 hardware datasheets spanning diverse component types and document lengths. We assess property faithfulness and introduce inter-graph completeness to measure relative semantic coverage without exhaustive ground truth. Our framework achieves 98.8% property faithfulness compared with 76.1% for the one-shot baseline, while fully covering 83.6% of faithful baseline properties; in the reverse direction, the baseline fully covers only 10.5% of ours. These improvements provide more reliable and comprehensive engineering information for downstream hardware design and verification, reducing the risk of overlooked constraints and incorrect decisions.
Haotian Qiao, Robert P. Dick
Department of Electrical Engineering and Computer Science University of Michigan Ann Arbor, MI, USA
Retrieval-Augmented Generation (RAG) fails systematically on queries requiring structural reasoning over interconnected entities. We compare eight retrieval architectures for aerospace supply chain intelligence, progressing from text retrieval through graph traversal to graph computation. Using a 46-node knowledge graph with 64 typed edges, we evaluate 23 queries across 10 intent categories and demonstrate that five query classes are structurally unreachable for vector retrieval. Our central finding is the operator vocabulary thesis: the barrier to LLM-based graph reasoning is not model intelligence but the computational operators available as tools. An LLM Query Planner with 9 typed traversal primitives outperforms bespoke handlers (F1 = 0.632 vs. 0.472) while generalizing to unseen queries. Adding 6 graph computation tools, the LLM selectively adopts them for exactly the query categories where traversal fails. We also identify a measurement gap: entity-level F1 systematically underscores structural queries where comprehensive answers are correct.
Modern LLMs and AI agents increasingly support data engineering workflows that integrate evidence from unstructured sources. Such pipelines typically do data retrieval, integration, and ranking before proceeding to more complex agentic reasoning or actions, e.g., for scientific discovery. The core retrieval problem in these workflows jointly executes multi-attribute filtering, multi-vector search, exact relational joins, and thresholded embedding-similarity joins. Given a planned query and monotone scoring function, our DASE query engine constructs and ranks candidate evidence tuples. It comprises (i) a multi-step reasoning query model over structured predicates, multiple vectors, and relational links; (ii) SemJI, a sparse materialized embedding-similarity join index for rare near-neighbor pairs; and (iii) a co-designed execution layer that combines predicate-aware ANN traversal, batched access, and threshold-based score aggregation. On scientific-discovery workloads, DASE retrieves candidate evidence for multi-step reasoning queries 6x to 46x faster than strong RDBMS, rerank, and vector-database baselines at comparable recall; and for tasks that require semantic-operator post-processing, DASE acts as a high-recall prefilter that makes downstream LLM evaluation both cheaper and more accurate -- e.g., on SemBench E-Commerce it improves BigQuery quality from 0.67 to 0.80 while cutting cost from 2.42to0.54.