Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models

Marco Luca Sbodio; Marcos Martínez Galindo; Vanessa Lopez; Blanca Biel; Pablo Canca; Pedro Delgado; Jesús I. Mendieta-Moreno; Raphael Tack

cs.AISep 15, 2026

Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models

Authors: Marco Luca Sbodio, Marcos Martínez Galindo, Vanessa Lopez, Blanca Biel, Pablo Canca, Pedro Delgado, Jesús I. Mendieta-Moreno, Raphael Tack,

Organizations: IBM Research · Dpto. Física Atómica, Molecular y Nuclear Instituto Carlos I de Física Teórica y Computacional Univ. of Granada, Spain · IFMIF DONES, Spain · Instituto de Ciencia de Materiales de Madrid (ICMM) CSIC, Spain ·

Abstract

The quest for new materials increasingly relies on predictive models and comprehensive simulations that span scales from atomic to macroscopic levels. However, essential data necessary for these models and simulations are often embedded in scientific literature as unstructured text, limiting reusability and posing challenges for researchers seeking to leverage existing knowledge effectively. While extracting structured data from unstructured text using large language models is gaining popularity, traditional methods typically generate key-value pairs data with straightforward schemas. In contrast, we introduce eolas, a modular pipeline that uses large language models to automatically transform scientific documents into knowledge graphs aligned with a specified ontology. We demonstrate eolas effectiveness in extracting useful information for scientists studying materials designed to endure the extreme temperatures and radiation levels found in fusion reactors. While a human expert might spend between thirty to ninety minutes extracting relevant data from an article, eolas can generate high-quality knowledge graphs in just a few minutes. These are presented in a tabular format with faceted navigation for easy human validation. Additionally, we introduce the first benchmark dataset designed to assess large language models capabilities in constructing knowledge graphs within the domain of irradiated materials. The analysis of 168 experiments using our dataset, various large language models and prompting techniques provides key insights that we summarize into practical guidelines for effectively extracting knowledge graphs aligned with an input ontology.

Figures & tables

Figure 1: Illustration of some of the most representative defects observed in materials under irradiation conditions: vacancies, where an atom is missing from a lattice site; self-interstitial (SIA) or foreign interstitials, where an atom occupies a site between the perfect crystal lattice sites; Frenkel pair, a vacancy and its corresponding interstitial atom; dumbbells, where two atoms share the same lattice site; or bigger defect structures, such as clusters or grain boundaries.

Figure 2: Screenshots of the eolas User Interface. (a) The interface displays the knowledge graph extracted from a document [ 22 ] (left) as tabular data (right). This table can be queried and filtered according to user needs. (b) Users can explore extraction results at a finer level by viewing tabular data corresponding to specific text fragments, with an optional visualization of the underlying knowledge graph (c). This semantically-rich knowledge graph captures measurements from multiple defect types, sizes and geometries for a material of interest, extracted from a text passage containing numerous interrelated entities and relations. The visualization shows relations between intermediate entities (grey nodes), the ontology classes they instantiate (pink nodes), and their associated values or leaf entities (blue nodes). These relationships describe: (1) materials and their crystal structure (eg., bcc Iron) (2) measurements (e.g., relaxation volume) for the different defect types and geometries for a given material; and (3) the methodology and parameters used to compute these values (e.g, Molecular Dynamics). The extracted knowledge for that passage is presented into a table ((b) top right), where leaf properties appear as columns and the connected intermediate nodes are represented within the same row. This detailed view allows for more precise analysis and understanding of individual components within the extracted data.

instructions, serialization	use instances	use examples	use consolidation	LLMs	number of configurations
S, BAML	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	48
S, TTL	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	48
D, TTL	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	48
S, VERB	$\left\{F\right\}$	$\left\{T\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	12
D, VERB	$\left\{F\right\}$	$\left\{T\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	12

Table 1: Summary of the 168 experiments using 6 different LLMs (

\mathcal{L}=\left\{\text{GPT\_120, GPT\_20, GRANITE, LLAMA\_3, LLAMA\_4, MISTRAL}\right\}

), and different prompt structures. The prompt may include simple (S) or detailed (D) instructions, and a serialization of the ontology schema in either BAML, TTL or a verbalized (VERB) textual format. The prompt may optionally (True, False) include ontology instances, and examples (few-shot vs zero-shot prompts). Finally, we may (True, False) use some heuristics to consolidate LLM results.

Figure 3: To mitigate the computational cost of the graph edit distance, we transform KGs from RDF format (a, b) into property graphs (c, d), which have fewer nodes/edges. The example shows the expected graph (a, c) corresponding to the following fragment of text "The corresponding binding energies are 1.43, 0.73, and 0.41 eV for the case of O, N, and C, respectively, in good agreement with previous DFT values [6,17,18,22]" (from [ 25 ] ). The predicted graph (b, d) is generated using LLAMA_3 with a few-shot prompt, detailed instructions, ontology schema serialized in turtle format (TTL), with no instances, and using consolidation. We highlight in green in (a, c) the node and edge (property and value) that are missing in the predicted graph, resulting in a graph edit distance of 2.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	37.8	38.7	30.6	32.4
	GPT_20	12.6	12.6	12.6	12.6	9.0	9.0	7.2	8.1

Table 2: Percentage of success at normalized graph edit distance

NGED^{*}=0.1

for all experimental configurations. Each cell reports the score

P_{j}(0.1)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	62.2	63.1	57.7	59.5
	GPT_20	12.6	12.6	12.6	12.6	27.9	28.8	27.9	28.8

Table 3: Percentage of success at normalized graph edit distance

NGED^{*}=0.2

for all experimental configurations. Each cell reports the score

P_{j}(0.2)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

Figure 4: Percentages of success

P_{\mathcal{F}}

at various thresholds of normalized graph edit distance (

NGED^{*}

) across different configuration families

\mathcal{F}

(

n=\ldots

is the number of configuration within the family). When significant results are found, we report the outcomes of statistical tests beneath each box plot. For comparisons between two families, we use the Mann-Whitney U test, otherwise we use the Kruskal-Wallis test followed by pairwise post hoc analysis using Dunn’s test with Holm’s correction. We conduct all possible post hoc pairwise comparisons, but we report only those that present statistically significant differences:

P_{\mathcal{F}_{i}}<P_{\mathcal{F}_{j}}

indicates that the distribution of percentages of success for

\mathcal{F}_{i}

is statistically significantly lower than that for

\mathcal{F}_{j}

Figure 5: Structure of the proposed ontology to model the domain of irradiated materials

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	27.9	28.8	19.8	19.8
	GPT_20	12.6	12.6	12.6	12.6	5.4	5.4	0.9	1.8

Appendix

Table 4: Percentage of success at normalized graph edit distance

NGED^{*}=0.05

for all experimental configurations. Each cell reports the score

P_{j}(0.05)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	53.2	52.3	48.6	50.5
	GPT_20	12.6	12.6	12.6	12.6	19.8	21.6	16.2	18.9

Appendix

Table 5: Percentage of success at normalized graph edit distance

NGED^{*}=0.15

for all experimental configurations. Each cell reports the score

P_{j}(0.15)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	67.6	69.4	65.8	67.6
	GPT_20	12.6	12.6	12.6	12.6	36.9	36.9	37.8	38.7

Appendix

Table 6: Percentage of success at normalized graph edit distance

NGED^{*}=0.25

for all experimental configurations. Each cell reports the score

P_{j}(0.25)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

Figure 7: (A) Ground truth and predicted graphs generated by LLaMA-3 in few-shot (using schema-only prompts with a GED = 2.0 in between both graphs). Multiple formation energy values are associated with distinct methods and reference sources, preserving fidelity to the original text structure. A standard RAG approached (B) failed to answer the question "What is the formation energy for dumbbell SIAs as calculated with DFT, MD or EAM?", despite retrieving the correct document and passage. A naive Graph-RAG approach (C), which prompts an LLM to retrieve an answer, using as input either the full ground truth KG or the document-level predicted one containing that passage, retrieves the correct answer.

Question	Answer (Graph-RAG)
What is the relaxation volume of C15 clusters of size 12 and 48?	The relaxation volume of C15 clusters of size 12 and 48 calculated with DFT is 0.91 $\Omega$ and 0.98 $\Omega$ , respectively.
What is the relaxation volume for all defect types?	The relaxation volume for all defect types is as follows: – Dumbbell SIA: 1.5 $\Omega$ – C15 clusters (size 12): 0.91 $\Omega$ – C15 clusters (size 48): 0.98 $\Omega$ – $\frac{1}{2}\langle 111\rangle$ dislocation loops: 0.905 $\Omega$ – $\langle 100\rangle$ dislocation loops: 0.873 $\Omega$ – Single vacancies: $-0.11$ $\Omega$ – Spherical cavities (radius 1 nm): 0.46 eV – L100 and L111 defects: 0.95 $\Omega$ – Loop 100 (size 6): 0.86 $\Omega$ – Loop 111 (size 6): 0.89 $\Omega$ … etc.
What is the relaxation volume for all defect types as calculated with DFT?	The relaxation volume for all defect types as calculated with DFT is not explicitly provided in the knowledge graph. However, the relaxation volume for specific defect types such as dumbbell SIAs and C15 clusters is mentioned. For dumbbell SIAs, the relaxation volume is 1.62 $\Omega_{0}$ according to DFT simulations by Ma et al. For C15 clusters, the relaxation volume is 0.91 $\Omega_{0}$ for C15 12 and 0.98 $\Omega_{0}$ for C15 48 .
What is the migration energy of a vacancy in iron according to DFT calculations?	The migration energy of a vacancy in iron according to DFT calculations is 0.65 eV.
What is the migration energy of a self-interstitial in iron according to DFT calculations?	The migration energy of a self-interstitial in iron according to DFT calculations is 0.3 eV.
What is the migration energy of a vacancy in iron according to molecular dynamics using empirical potentials?	NA

Appendix

Table 7: Some example competency questions and their answers using Graph-RAG. NA is retrieved correctly when the answer is not in the input KG.

Figure 8: Example of an automatically generated prompt with a detailed instruction, including an ontology-verbalization listing intermediate classes with their attribute and object relations, and a one-shot KG example, serialised in turtle, corresponding to the given passage (the Context) and compliant to the ontology. The corresponding KG for that passage is also visualized: intermediate nodes are shown in pink, leaf entities in blue and datatype literals in green. The sub-graph can also be presented as a table.

Figure 9: Predicted graphs generated by Mistral Large compared to ground truth examples. (A) Mistral-large zero-shot prediction using the ontology with instances require 4 edits (GED=4). Deviations include: using has_defect_size instead of has_number_vacancies , discrepancies in value formatting (e.g.: " $>4$ " vs. "4–", " $>1$ eV" vs "1- eV"), and a missing has_energetics relation in the MethodologyEnergetics node. These errors arise from the model not having seen examples illustrating conventions for representing ranges or for distinguishing when to use similar properties. (B) Mistral Large in few-shot mode, even with access to the full ontology schema, fails to separate Interstitial_Cluster and Vacancy_Cluster into distinct DefectsEnergetics nodes (GED = 10.0), violating the functional property constraint of has_defect_type , which permits only one value per node. It also extracts extra nodes for large clusterswith a valid Extrapolation method, which is correctly mentioned in the text but absent from the ontology and ground truth.

KG Construction

Knowledge graph construction turns text, tables, images, and databases into structured entities and relationships that machines can query and reason over.

See all 197 papers

Oct 21, 2026

Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models

Over the past decade, substantial efforts have been devoted to developing machine learning (ML) methods for extracting materials data and, where possible, materials knowledge directly from the scientific literature. This review traces the evolution of these methods from heuristic, rule-based pipelines through natural language processing (NLP) models and transformer-based encoders to, lastly, large language models (LLMs) and agentic systems capable of extracting materials information across multiple modalities, including text, tables, and figures. Beside construction of large, queryable materials databases and knowledge graphs that support downstream materials discovery, we discuss how these advances have enabled new capabilities, such as, generic question answering across materials science, AI co-scientists and chatbots for research assistance, and closed-loop optimization and discovery when integrated with autonomous laboratories. We critically assess the datasets generated using these approaches, their extraction accuracy and computational costs, and the respective strengths and limitations of different methodologies, and even provide practical guidelines for developing more reliable and scalable materials data extraction pipelines. Finally, we identify key open challenges, including the extraction of full materials structure-property-processing-performance tetrahedron at the sample level, the scarcity of reliable benchmark datasets for systematic evaluation, and the resolution of non-standard materials terminology across different document modalities.

Aravindan Kamatchi Sundaram, B. Pabitramohan Prusty, Rohit Batra

Department of Metallurgical and Materials Engineering, Indian Institute of Technology Madras, Chennai 600036, India · Center for Atomistic Modelling and Materials Design, IIT Madras, Chennai 600036, India · Wadhwani School of Data Science & Artificial Intelligence, IIT Madras, Chennai 600036, India

Document Information Extraction LLM Information Extraction Materials Science

Ask PDF

Sep 15, 2026

LLM-Guided Ontology-Driven Knowledge Graph Construction from Unstructured Text

Ontology-driven knowledge graph construction from industrial text remains challenging due to the domain specificity of documents, the scarcity of annotated resources, and the complexity of ontology engineering workflows. This paper presents and investigates the applicability of an ontology learning pipeline that combines compact open-source Large Language Models (LLMs), reusable prompting strategies, and open knowledge bases to support the extraction, structuring, enrichment, and evaluation of knowledge from textual corpora. The approach is tested and evaluated on a private French corpus of power-grid incident reports, using locally deployable open-source LLMs ranging from 7B to 32B parameters. Starting from unstructured reports, the approach extracts entities and relations, generates RDF triples, constructs related OWL ontology, enriches it using external knowledge sources, assesses the quality of the ontology, and subsequently constructs a populated knowledge graph grounded in the resulting ontology schema. Experiments on 80 manually annotated private reports show that schema-guided prompting significantly improves extraction quality, while quantized models provide an effective trade-off between performance and computational cost. These results demonstrate the feasibility of transforming domain-specific industrial text into ontology-based knowledge graphs using locally deployed open-source LLMs, while supporting the generalization of the extraction process through reusable prompting strategies.

Abdelhadi Belfadel, Maxence Gagnant, Joseph Kattan +1

Institute for Technological Research - IRT SystemX 2 Bd Thomas Gobert, 91120 Palaiseau, France.

KG Construction Ontology Learning LLM Information Extraction

Ask PDF

Apr 8, 2026

From Papers to Property Tables: A Priority-Based LLM Workflow for Materials Data Extraction

Scientific data are widely dispersed across research articles and are often reported inconsistently across text, tables, and figures, making manual data extraction and aggregation slow and error-prone. We present a prompt-driven, hierarchical workflow that uses a large language model (LLM) to automatically extract and reconstruct structured, shot-level shock-physics experimental records by integrating information distributed across text, tables, figures, and physics-based derivations from full-text published research articles, using alloy spall strength as a representative case study. The pipeline targeted 37 experimentally relevant fields per shot and applied a three-level priority strategy: (T1) direct extraction from text/tables, (T2) physics-based derivation using verified governing relations, and (T3) digitization from figures when necessary. Extracted values were normalized to canonical units, tagged by priority for traceability, and validated with physics-based consistency and plausibility checks. Evaluated on a benchmark of 30 published research articles comprising 11,967 evaluated data points, the workflow achieved high overall accuracy, with priority-wise accuracies of 94.93% (T1), 92.04% (T2), and 83.49% (T3), and an overall weighted accuracy of 94.69%. Cross-model testing further indicated strong agreement for text/table and equation-derived fields, with lower agreement for figure-based extraction. Implementation through an API interface demonstrated the scalability of the approach, achieving consistent extraction performance and, in a subset of test cases, matching or exceeding chat-based accuracy. This workflow demonstrates a practical approach for converting unstructured technical literature into traceable, analysis-ready datasets without task-specific fine-tuning, enabling scalable database construction in materials science.

Koushik Rameshbabu, Jing Luo, Ali Shargh +2

Department of Applied Mathematics and Statistics, Johns Hopkins University · Department of Mechanical Engineering, Johns Hopkins University

Document Information Extraction LLM Information Extraction Materials Science

Ask PDF

Abstract

KG Construction

Explore similar work

Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models

LLM-Guided Ontology-Driven Knowledge Graph Construction from Unstructured Text

From Papers to Property Tables: A Priority-Based LLM Workflow for Materials Data Extraction