Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models

Marco Luca Sbodio; Marcos Martínez Galindo; Vanessa Lopez; Blanca Biel; Pablo Canca; Pedro Delgado; Jesús I. Mendieta-Moreno; Raphael Tack

cs.AISep 15, 2026

Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models

Authors: Marco Luca Sbodio, Marcos Martínez Galindo, Vanessa Lopez, Blanca Biel, Pablo Canca, Pedro Delgado, Jesús I. Mendieta-Moreno, Raphael Tack,

Organizations: IBM Research · Dpto. Física Atómica, Molecular y Nuclear Instituto Carlos I de Física Teórica y Computacional Univ. of Granada, Spain · IFMIF DONES, Spain · Instituto de Ciencia de Materiales de Madrid (ICMM) CSIC, Spain ·

Abstract

The quest for new materials increasingly relies on predictive models and comprehensive simulations that span scales from atomic to macroscopic levels. However, essential data necessary for these models and simulations are often embedded in scientific literature as unstructured text, limiting reusability and posing challenges for researchers seeking to leverage existing knowledge effectively. While extracting structured data from unstructured text using large language models is gaining popularity, traditional methods typically generate key-value pairs data with straightforward schemas. In contrast, we introduce eolas, a modular pipeline that uses large language models to automatically transform scientific documents into knowledge graphs aligned with a specified ontology. We demonstrate eolas effectiveness in extracting useful information for scientists studying materials designed to endure the extreme temperatures and radiation levels found in fusion reactors. While a human expert might spend between thirty to ninety minutes extracting relevant data from an article, eolas can generate high-quality knowledge graphs in just a few minutes. These are presented in a tabular format with faceted navigation for easy human validation. Additionally, we introduce the first benchmark dataset designed to assess large language models capabilities in constructing knowledge graphs within the domain of irradiated materials. The analysis of 168 experiments using our dataset, various large language models and prompting techniques provides key insights that we summarize into practical guidelines for effectively extracting knowledge graphs aligned with an input ontology.

Figures & tables

Figure 1: Illustration of some of the most representative defects observed in materials under irradiation conditions: vacancies, where an atom is missing from a lattice site; self-interstitial (SIA) or foreign interstitials, where an atom occupies a site between the perfect crystal lattice sites; Frenkel pair, a vacancy and its corresponding interstitial atom; dumbbells, where two atoms share the same lattice site; or bigger defect structures, such as clusters or grain boundaries.

Figure 2: Screenshots of the eolas User Interface. (a) The interface displays the knowledge graph extracted from a document [ 22 ] (left) as tabular data (right). This table can be queried and filtered according to user needs. (b) Users can explore extraction results at a finer level by viewing tabular data corresponding to specific text fragments, with an optional visualization of the underlying knowledge graph (c). This semantically-rich knowledge graph captures measurements from multiple defect types, sizes and geometries for a material of interest, extracted from a text passage containing numerous interrelated entities and relations. The visualization shows relations between intermediate entities (grey nodes), the ontology classes they instantiate (pink nodes), and their associated values or leaf entities (blue nodes). These relationships describe: (1) materials and their crystal structure (eg., bcc Iron) (2) measurements (e.g., relaxation volume) for the different defect types and geometries for a given material; and (3) the methodology and parameters used to compute these values (e.g, Molecular Dynamics). The extracted knowledge for that passage is presented into a table ((b) top right), where leaf properties appear as columns and the connected intermediate nodes are represented within the same row. This detailed view allows for more precise analysis and understanding of individual components within the extracted data.

instructions, serialization	use instances	use examples	use consolidation	LLMs	number of configurations
S, BAML	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	48
S, TTL	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	48
D, TTL	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	48
S, VERB	$\left\{F\right\}$	$\left\{T\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	12
D, VERB	$\left\{F\right\}$	$\left\{T\right\}$	$\left\{T,F\right\}$	$\mathcal{L}$	12

Table 1: Summary of the 168 experiments using 6 different LLMs (

\mathcal{L}=\left\{\text{GPT\_120, GPT\_20, GRANITE, LLAMA\_3, LLAMA\_4, MISTRAL}\right\}

), and different prompt structures. The prompt may include simple (S) or detailed (D) instructions, and a serialization of the ontology schema in either BAML, TTL or a verbalized (VERB) textual format. The prompt may optionally (True, False) include ontology instances, and examples (few-shot vs zero-shot prompts). Finally, we may (True, False) use some heuristics to consolidate LLM results.

Figure 3: To mitigate the computational cost of the graph edit distance, we transform KGs from RDF format (a, b) into property graphs (c, d), which have fewer nodes/edges. The example shows the expected graph (a, c) corresponding to the following fragment of text "The corresponding binding energies are 1.43, 0.73, and 0.41 eV for the case of O, N, and C, respectively, in good agreement with previous DFT values [6,17,18,22]" (from [ 25 ] ). The predicted graph (b, d) is generated using LLAMA_3 with a few-shot prompt, detailed instructions, ontology schema serialized in turtle format (TTL), with no instances, and using consolidation. We highlight in green in (a, c) the node and edge (property and value) that are missing in the predicted graph, resulting in a graph edit distance of 2.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	37.8	38.7	30.6	32.4
	GPT_20	12.6	12.6	12.6	12.6	9.0	9.0	7.2	8.1

Table 2: Percentage of success at normalized graph edit distance

NGED^{*}=0.1

for all experimental configurations. Each cell reports the score

P_{j}(0.1)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	62.2	63.1	57.7	59.5
	GPT_20	12.6	12.6	12.6	12.6	27.9	28.8	27.9	28.8

Table 3: Percentage of success at normalized graph edit distance

NGED^{*}=0.2

for all experimental configurations. Each cell reports the score

P_{j}(0.2)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

Figure 4: Percentages of success

P_{\mathcal{F}}

at various thresholds of normalized graph edit distance (

NGED^{*}

) across different configuration families

\mathcal{F}

(

n=\ldots

is the number of configuration within the family). When significant results are found, we report the outcomes of statistical tests beneath each box plot. For comparisons between two families, we use the Mann-Whitney U test, otherwise we use the Kruskal-Wallis test followed by pairwise post hoc analysis using Dunn’s test with Holm’s correction. We conduct all possible post hoc pairwise comparisons, but we report only those that present statistically significant differences:

P_{\mathcal{F}_{i}}<P_{\mathcal{F}_{j}}

indicates that the distribution of percentages of success for

\mathcal{F}_{i}

is statistically significantly lower than that for

\mathcal{F}_{j}

Figure 5: Structure of the proposed ontology to model the domain of irradiated materials

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	27.9	28.8	19.8	19.8
	GPT_20	12.6	12.6	12.6	12.6	5.4	5.4	0.9	1.8

Appendix

Table 4: Percentage of success at normalized graph edit distance

NGED^{*}=0.05

for all experimental configurations. Each cell reports the score

P_{j}(0.05)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	53.2	52.3	48.6	50.5
	GPT_20	12.6	12.6	12.6	12.6	19.8	21.6	16.2	18.9

Appendix

Table 5: Percentage of success at normalized graph edit distance

NGED^{*}=0.15

for all experimental configurations. Each cell reports the score

P_{j}(0.15)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

		zero-shot				few-shot
	examples	False				True
	instances	False		True		False		True
	consolidation	False	True	False	True	False	True	False	True
instructions serialization	LLM
S, TTL	GPT_120	12.6	12.6	12.6	12.6	67.6	69.4	65.8	67.6
	GPT_20	12.6	12.6	12.6	12.6	36.9	36.9	37.8	38.7

Appendix

Table 6: Percentage of success at normalized graph edit distance

NGED^{*}=0.25

for all experimental configurations. Each cell reports the score

P_{j}(0.25)

(see equation 3 ) for the configuration as a percentage. Colors indicate best , 2nd best , and 3rd best for each column.

Figure 7: (A) Ground truth and predicted graphs generated by LLaMA-3 in few-shot (using schema-only prompts with a GED = 2.0 in between both graphs). Multiple formation energy values are associated with distinct methods and reference sources, preserving fidelity to the original text structure. A standard RAG approached (B) failed to answer the question "What is the formation energy for dumbbell SIAs as calculated with DFT, MD or EAM?", despite retrieving the correct document and passage. A naive Graph-RAG approach (C), which prompts an LLM to retrieve an answer, using as input either the full ground truth KG or the document-level predicted one containing that passage, retrieves the correct answer.

Question	Answer (Graph-RAG)
What is the relaxation volume of C15 clusters of size 12 and 48?	The relaxation volume of C15 clusters of size 12 and 48 calculated with DFT is 0.91 $\Omega$ and 0.98 $\Omega$ , respectively.
What is the relaxation volume for all defect types?	The relaxation volume for all defect types is as follows: – Dumbbell SIA: 1.5 $\Omega$ – C15 clusters (size 12): 0.91 $\Omega$ – C15 clusters (size 48): 0.98 $\Omega$ – $\frac{1}{2}\langle 111\rangle$ dislocation loops: 0.905 $\Omega$ – $\langle 100\rangle$ dislocation loops: 0.873 $\Omega$ – Single vacancies: $-0.11$ $\Omega$ – Spherical cavities (radius 1 nm): 0.46 eV – L100 and L111 defects: 0.95 $\Omega$ – Loop 100 (size 6): 0.86 $\Omega$ – Loop 111 (size 6): 0.89 $\Omega$ … etc.
What is the relaxation volume for all defect types as calculated with DFT?	The relaxation volume for all defect types as calculated with DFT is not explicitly provided in the knowledge graph. However, the relaxation volume for specific defect types such as dumbbell SIAs and C15 clusters is mentioned. For dumbbell SIAs, the relaxation volume is 1.62 $\Omega_{0}$ according to DFT simulations by Ma et al. For C15 clusters, the relaxation volume is 0.91 $\Omega_{0}$ for C15 12 and 0.98 $\Omega_{0}$ for C15 48 .
What is the migration energy of a vacancy in iron according to DFT calculations?	The migration energy of a vacancy in iron according to DFT calculations is 0.65 eV.
What is the migration energy of a self-interstitial in iron according to DFT calculations?	The migration energy of a self-interstitial in iron according to DFT calculations is 0.3 eV.
What is the migration energy of a vacancy in iron according to molecular dynamics using empirical potentials?	NA

Appendix

Table 7: Some example competency questions and their answers using Graph-RAG. NA is retrieved correctly when the answer is not in the input KG.

Figure 8: Example of an automatically generated prompt with a detailed instruction, including an ontology-verbalization listing intermediate classes with their attribute and object relations, and a one-shot KG example, serialised in turtle, corresponding to the given passage (the Context) and compliant to the ontology. The corresponding KG for that passage is also visualized: intermediate nodes are shown in pink, leaf entities in blue and datatype literals in green. The sub-graph can also be presented as a table.

Figure 9: Predicted graphs generated by Mistral Large compared to ground truth examples. (A) Mistral-large zero-shot prediction using the ontology with instances require 4 edits (GED=4). Deviations include: using has_defect_size instead of has_number_vacancies , discrepancies in value formatting (e.g.: " $>4$ " vs. "4–", " $>1$ eV" vs "1- eV"), and a missing has_energetics relation in the MethodologyEnergetics node. These errors arise from the model not having seen examples illustrating conventions for representing ranges or for distinguishing when to use similar properties. (B) Mistral Large in few-shot mode, even with access to the full ontology schema, fails to separate Interstitial_Cluster and Vacancy_Cluster into distinct DefectsEnergetics nodes (GED = 10.0), violating the functional property constraint of has_defect_type , which permits only one value per node. It also extracts extra nodes for large clusterswith a valid Extrapolation method, which is correctly mentioned in the text but absent from the ontology and ground truth.

KG Construction

Knowledge graph construction turns text, tables, images, and databases into structured entities and relationships that machines can query and reason over.

See all 197 papers

Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models
Oct 21, 2026Aravindan Kamatchi Sundaram, B. Pabitramohan Prusty, Rohit BatraDocument Information Extraction LLM Information Extraction
Department of Metallurgical and Materials Engineering, Indian Institute of Technology Madras, Chennai 600036, India · Center for Atomistic Modelling and Materials Design, IIT Madras, Chennai 600036, India · Wadhwani School of Data Science & Artificial Intelligence, IIT Madras, Chennai 600036, India
LLM-Guided Ontology-Driven Knowledge Graph Construction from Unstructured Text
Sep 15, 2026Abdelhadi Belfadel, Maxence Gagnant, Joseph Kattan +1KG Construction Ontology Learning
Institute for Technological Research - IRT SystemX 2 Bd Thomas Gobert, 91120 Palaiseau, France.
From Papers to Property Tables: A Priority-Based LLM Workflow for Materials Data Extraction
Apr 8, 2026Koushik Rameshbabu, Jing Luo, Ali Shargh +2Document Information Extraction LLM Information Extraction
Department of Applied Mathematics and Statistics, Johns Hopkins University · Department of Mechanical Engineering, Johns Hopkins University

Oct 21, 2026J/K move · Enter open · S save

Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models

Aravindan Kamatchi Sundaram, B. Pabitramohan Prusty, Rohit Batra

Department of Metallurgical and Materials Engineering, Indian Institute of Technology Madras, Chennai 600036, India · Center for Atomistic Modelling and Materials Design, IIT Madras, Chennai 600036, India · Wadhwani School of Data Science & Artificial Intelligence, IIT Madras, Chennai 600036, India

Document Information Extraction LLM Information Extraction Materials Science

Open paper AskPDF arXiv

Over the past decade, substantial efforts have been devoted to developing machine learning (ML) methods for extracting materials data and, where possible, materials knowledge directly from the scientific literature. This review traces the evolution of these methods from heuristic, rule-based pipelines through natural language processing (NLP) models and transformer-based encoders to, lastly, large language models (LLMs) and agentic systems capable of extracting materials information across multiple modalities, including text, tables, and figures. Beside construction of large, queryable materials databases and knowledge graphs that support downstream materials discovery, we discuss how these advances have enabled new capabilities, such as, generic question answering across materials science, AI co-scientists and chatbots for research assistance, and closed-loop optimization and discovery when integrated with autonomous laboratories. We critically assess the datasets generated using these approaches, their extraction accuracy and computational costs, and the respective strengths and limitations of different methodologies, and even provide practical guidelines for developing more reliable and scalable materials data extraction pipelines. Finally, we identify key open challenges, including the extraction of full materials structure-property-processing-performance tetrahedron at the sample level, the scarcity of reliable benchmark datasets for systematic evaluation, and the resolution of non-standard materials terminology across different document modalities.

Abstract

KG Construction

Explore similar work

Automated Materials Data and Knowledge Extraction using Machine Learning and Large Language Models

LLM-Guided Ontology-Driven Knowledge Graph Construction from Unstructured Text

From Papers to Property Tables: A Priority-Based LLM Workflow for Materials Data Extraction