Methods for Knowledge Graph Construction from Text Collections: Development and Applications

Vanni Zavarella

Figure 2.1 : Fragment of a TV series knowledge graph (a.), with corresponding RDF serialization in Turtle format (b.).

Figure 2.2 : Example of applying Entity Extraction and Linking and Relation Extraction to an input text, using DBPedia as a reference KB.

Figure 2.3 : Dependency tree of an example sentence (root node omitted).

Figure 2.4 : An example of sentence to triple transformation using Open IE.

Figure 2.5 : A sample BiLSTM-CRF architecture for NER. Adapted from [ 67 ] .

Figure 2.6 : A CNN architecture for relation classification of entity pairs in a sentence. Adapted from [ 147 ] .

Figure 2.7 : A span-based architecture for joint NER-RE. Adapted from [ 37 ] .

Figure 2.8 : A baseline instruction prompt for RE.

Figure 2.9 : Schema of a training data template for instruction tuning on a RE task.

Figure 3.1 : Snapshot of a Kibana dashboard visualization of the tweet collection Elastic Search index.

Figure 3.2 : Merged flowchart of the pipelines for knowledge graph generation from micro-blogging and news wire text sources.

Figure 3.3 : Example of tweet preprocessing.

Figure 3.4 : Visualization of candidate entities (highlighted in blue) extracted from a few sample tweets.

Target dependency paths
$[nsubj,dobj]$
$[acl,relcl,dobj]$
$[acl,dobj]$
$[nsubjpass,agent,pobj]$
$[nsubj,dobj,conj]$
$[nsubj,conj]$

Table 3.1 : List of target and some of the discarded relation dependency paths.

Figure 3.5 :

S

score over number of generated clusters for a subset of best-scoring UMAP-HDSCAN hyperparameter configurations on GLOVE embeddings of relations, from the tweet collection, with the picked up sub-optimal value circled in red.

Relation Verb	Relation Predicate	Example
fuel	FUEL	‘How the UR+ Ecosystem is Fueling Cobot Market Growth’
driven by	FUEL	‘Digital transformation in Ho Chi Minh is being driven by remote working’
accelerated by	FUEL	‘huge social trends being accelerated by the pandemic.’
identify	IDENTIFY	‘Machine learning can identify signs of Alzheimers in patients ’
quantify	IDENTIFY	‘Research quantifies G’s potential in roaming and manufacturing ’
predict	IDENTIFY	‘AI-supported test can predict eye disease that leads to blindness’

Table 3.2 : Sample relation verb-predicate mapping.

Extraction Method	Precision
OpenIE Extractor	0.52
PoST Extractor	0.17
Dependency-based Extractor	0.77
Entity and Relationship Refiner	0.31
Triplétoile	0.82

Table 3.3 : Precision of the triples extracted from a set of alternative methods from a collection of 500 tweets, using a combination of Triplétoile and DyGIEpp input entities.

Subject Entity	Relation	Object Entity
pandemic	accelerate	digital_transformation
artificial_intelligence	impact	insurance_sector
microsoft	buy	riskiq
data-driven_insight	drive	decision-making
agile_business	demand	effective_marketing_capability
hootsuite	buy	ai_chatbot_firm

Table 3.4 : A sample of statements extracted from the tweet collection by the Triplétoile pipeline.

Figure 3.6 : A shortened example of reification for a Statement concerning the instance machine_learning , grounded by 6 tweets, with the three dots referring to the hidden dtsmm-ont:comesfromTweet predicates.

Figure 3.7 : A sample DTSMM_KG subgraph showing a few claims for the instance machine_learning .

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 3.8 : A sample reification for a statement concerning the DTSMM_KG resources dhnewskg:drug_tamoxifen and dhnewskg:receptor .

Subject Entity	Relation	Object Entity	Support
Italy	report	coronavirus death	374
clinical trial	involve	patient	128
interactive graphic	track	global spread	90
fitch	undertake	sensitivity analysis	75
administration	approve	drug	47
meningitis immunity	fight	endometrial cancer	35

Table 3.5 : Sample statements from DTSMM_KG , with their support.

DBpedia Entity Type	#Matched Entities	#Unique Entities
DBpedia:Organisation	20050	2640
DBpedia:Company	15667	1736
DBpedia:Country	12124	324
DBpedia:Disease	7881	918
DBpedia:Person	6594	2611
DBpedia:ChemicalSubstance	6583	1378

Table 3.6 : Number of matches and unique matches of the 20 most represented DBpedia entity types in DHNEWS_KG .

Figure 3.9 : Query returning all DHNEWS_KG statements with the graph entity dhnewskg:biogen as rdf:subject .

Figure 4.1 : Sample Entity and Relation annotation from the SciERC dataset (a) and from an AECO paper abstract [ 47 ] (b), following the SciERC annotation schema. The latter shows a Method “Microalgae Photobioreactor System” and Task “Indoor Air Remediation” extracted respectively as the head and tail of a Used-for relation, resulting in the triple ⟨Microalgae Photobioreactor System; Used-for ;Indoor Air Remediation⟩, which encodes the claim that a “Microalgae Photobioreactor System” method is applied to solve/achieve the task of “Indoor Air Remediation”.

Figure 4.2 : Flowchart of the text processing pipeline used in this research. The pipeline is structured into four main phases: 1. Data collection from the OpenAlex API; 2. Topic Modeling using optimization of BERTopic architecture and expert tuning; 3. Bibliometric analysis of collaboration networks for each consolidated topic; 4. Extraction of scientific Knowledge Graphs and generation of trend analysis for each consolidated topic, using the SKG-AECO pipeline.

Figure 4.3 : Log scale monthly time series of the overall publications and publications per country, for the 15 top publishing countries.

	Train	Test
Sentences	816	200
Negative Sentences	536	124
Task Entities	512	123
Method Entities	388	89
Metric Entities	51	22
Used-for Relations	196	42

Table 4.1 : Summary counts of the SciERC AECO dataset.

Figure 4.4 : Average topic coherence values against the number of clusters for a subset of best-performing UMAP-HDBSCAN hyperparameter settings.

Figure 4.5 : Reduced 2-dimensional visualization of the optimized 52 topic clusters of research papers, embedded using a Sentence Transformer model. The indicator lines originating from cluster labels (laid out here in rings around the data map, for clarity) point to each cluster’s medoids. The -1:no_topic label denotes the set of outlier articles.

Figure 4.6 : Evolution over time of AECO macro topics.

Figure 4.7 : Sample conversion of a SciERC AECO training instance into a prompt-completion pair for instruction fine-tuning using json-style annotation formalism.

T1

,

T2

are entity indexes. Notice that no definitions of Entity and Relation semantics are provided in the instruction part of the prompt.

Figure 4.8 : A detailed flowchart of the SKG-AECO pipeline.

Extraction Method	Precision
Random Triples	0.695
High Support	0.758
Low Support	0.637
LLM-generated	0.683

Table 4.2 : Triple evaluation over a set of 300 triples.

Figure 4.9 : Sample SPARQL query returning aeco-ont:Task entities that are claimed to be applying the Method aeco:green_roof according to the AECO graph statements, together with the URLs of the papers supporting the claims.

Figure 4.10 : Trend analysis of the top 20 Tasks for macro-cluster 0, with the Tasks listed in the legend at the bottom of the plot. The y-axis measures the ratio of articles mentioning the Tasks to the overall number of articles in the cluster.

Figure 4.11 : Trend analysis of the top 20 Methods for macro-cluster 0, with the Methods listed in the legend at the bottom of the plot. The y-axis measures the ratio of articles mentioning the Method to the overall number of articles in the cluster.

Entity Type	Examples
Drug	morphine, ibuprofen
Adverse Drug Event (ADE)	nausea, seizures
Reason	vitamin K deficiency
Dosage	2 units, stress dose
Strength	10 mg, 60 mg/0.6 mL
Form	capsule, syringe, tablet

Table 5.1 : The entity types annotated in the n2c2 2018 shared task .

MIMICause Relation	HF label	# Train Instances	# Eval Instances	# Test Instances
Cause(e1,e2)	0	254	64	36
Cause(e2,e1)	1	266	67	37
Enable(e1,e2)	2	126	31	17
Enable(e2,e1)	3	126	32	18
Prevent(e1,e2)	4	188	47	26
Prevent(e2,e1)	5	179	45	25

Table 5.2 : Distribution of causal relation labels over train, eval and test splits of the MIMICause dataset. The second column contains the numerical equivalents of relation labels in the adopted Hugging Face distribution.)

text	drug	effect	label
Methemoglobinemia after axillary block with bupivacaine and additional injection of lidocaine in the operative field.	bupivacaine	Methemoglobinemia	1
Methemoglobinemia after axillary block with bupivacaine and additional injection of lidocaine in the operative field.	lidocaine	Methemoglobinemia	1
Cutaneous sarcoidosis during interferon alfa and ribavirin treatment of hepatitis C virus infection: two cases.	ribavirin	cutaneous sarcoidosis	1
Cutaneous sarcoidosis during interferon alfa and ribavirin treatment of hepatitis C virus infection: two cases.	ribavirin	infection	0
Cutaneous sarcoidosis during interferon alfa and ribavirin treatment of hepatitis C virus infection: two cases.	interferon	hepatitis	0

Table 5.3 : Positive examples of drug reaction case reports from the ADE dataset and two synthetic negative instances. The negative instances in rows four and five are generated by entity pair sampling from the sentence in row three.

Drug	Condition	Review
ciprofloxacin	urinary tract infection	i had a urinary tract infection so bad that when i pee it smells but when i started taking ciprofloxacin it worked it’s a good medicine for a urinary tract infections.
ziana	acne	when i first started using ziana, i only had acne in between my eyebrows, chin, and the nose area. my acne worsened while using it and then it got better. but after about 4 months of using it, it became ineffective. so i now have acne between my eyebrows, chin, cheeks, forehead, and the nose area. its great at first but after a while it made my face even worse than before i used the product.
nuvaring	birth control	i tried the nuvaring. this was my first form of any birth control. this was very easy to put inside and very easy to take out. i didn’t feel the ring ever. i thought it was amazing until i started to get huge deep pimples. they were impossible to get rid of.

Table 5.4 : Sample reviews with target entity metadata from the Drug Reviews dataset.

Figure 5.1 : Sample zero-shot prompt with explicit semantic definition of the MIMICause relation labels (top) and a LLM-generated response (bottom).

Figure 5.2 : Transformation of a MIMICause datapoint into an instruction prompt for model fine-tuning.

	Model	Size	Method	N examples	Micro F1	Macro F1
BL	BERT+Ent	110M	FT	1953	-	0.54
BL	Clinical-BERT+Ent	110M	FT	1953	-	0.56
zero-shot	MistralOrca	7B	InstPrompt	-	0.096	0.064
	DeepSeek-Qwen-Distill	32B	InstPrompt	-	0.267	0.172
	MistralOrca	7B	SumAsk	-	0.449	0.343
	DeepSeek-Qwen-Distill	32B	SumAsk	-	0.338	0.163

Table 5.5 : F1 performance values on the test split of the MIMICause dataset of various combinations of models, parameter sizes, learning examples, and learning methods, against the two encoder model Baselines (BL). The reported Macro F1 values are averaged over the nine MIMICause relation categories. In the Method column, FT denotes Fine-Tuning, SumAsk and 2-Chain correspond to the two implementations of Prompt Chaining, iCL represents In-context Learning and CoT refers to Chain-of-Thought. The best-performing configurations within each general learning category are highlighted in bold.

Figure 5.3 : Training instance frequency analysis. Distributions of the F1 scores (Y-axis), collapsed by all fine-tuned models, on each of the nine classes in MIMICause, as a function of the each class’ occurrence in the train split of the MIMICause dataset (X-axis).

Figure 5.4 : Training instance frequency analysis per model: F1 scores (Y-axis) obtained by the different fine-tuned models on each of the nine classes in MIMICause, as a function of the each class’ occurrence in the train split of the MIMICause dataset (X-axis).

	Prompt Dimension	v0	v1	v2	v3	v4	Std Dev (%)
instr. prompt	Persona	0.096	0.100	0.189	0.141	0.141	$\pm 28.35$
	Phrasing	0.096	0.059	0.063	0.052	0.107	$\pm 32.44$
	Definitions	0.096	0.115	0.0929	0.171	0.237	$\pm 43.18$
	Format	0.096	0.130	0.126	0.167	0.118	$\pm 20.20$
	Explain	0.096	0.078	0.163	0.115	0.089	$\pm 30.92$
	Ordering	0.096	0.223	0.282	0.301	0.278	$\pm 35.37$

Table 5.6 : Micro F1 performance scores on the test split of the MIMICause dataset of five variants of the base instruction prompt illustrated in Figure 5.1 in Appendix F , for the MistralOrca model. Each row contains prompt design variants with respect to the target dimension indicated in the left column headings. The last column reports standard deviation over those variations.

Figure 5.5 : Sensitivity analysis of the best performing fine-tuned model, MedLlama , to variation over Lora rank, learning rate, and training batch size hyperparameters (respectively

rank

,

lr

, and

bs

in the Figure). The plots show the validation loss values for 12 intervals of 50 evaluation steps, for a total of 18 model configurations listed.

Model	20% ADE	ADE balanced
	F1	macro F1	micro F1
Mistral-7B-v0.1	0.956	0.850	0.851
MistralOrca	0.959	0.766	0.772
Llama-2-7b	0.913	0.698	0.712
Llama-3.1-8B-Instruct	0.978	0.685	0.708
gemma-2-9b	0.948	0.789	0.793

Table 5.7 : F1 scores of the fine-tuned models on a 0.2 random sample and a synthetic, 800-sized balanced sample of ADE case reports.

Relation Type	Cohen $\kappa$	Fleiss $\kappa_{F}$	Precision
Cause	0.706	0.707	0.70
Enable	0.591	0.576	0.60
Prevent	0.831	0.808	0.78
Hinder	0.763	0.746	0.60
Other	0.770	0.741	0.97
Overall	0.739	0.728	0.73

Table 5.8 : Precision scores of CLiMA on the Drug Reviews (Druglib.com) data sample aggregated for relation groups, together with average pair-wise Cohen

\kappa

and Fleiss

\kappa_{F}

IAA coefficients among human annotators.

Drug	Causal_Relation	Condition	Support
mirena	Cause	birth control	60
accutane	Cause	acne	33
viibryd	Enable	depression	5
methotrexate	Enable	psoriasis	2
lexapro	Prevent	anxiety	269
vyvanse	Prevent	adhd	142

Table 5.9 : Sample statements for the 5 causal relation categories extracted by the MedLlama model, with their support values in the Drug Reviews dataset.

Figure 5.6 : A sample reification for the statement assessing a csldrg-ont:Cause relation between the instances csldrg:accutane and csldrg:acne , extracted from 33 reviews (only 2 are shown here for the sake of simplicity.)

Condition Ontology	Coverage	Drug Ontology	Coverage
OCHV	246	RXNORM	869
IOBC	242	MESH	831
MEDDRA	232	NCIT	812
MESH	229	OCHV	789
MDM	227	MDM	687
SNOMEDCT	226	CHEBI	589

Table 5.10 : Coverage of the top 10 ontologies for Drug and Condition entities in the causal KG.

Figure 5.7 : Sample SPARQL query returning all CausalDrugsKG statements with the graph entity csldrg:accutane as csldrg-ont:subject .

Figure 5.8 : Chord diagram illustrating the strength (statement support) of relationships among top Drug and Condition entities in CausalDrugsKG .

Figure 6.1 : Log scale triple distribution over support, for the

\langle Method;Used-for;Task\rangle

and

\langle Method;Used-for;Method\rangle

triple set used in the trend analysis.

	Hyperparameter	Explanation	Value Range
UMAP	n_neighbors	the number of neighboring data points UMAP will look at when attempting to learn the manifold structure of the data	[5,15,30]
	min_dist	the minimum distance apart that points are allowed to be in the low dimensional representation	[0.0,0.1]
	n_components	the dimensionality of the reduced space	[2,5,10,30]
HDBSCAN	min_cluster_size	affects the final number of generated clusters	[5,10,20,50,100,200]
	min_samples	the minimum number of points required in the neighborhood of a point for it to be considered a core point	[None,1,2,5]
	metric	used to calculate the distances	[”euclidean”,”cosine”]

Table 1: Main hyperparameters searched for upon optimizing the UMAP-HDBSCAN interaction.

Embedding Model	Silhouette $\cdot$ Clustered Ratio	Num Clusters
BERT	0.9387	1107
BERT	0.9287	918
BERT	0.9171	1063
Sentence-BERT	0.6852	869
Sentence-BERT	0.6794	978
Sentence-BERT	0.6767	1050

Table 2: The table presents clustering score values and the number of output clusters for the top three performing UMAP-HDBSCAN configurations across three tested embedding models. It’s worth noting that the dataset comprises a total of 29,335 relation instances for contextualized BERT and Sentence-BERT embeddings. In contrast, for static GloVe embeddings, we consolidated single occurrences of each relation form, resulting in a final set of 2,539 relations due to their context-independent vector representations.

Figure 2: Topic Coherence score heat map for the optimized topic model.

Figure 3: Optimized topics displayed in a reduced 2-dimensional embedding space, showing inter-topic distances.

Figure 4: Dendrogram representation of the optimized topics’ hierarchical clustering. The leaves of the tree represent the 52 clusters, the intermediate nodes represent merged clusters, and the height of the merging (distance from the leaves) indicate topic similarity as based on the cosine distance matrix between topic embeddings.

Topic	Count	Representation	Terms
0	39619	Energy Efficiency and Thermal Comfort in Building Environments	[’energy’, ’thermal’, ’lighting’, ’building’, ’comfort’, ’buildings’, ’temperature’, ’heat’, ’study’, ’performance’]
1	22740	Indoor Air Quality and Sustainable Air Conditioning Systems	[’air’, ’indoor’, ’ventilation’, ’conditioning’, ’cooling’, ’quality’, ’temperature’, ’energy’, ’concentrations’, ’control’]
2	14635	Urban Development Strategies and Sustainable City Planning	[’urban’, ’land’, ’planning’, ’development’, ’city’, ’spatial’, ’cities’, ’growth’, ’public’, ’expansion’]
3	10650	Enhancing Child-Friendly Urban Spaces Through Design	[’design’, ’space’, ’product’, ’architectural’, ’public’, ’architecture’, ’students’, ’library’, ’research’, ’study’]
4	10244	Smart city development and urban data management	[’smart’, ’city’, ’parking’, ’cities’, ’data’, ’occupancy’, ’urban’, ’information’, ’development’, ’based’]
5	9518	Urban Resilience and Green Infrastructure in Climate Change Planning	[’urban’, ’resilience’, ’infrastructure’, ’water’, ’climate’, ’green’, ’planning’, ’cities’, ’flood’, ’change’]

Table 3: The consolidated set of macro-topics resulting from topic merging, with their document counts, LLM-generated descriptions, and term based representation.

Figure 5: Average topic coherence values against the ratio of clustered datapoints for a subset of HDBSCAN hyperparameter settings. The color-coded values of the number of resulting clusters are also shown.

Parameter	HF parameter	Prompt	Value
Use sampling	do_sample	all	False
4 bit quantization	load_in_4bit	all	True
quantization scheme	bnb_4bit_quant_type	all	nf4
maximum num of new tokens	max_new_tokens	InstPrompt	256
		iCL	256
		CoT	512

Table 4: Model instantiation and inference parameters used across all zero-shot and few-shot experiments. max_new_tokens parameter for two-step prompt methods SumAsk and 2-Chain is specified as a pair of values, one for each inference call. max_new_tokens is raised to 900 for inference with DeepSeek-Qwen-Distill in all prompting methods, in order to accommodate for the long reasoning chains of this model.

Parameter	HF parameter	Value
LoRA rank	r	8
LoRA alpha	lora_alpha	32
LoRA dropout	lora_dropout	0.05
LoRA target modules	target_modules	[”q_proj”, ”v_proj”]
Validation Metric	metric_for_best_model	eval_loss
Train batch size	per_device_train_batch_size	8

Table 5: LoRA configuration and training parameters used across all fine-tuning experiments.

Figure 7: Comparing Mistral Orca and DeepSeek-Qwen-Distill model responses for a sample zero-shot CoT prompt. The target label to be extracted is 7 ( E2 hinders E1 ). Notice as DeepSeek-Qwen-Distill evaluates, among others (not shown for simplicity), Label 5 option, but eventually opts for Label 7.

Figure 8: Confusion matrix across the nine causal relation classes for the best-performing model, the instruction fine-tuned MedLlama .

Methods for Knowledge Graph Construction from Text Collections: Development and Applications

Abstract

Figures & tables

KG Construction

Traceable Context-Enriched Knowledge Graph Generation with Induced Schemas

A Unified Benchmark for Evaluating Knowledge Graph Construction Methods and Graph Neural Networks

Construction of Knowledge Graph based on Language Model

Abstract

KG Construction

Explore similar work

Traceable Context-Enriched Knowledge Graph Generation with Induced Schemas

A Unified Benchmark for Evaluating Knowledge Graph Construction Methods and Graph Neural Networks

Construction of Knowledge Graph based on Language Model