HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature

Devvrat Joshi; Islem Rekik

cs.CLMar 24, 2026

HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature

Authors: Devvrat Joshi, Islem Rekik

Organizations: BASIRA Lab, Imperial-X (I-X) and Department of Computing Imperial College London, London, United Kingdom

Abstract

Automated knowledge graph (KG) construction is essential for navigating the rapidly expanding body of scientific literature. However, existing approaches struggle to recognize long multi-word entities, often fail to generalize across domains, and typically overlook the hierarchical nature of scientific knowledge. While general-purpose large language models (LLMs) offer adaptability, they are computationally expensive and yield inconsistent accuracy on specialized tasks. As a result, current KGs are shallow and inconsistent, limiting their utility for exploration and synthesis. We propose a two-stage framework for scalable, zero-shot scientific KG construction. The first stage, Z-NERD, introduces (i) Orthogonal Semantic Decomposition (OSD), which promotes domain-agnostic entity recognition by isolating semantic "turns" in text, and (ii) a Multi-Scale TCQK attention mechanism that captures coherent multi-word entities through n-gram-aware attention heads. The second stage, HGNet, performs relation extraction with hierarchy-aware message passing, explicitly modeling parent, child, and peer relations. To enforce global consistency, we introduce two complementary objectives: a Differentiable Hierarchy Loss to discourage cycles and shortcut edges, and a Continuum Abstraction Field (CAF) Loss that embeds abstraction levels along a learnable axis in Euclidean space. This is the first approach to formalize hierarchical abstraction as a continuous property within standard Euclidean embeddings, offering a simpler alternative to hyperbolic methods. We release SPHERE (https://github.com/basiralab/SPHERE), a multi-domain benchmark for hierarchical relation extraction. Our framework establishes a new state of the art on SciERC, SciER, and SPHERE, improving NER by 8.08% and RE by 5.99% on out-of-distribution tests. In zero-shot settings, gains reach 10.76% for NER and 26.2% for RE.

Figures & tables

Models	SciERC	SciER	BioRED	SemEval	CS		Physics		Bio		MS
					Sup	ZS	Sup	ZS	Sup	ZS	Sup	ZS
Supervised Baselines
SciBERT Ye et al. (2022)	67.52	70.71	89.15	49.14	68.19	57.02	72.90	61.22	75.83	68.45	67.29	57.14
PL-Marker Yan et al. (2023)	70.32	74.04	86.41	47.69	68.64	56.39	72.83	60.51	75.78	66.17	66.72	57.92
HGERE Yan et al. (2023)	75.92	81.19	89.43	48.25	69.82	58.95	72.46	60.67	76.42	68.51	67.24	58.03
UniversalNER-7b Zhou et al. (2024)	66.09	73.13	88.46	47.60	OOM

Table 1: F1 scores (%) of different models on NER benchmarks. SPHERE: CS, Physics, Bio, MS report both supervised (Sup) and zero-shot (ZS) results. OSD refers to orthogonal semantic decomposition. Z-NERD w/o OSD and w/o TCQK refer to ablations.

Models	SciERC			SciER			BioRED	SemEval 2017
	Hier.	Peer	Overall	Hier.	Peer	Overall	Overall	Hier.	Peer	Overall
Supervised Models
PL-Marker Ye et al. (2022)	35.60	44.97	41.63	40.25	61.84	56.78	29.87	32.96	43.40	37.19
HGERE Yan et al. (2023)	37.72	47.35	43.86	43.79	64.35	58.47	32.39	33.81	45.73	38.63
PURE Zhong and Chen (2021)	34.39	38.46	36.78	38.53	56.21	49.35	29.41	28.94	41.35	34.92
Zero-Shot LLM Models

Table 2: Rel+ F1 scores (%) of different models on SciERC, SciER, BioRED, and SemEval 2017 for two relation types ( Hierarchical and Peer ), along with overall F1 across all relation types. For BioRED, which only has Peer relations, the overall score equals the peer score. The same entity prediction method is used across models for fair comparison. All values are rounded to two decimals.

Supervised Models
Models	Comp. Sci.			Physics			Biology			Mat. Sci.
	Hier.	Peer	All	Hier.	Peer	All	Hier.	Peer	All	Hier.	Peer	All
PL-Marker Ye et al. (2022)	51.98	57.04	55.29	50.22	56.48	53.51	52.35	53.76	53.03	52.96	53.27	53.12
HGERE Yan et al. (2023)	54.20	59.86	57.93	53.17	58.90	56.28	54.52	56.47	55.21	55.84	55.86	55.43
Proposed Approaches
HGNet (ours)	77.40	81.36	79.51	76.93	83.47	80.60	82.53	84.29	83.74	81.91	85.64	83.65

Table 3: Rel+ F1 scores (%) on SPHERE dataset variants (Computer Science, Physics, Biology, Material Science) for two relation types ( Hier. , Peer ). We also report overall F1 across all relation types. All models use the same entity prediction method for a fair comparison. Subscripts indicate improvement over SOTA model HGERE.

Models	Comp. Sci.			Mat. Sci.
	Hier.	Peer	All	Hier.	Peer	All
PL-Marker Ye et al. (2022)	28.72	28.41	28.56	33.10	34.22	33.85
HGERE Yan et al. (2023)	29.93	29.63	29.81	36.27	39.41	37.97
HGNet (ours)	59.36	64.07	62.60	69.92	71.33	70.62

Table 4: Zero-shot Rel+ F1 (%) when trained on Physics+Biology and evaluated on Comp. Sci. and Mat. Sci. datasets. Due to the expensive nature of these experiments, we only tested our zero-shot performance on the best performing state-of-the-art models in our previous experiments.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Continuous axis of abstraction for topics in physics.

Hyperparameter	Z-NERD	HGNet
Encoder Base Model	SciBERT-base	SciBERT-base
Learning Rate	$2\times 10^{-5}$	$1\times 10^{-5}$
Batch Size	16	8
Optimizer	AdamW	AdamW
Dropout Rate	0.1	0.2
Max Sequence Length	512	512

Appendix

Table 5: Key hyperparameters for Z-NERD and HGNet.

Dataset	Domain	Docs	Relations	Graph Scope	Hierarchy Source
SciERC	CS (AI)	500	$\sim$ 4.6k	Local (Doc-Level)	Inferred from Text
BioRED	Biomed	600	$\sim$ 38k	Local (Doc-Level)	Inferred from Text
SciER	CS	106	$\sim$ 12k	Local (Doc-Level)	Inferred from Text
SPHERE	4 Domains	10,000	111,000	Global (Corpus-Level)	Pre-defined Scaffold

Appendix

Table 6: Comparison of SPHERE against standard scientific IE benchmarks. A key distinction is Graph Scope: standard datasets define hierarchies locally within isolated documents (fragmented), whereas SPHERE is generated from a single Global Knowledge Graph, ensuring hierarchical consistency across the entire corpus.

Metric	Training Source	SciERC (Test)	SciER (Test)
HGERE Yan et al. (2023)	Full Supervised Training	43.86%	56.28%
HGERE (Zero-shot transfer)	SPHERE-CS Training Only	25.62%	28.34%
HGNet (Zero-shot transfer)	SPHERE-CS Training Only	46.55%	59.17%

Appendix

Table 7: Performance comparison on SciERC and SciER datasets.

Panel of Figure 2 — Figure 2: Average Orthogonal Semantic Velocity for tokens at entity boundaries (’Start’/’End’) vs. ’Non-entity’ tokens for SPHERE-CS (left) and SciER (right). The clear separation provides visual evidence for Hypothesis 3.1.

Panel of Figure 3 — Figure 3: Distribution of abstraction scores across different scientific domains, showing distinct patterns for each field.

Model	Geometry	SciERC (Overall)	SPHERE-CS (Overall)
HGCN Chami et al. (2019)	Hyperbolic ( $\mathbb{D}^{n}$ )	45.82	66.35
Order-Embeddings Vendrov et al. (2015)	Cone ( $\mathbb{R}^{n}_{+}$ )	44.27	67.97
HGNet (Ours)	Euclidean + CAF	53.19	79.51

Appendix

Table 8: Comparison against Geometric Baselines (Rel+ F1 %). HGNet outperforms non-Euclidean methods on both standard and large-scale hierarchical datasets.

Model	Prompting Strategy	SciERC	SciER
Llama-3-8B	Zero-Shot	13.72	14.95
Llama-3-8B	3-Shot CoT	19.45	25.18
HGNet	Supervised	53.19	62.36

Appendix

Table 9: Impact of Prompting Strategy on Llama-3-8B Performance (Rel+ F1 %).

Figure 4: Example of physics domain hierarchical knowledge graph. Hierarchy radially extends outward.

Figure 5: Loss plot of HGNet on computer science SPHERE dataset.

Large Language Models
Model	Params	GFLOPs	Speed (doc/s)	Mem (GB)	Zero-Shot Gen.
Llama-3-70B	$\sim$ 70B	$>$ 140k	$\sim$ 0.5	OOM	High
Llama-3-8B	$\sim$ 8B	$>$ 16k	$\sim$ 4.2	16.0+	Moderate
Specialized SOTA
PL-Marker	$\sim$ 220M	44.0	12.4	7.2	Low
HGERE	$\sim$ 220M	22.5	14.1	9.5	Low

Appendix

Table 10: Efficiency Landscape on SciERC (A30 GPU, Batch=8). GFLOPs are estimated per input instance. HGNet (Full Pipeline) provides superior throughput compared to pipeline-based SOTA models despite robust parameter capacity.

Stage	Component	Params (M)	% of Total
Stage 1: Z-NERD	Specialized SciBERT Encoder	109.5	37.4%
Stage 1: Z-NERD	Multi-Scale TCQK (8 Heads)	42.4	14.5%
Stage 2: HGNet	Specialized SciBERT Encoder	109.5	37.4%
Stage 2: HGNet	Hierarchical GNN Layers	31.6	10.7%
Total	Full Two-Stage Pipeline	$\sim$ 293.0	100%

Appendix

Table 11: Detailed parameter breakdown of the HGNet framework (Two-Stage Configuration, worst case parameters). The Z-NERD stage incorporates high-capacity TCQK attention (8 Heads) to resolve complex multi-word boundaries, while HGNet utilizes specialized message-passing layers.

Figure 6: Main figure explaining the proposed Z-NERD algorithm. For TCQK, multi-head for each convolution has been shown as single head for simplicity. B refers to begin entity, I refers to inside entity and O refers to outside entity.

Figure 7: Main figure of proposed HGNet illustrating all proposed components. For clarity, we omit $\mathcal{L}_{\text{regression}}$ , since it is simply a regression loss applied over the graph topology, similar in nature to standard losses such as mean squared error or binary cross entropy.

KG Construction

Knowledge graph construction turns text, tables, images, and databases into structured entities and relationships that machines can query and reason over.

See all 186 papers

Apr 21, 2026

Construction of Knowledge Graph based on Language Model

Knowledge Graph (KG) can effectively integrate valuable information from massive data, and thus has been rapidly developed and widely used in many fields. Traditional KG construction methods rely on manual annotation, which often consumes a lot of time and manpower. And KG construction schemes based on deep learning tend to have weak generalization capabilities. With the rapid development of Pre-trained Language Models (PLM), PLM has shown great potential in the field of KG construction. This paper provides a comprehensive review of recent research advances in the field of construction of KGs using PLM. In this paper, we explain how PLM can utilize its language understanding and generation capabilities to automatically extract key information for KGs, such as entities and relations, from textual data. In addition, We also propose a new Hyper-Relarional Knowledge Graph construction framework based on lightweight Large Language Model (LLM) named LLHKG and compares it with previous methods. Under our framework, the KG construction capability of lightweight LLM is comparable to GPT3.5.

Qiubai Zhu, Qingwang Wang, Haibin Yuan +2

Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China · Yunnan Key Laboratory of Computer Technologies Application, Kunming University of Science and Technology, Kunming 650500, China · Tin Branch, Yunnan Tin Co., LTD.

KG Construction LLM Information Extraction Knowledge Graphs

Ask PDF

Jun 11, 2026

Agents-K1: Towards Agent-native Knowledge Orchestration

Current LLM-based research agents have advanced through agent orchestration, yet largely overlook scientific knowledge orchestration. Existing works often reduce papers to abstracts, surface mentions, and flat cites edges, omitting key entities, claims, evidence, mechanisms, and method lineages essential for scientific reasoning. To this end, we introduce Agents-K1, an end-to-end knowledge orchestration pipeline that converts raw documents into agent-native scientific knowledge graphs. Agents-K1 integrates three components under a unifying theoretical foundation: a multimodal parser whose five-module schema captures entities, multimodal evidence, citations, and typed inter-entity relations across the full paper rather than abstracts alone; a 4B information-extraction backbone trained with GRPO under a rule-based reward; and a graphanything CLI, a tri-source agent interface that unifies web search, multimodal graph retrieval, and cross-document traversal. On top of this, we process 2.46 million scientific papers across six subjects to produce Scholar-KG, of which we release a one-million-paper subset, and the full Scholar-KG is accessible via the SCP link below. The same pipeline can be extended to general-domain corpora and to schema-conformant data synthesis. Extensive experiments demonstrate that Agents-K1 achieves superior performance in scientific information extraction, knowledge graph construction, and multi-hop scientific reasoning.

Zongsheng Cao, Bihao Zhan, Jinxin Shi +23

Shanghai Artificial Intelligence Laboratory · East China Normal University · Fudan University

KG Construction AI for Science Document Information Extraction Knowledge Representation LLM Agents

Ask PDF

Sep 2, 2026

HyGRAIL: Cost-Aware and Evidence-Grounded Scientific Hypothesis Discovery over Knowledge Graphs

Scientific knowledge graphs organize entities and relations extracted from scientific literature, but they remain inherently incomplete. Missing typed links in such graphs can therefore represent plausible scientific hypotheses, such as unexplored associations between materials and applications. However, scientific hypothesis discovery is challenging because true discoveries are extremely sparse among typed candidate pairs: graph neural networks (GNNs) are efficient but unreliable for ambiguous cases, while large language models (LLMs) are knowledgeable but too costly to apply exhaustively and are not naturally grounded in graph structures. We propose HyGRAIL, a cost-aware and evidence-grounded framework that combines heterogeneous GNN triage with LLM-based hypothesis review. HyGRAIL first uses a GNN to score candidate hypotheses and identify a validation-calibrated ambiguous region, routing only graph-uncertain cases to LLM review. For each routed hypothesis, HyGRAIL retrieves node-level associations and multi-hop relational paths from the knowledge graph (KG), then converts this structured evidence into natural language through template-based or LLM-based naturalization. An LLM review agent finally judges each hard hypothesis using the naturalized evidence and validation-selected decision criteria. On MatKG, HyGRAIL achieves the best F1 score of 0.429, improving over the strongest prior baseline by 0.242 F1 points and over the GNN-only baseline by 0.322. Meanwhile, GNN triage reduces the LLM call rate by 54.36% on average. Ablation studies further show that retrieved graph evidence is crucial for reliable hypothesis verification and that compact, two-sided evidence is more effective than simply increasing retrieval quantity.

Yihang Sun, Zhihan Zhu, Zhiyuan Jiang +3

University of Illinois Urbana-Champaign

LLM Grounding Scientific Hypothesis Generation KG Completion

Ask PDF

Abstract

KG Construction

Explore similar work

Construction of Knowledge Graph based on Language Model

Agents-K1: Towards Agent-native Knowledge Orchestration

HyGRAIL: Cost-Aware and Evidence-Grounded Scientific Hypothesis Discovery over Knowledge Graphs