Hierarchy-Aware Semantic Losses for Knowledge Graph Link Prediction
Authors: Filip Kronström, Ross D. King
Organizations: Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, Sweden · Department of Chemical Engineering and Biotechnology, University of Cambridge, United Kingdom
Knowledge graphs are often accompanied by ontological class hierarchies that encode valuable semantic information, yet many link prediction methods either ignore such hierarchies or incorporate them indirectly through additional graph edges. Recent work introduced hierarchy-aware graph neural networks (GNNs), which use semantic losses derived from box embeddings to encourage satisfaction of subclass relationships during GNN-based representation learning. While this approach has shown promise for biological regression tasks, its effectiveness for knowledge graph link prediction has not been investigated. In this paper we evaluate hierarchy-aware semantic losses on link prediction across three benchmark datasets: AIFB, CoDEx, and BioKG. We combine graph neural network encoders with box-embedding-based semantic losses that encourage learned representations to better satisfy ontology-derived class hierarchies, and compare this approach to both standard link prediction models and models incorporating subclass relations as graph edges. Across all datasets, hierarchy-aware semantic losses significantly improve mean reciprocal rank (MRR) and consistently outperform models that incorporate hierarchy information through additional subclass edges. Relative to the baseline GNN models, MRR improved by 7.6%, 2.4%, and 15.5% on AIFB, CoDEx, and BioKG, respectively. Furthermore, semantic losses consistently outperform the alternative of augmenting the graph with subclass edges. These results are consistent with ontology-derived class hierarchies providing complementary information to graph structure, and suggest that encouraging hierarchical consistency through semantic losses is an effective and comparatively parameter-efficient mechanism for improving knowledge graph link prediction.
Figures & tables
Figure 1: Overview of the hierarchy-aware GNN architecture. Semantic losses are applied to the GNN encoder to produce embeddings encouraged to better satisfy subsumptions defined in ontologies. This is optimised jointly with a task-specific decoder, e.g., for link prediction.
\smaller Dataset
∣E∣
∣R∣
\smaller Train triples
\smaller Val/Test triples
\smaller Hierarchy entities
\smaller Subclass axioms
\smaller Coverage (%)
\smaller Max depth
\smaller Median depth
\smaller AIFB
2,431
36
30,102
416
172
3,890
100.0
5
2
\smaller CoDEx
77,951
69
551,193
30,622
1,986
69,974
80.2
22
10
\smaller BioKG
93,773
51
4,762,678
162,870
64,985
455,560
84.7
58
6
Table 1: Overview of dataset and hierarchy characteristics. ∣E∣ and ∣R∣ denote the number of entities and relations in the original KG. Hierarchy entities and subclass axioms denote the number of additional entities and subclass relations introduced by the hierarchy. Coverage indicates the percentage of KG entities represented in the hierarchy. Domain-specific hierarchy characteristics for BioKG are reported in tab:biokg.
Domain
∣E∣
Hierarchy entities
Subclass axioms
Coverage (%)
Max depth
Median leaf node depth
function
45,085
618
93,040
100.0
16
6
disease
10,687
25,804
163,606
83.4
58
24
sideeffect
9,969
28,217
166,898
91.2
57
16
protein
17,499
2,040
21,906
82.9
6
2
drug
10,533
8,306
10,110
17.1
11
6
Table 2: Hierarchy characteristics for the five domains in BioKG.
ComplEx
R-GCN / GraphSAGE
Dataset
Metric
Baseline
SC-Edges
Baseline
SC-Edges
SemLoss
AIFB
MRR
0.4056 \smaller ± 0.0085
0.4336 \smaller ± 0.0062
0.4758 \smaller ± 0.0237
0.4872 \smaller ± 0.0257
0.5118 \smaller ± 0.0151
#Params (M)
1.32
1.35
0.51
0.51
0.51
CoDEx
MRR
0.3734 \smaller ± 0.0010
0.3700 \smaller ± 0.0014
0.3999 \smaller ± 0.0069
0.4035 \smaller ± 0.0074
0.4095 \smaller ± 0.0025
#Params (M)
39.95
40.96
6.63
6.74
6.69
BioKG
MRR
0.7440 \smaller ± 0.0044
0.6963 \smaller ± 0.0223
0.7232 \smaller ± 0.0785
0.7644 \smaller ± 0.0393
0.8354 \smaller ± 0.0144
Table 3: Link prediction performance (MRR) and number of trainable parameters. Results are reported as mean ± standard deviation over 10 runs. All models trained with the semantic loss significantly outperform the corresponding baseline models (Welch’s two-sided t-test, p<0.05 ).
Figure 2: Evolution of domain-specific semantic losses during training on BioKG. Curves show the median semantic loss over 10 training runs and shaded regions indicate the interquartile range. Losses are reported separately for each ontology domain and GNN layer. The semantic losses decrease throughout training for all domains, indicating increasing satisfaction of ontology-derived hierarchy constraints, although the magnitude of improvement varies between domains. Layer 0 corresponds to the initial node embeddings prior to message passing.
Figure 3: Exploratory comparison between performance improvements obtained by hierarchy-aware semantic losses and hierarchy characteristics across the three evaluated datasets. The x-axis shows the improvement in mean reciprocal rank ( Δ MRR) relative to the corresponding baseline model. Hierarchy density is defined as the number of subclass axioms divided by the number of hierarchy entities. Hierarchy availability is defined as the number of hierarchy entities divided by the number of hierarchy-covered graph entities and reflects the amount of hierarchy information available per covered graph entity.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Embedding dimension
Hidden dimensions
R-GCN bases
NN decoder units
AIFB
32
[64,64,64]
40
-
CoDEx
32
[64,128]
-
[32,8,1]
BioKG
128
[256,256]
-
[64,32,8,1]
Appendix
Table 4: Hyperparameters for the different datasets.
ComplEx
R-GCN / GraphSAGE
Dataset
Metric
Baseline
SC-Edges
Baseline
SC-Edges
SemLoss
AIFB
Hits@1
0.2389 \smaller ± 0.0156
0.2764 \smaller ± 0.0113
0.3478 \smaller ± 0.0227
0.3565 \smaller ± 0.0289
0.3861 § \smaller ± 0.0157
Hits@3
0.5041 \smaller ± 0.0103
0.5303 \smaller ± 0.0106
0.5334 \smaller ± 0.0311
0.5454 \smaller ± 0.0275
0.5671 ∗†‡ \smaller ± 0.0190
Hits@10
0.7327 \smaller ± 0.0113
0.7334 \smaller ± 0.0042
0.7409 \smaller ± 0.0306
0.7558 \smaller ± 0.0289
0.7687 ∗†‡ \smaller ± 0.0127
CoDEx
Hits@1
0.3020 \smaller ± 0.0019
0.2971 \smaller ± 0.0020
0.3266 \smaller ± 0.0078
0.3310 \smaller ± 0.0082
0.3380 § \smaller ± 0.0030
Hits@3
0.4111 \smaller ± 0.0008
0.4085 \smaller ± 0.0016
0.4423 \smaller ± 0.0066
0.4448 \smaller ± 0.0070
0.4478 †‡ \smaller ± 0.0045
Appendix
Table 5: Link prediction performance (Hits@K). Results are reported as mean ± standard deviation over 10 runs. \lx@sectionsign indicates significant improvement for the model trained with the semantic loss over all baseline models. † and ‡ indicate significant improvements compared to the ComplEx and ComplEx with subClassOf -edges respectively. * indicates significant improvement compared to the GNN baseline. Improvements are tested with Welch’s two-sided t-test ( p<0.05 ).
We present a method for finding hierarchy-aware embeddings of knowledge graphs (KGs) using graph neural networks (GNNs) enriched with a semantic loss derived from underlying ontologies. This method yields embeddings that better reflect domain knowledge. To demonstrate their utility, we predict and interpret the effects of gene deletions in the yeast Saccharomyces cerevisiae and learn box embeddings for KGs in the absence of a prediction task. We further show how box embeddings can serve as the basis for evaluating KG revisions. Our yeast KG is constructed from community databases and ontology terms. Low-dimensional box embeddings combined with GNNs are used to predict cell growth for double gene knockouts. Over 10-fold cross validation, these predictions have a mean R2scoreof~0.360, significantly higher than baseline comparisons, demonstrating that high-level qualitative knowledge is informative about experimental outcomes. Incorporating semantic loss terms in the training of the models improves their predictive performance (R2=0.377) by aligning embeddings with ontology structure. This shows that class hierarchies from ontologies can be exploited for quantitative prediction. We also test the trained models on triple gene knockouts, showing they generalise to data beyond those seen in training. Additionally, by identifying co-occurring relations in the yeast KG important for the cell-growth predictions, we construct hypotheses about interacting traits in yeast. A biological experiment validates one such finding, revealing an association between inositol utilisation and osmotic stress resistance, highlighting the model's potential to guide biological discovery.
Filip Kronström, Alexander H. Gower, Daniel Brunnsåker +2
Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, Sweden · Department of Life Sciences, Chalmers University of Technology, Sweden · Department of Industrial Biotechnology, KTH Royal Institute of Technology, Sweden +1
We introduce Graph-Augmented Sequence-to-Sequence (GA-S2S), a novel framework that integrates a T5-small encoder-decoder with a Relational Graph Attention Network (RGAT) to improve link prediction in knowledge graphs. While existing Seq2Seq models rely solely on surface-level textual descriptions of entities and relations and at best, flatten the neighborhoods of a query entity into a single linear sequence, thereby discarding the inherent graph structure, GA-S2S jointly encodes both textual features and the full k-hop subgraph topology surrounding the query entity. By integrating raw encoder outputs with RGAT's relation-aware embeddings, our model captures and leverages richer multi-hop relational patterns and textual information. Our preliminary experiments on the CoDEx dataset demonstrate that GA-S2S outperforms competitive Seq2Seq-based baseline models, achieving up to a 19% relative gain in link prediction accuracy.
Luu Huu Phuc, Ratan Bahadur Thapa, Mojtaba Nayyeri +3
Analytic Computing, KI, University of Stuttgart, Stuttgart, Germany · Bosch Center for Artificial Intelligence, Stuttgart, Germany · WAIS, University of Southampton, United Kingdom
Knowledge graph (KG) foundation models (KGFMs) are zero-shot generalizers: trained once, they can predict links on unseen graphs without retraining. However, understanding when and how they can robustly generalize across KGs is still an open question. In this paper, we shed some light on their generalization mechanisms highlighting how their performance on unseen KGs is not uniform when it comes to partially seen links, which we call half-links. In fact, we show that to predict a test triple (h,r,t) it might suffice in practice to have observed the half-link (h,r) or (r,t) in the inference graph. This yields a taxonomy of four scenarios when combinations of these half-links are observed or not. In a rigorous stratified analysis over these scenarios, we reveal that SoTA KGFMs use seen half links for predictions, while unseen half-links pose different challenges. As such, our finer-grained taxonomy can be a diagnostic protocol for robust KGFM generalization and highlights where novel KGFMs can improve.
Cosimo Gregucci, Obaidah Theeb, Daniel Hernandez +2
Institute for AI, University of Stuttgart · University of Edinburgh · University of Southampton