Entity Disambiguation (ED) is a key task for constructing and using knowledge graphs. State-of-the-art neural approaches commonly model ED as a single task, although it consists of two distinct subproblems: retrieving candidate entities and selecting the correct one given context. Dual-encoder models optimize for both within a shared embedding space, forcing representations to balance high-recall retrieval with fine-grained selection, and they require trained retrievers, which are costly to maintain as knowledge graphs change. While recent work has begun to combine retrievers with LLM-based selectors, the interplay between the two stages has not been studied systematically. In this paper, we present a systematic comparison of retrieval strategies for candidate generation under a shared LLM-based selection stage, combining sparse retrieval (BM25), Web KB search, and a state-of-the-art trained dense retriever with several open- and closed-source LLMs. We show that, once selection is delegated to a capable LLM, training the retriever provides only modest additional value: a fully training-free BM25 retriever paired with an LLM selector reaches a new state of the art on the ZELDA benchmark, raising inKB micro-F1 from 82.3 to 86.3 (+4); pairing the same LLM with a trained dense retriever reaches 88.5. Decoupling retrieval from selection also exposes a limitation of current ED systems: when the correct entity is missing from retrieved candidates, they are forced to predict an incorrect entity. In contrast, our framework allows for abstention when retrieval failure is detected. In an evaluation setting that rewards correct abstentions, the training-free BM25 + LLM pipeline reaches 90.7 F1.
Figures & tables
Figure 1: Overview of the RAISED framework. Given a mention, its surrounding context, and a large candidate pool from a knowledge base, RAISED operates in two stages. The Retriever ranks entities and returns the top- 16 candidates; it can be instantiated with sparse, dense, or Web KB search backends. The Selector then prompts an LLM with the mention, surrounding context, and candidate descriptions in a multiple-choice format, and identifies the correct entity or abstains when no candidate is appropriate.
Aida-
Tweeki
Reddit-
Reddit-
Wned-
Wned-
Slinks-
Slinks-
Slinks-
AVG
Method
B
Posts
Comm.
Cweb
Wiki
Tail
Shadow
Top
Dual Encoder
VERBALIZED
88.2
78.9
92.2
88.4
71.5
90.8
98.2
66.3
65.9
82.3
RAISED — Wikipedia API candidates
Zero-shot inference
GPT-4o-mini
85.9
85.2
93.2
93.5
73.7
72.9
98.2
55.7
77.6
81.8
Table 1: Comparison of retrieval selection settings (RAISED) and the strongest dual-encoder baseline (VERBALIZED) on the ZELDA benchmark. The VERBALIZED baseline score is taken from [ 26 ] . Values indicate inKB micro-F1 score. RAISED variants are evaluated under three candidate-selection settings (Wikipedia API, BM25, and VERBALIZED). The best score in each column is shown in bold , and the second-best is underlined .
WIKIPEDIA API
BM25
VERBALIZED
DATASET
R@1
R@8
R@16
R@1
R@8
R@16
R@1
R@8
R@16
Datasets in the ZELDA Benchmark
SLINKS SHADOW
6.4
69.5
82.5
12.3
73.0
84.0
66.3
93.6
95.4
SLINKS TOP
35.2
89.7
93.6
47.6
88.8
92.6
66.3
87.4
89.6
SLINKS TAIL
96.3
99.5
99.5
94.3
98.0
98.0
98.4
99.2
99.2
TWEEKI
82.0
93.6
95.0
74.1
87.2
89.0
78.3
91.7
92.6
Table 2: Retriever recall@ k on the ZELDA test sets for the Wikipedia API, BM25 over the ZELDA candidate dictionary, and the dual-encoder of VERBALIZED (run with the authors’ released code).
NOC-aware
NOC %
NOC metrics
Method
F1
Gold
Pred
Precision
Recall
F1
RAISED — Wikipedia API candidates
GPT-4o-mini
87.5
12.1
4.9
75.9
36.5
47.3
GPT-5.4-mini
87.9
12.1
3.8
70.9
29.3
38.3
Qwen3-32B
85.9
12.1
7.3
60.7
38.9
43.5
Mistral-Small-24B
85.0
12.1
7.1
63.5
22.0
28.7
Table 3: Abstention analysis: performance when the model outputs “None of the Candidates” (NOC). NOC-aware F1 is the average inKB micro-F1 across the nine ZELDA datasets when correct NOC predictions are rewarded. Gold is the share of instances where the gold entity is genuinely absent from the candidate set; Pred is how often the model abstains. Precision, Recall, and F1 in the rightmost block are simple means across the nine datasets, computed over gold-NOC instances only.
Knowledge graph construction turns text, tables, images, and databases into structured entities and relationships that machines can query and reason over.
Knowledge graphs (KGs) are increasingly used as structured context for Large Language Models (LLMs), but industrial KG-RAG systems often need to integrate public and domain-specific KGs constructed from heterogeneous databases. This integration relies on Entity Alignment (EA), where lexical matching alone is insufficient under predicate-name variation and incomplete local neighborhoods. We address EA for KG integration by constructing a pairwise EA dataset and proposing two complementary modules: Predicate Importance Estimation (PIE) and Decoupled Rationale-Score Distillation (DRSD). PIE is a compact embedding-based approach that removes the subject information from each 1-hop triple, encodes the resulting subjectless triples, and aggregates them with learnable predicate-importance weights to build predicate-aware entity embeddings. DRSD trains a distilled small language model (SLM) with pseudo-answers produced by a teacher LLM through distinct prompts. By converting binary EA labels into text-based supervision and decoupling confidence-score estimation from label-consistent rationales, DRSD enables the SLM to learn task-specific reasoning while retaining a less label-biased confidence signal. Experiments show that PIE and DRSD improve EA classification. Moreover, because DRSD decouples confidence-score estimation from the decision, a discrepancy between the two flags an uncertain prediction for human review, thereby enabling a practical discrepancy between automatic acceptance and human-in-the-loop verification.
Large language models are increasingly used for knowledge graph question answering (KGQA), but can fail to correctly ground answers in the underlying graph. Current approaches to LLM-based KGQA either rely on full semantic parsing into executable queries such as SPARQL, which is brittle in practice due to complex schemas or incompleteness of real-world KGs, or on LLM-reasoning and answer generation over KGs, which can be more robust but lacks formal guarantees. In this work, we study a complementary setting in which candidate answers are generated by an LLM-based system and subsequently verified using lightweight symbolic constraints derived from the question. We introduce Constrained Entity Selection under Partial Knowledge (CES-PK), a problem formulation that focuses on eliminating invalid answers and providing symbolic support for valid ones without requiring construction of executable logical forms. To account for incomplete KGs, we employ a three-valued constraint semantics (satisfied, violated, unknown) that avoids incorrect rejections under open-world assumptions. To demonstrate the effects of our method, we instantiate this framework over the Hetionet biomedical knowledge graph and evaluate the impact of type, relation, and exclusion constraints. Experiments show that precision improves by filtering invalid candidates, while recall is preserved due to retaining candidates whose constraints are not explicitly violated. Satisfied constraints provide additional positive symbolic evidence to rank remaining candidates.
Emanuel Kitzelmann
Brandenburg University of Applied Sciences, Brandenburg a.d.H., Germany
Biomedical Entity Linking disambiguates mentions to entities in a knowledge base (KB), making it the cornerstone of information extraction pipelines. While embedding-based models are a popular approach for the task, they suffer from a key limitation. They compress mentions (and entities) into a single vector, forcing the model to average away crucial fine-grained differences. We present BELXTR, a novel embedding model based on the multi-vector (a.k.a. late interaction) architecture, which allows to leverage token-level matching information. BELXTR extends the original XTR model to biomedical entity linking by integrating an existing task-specific training objective and exploring active query expansion. Experiments across ten corpora and five KBs show that BELXTR improves upon current state-of-the-art in half of the corpora with an average improvement of 5pp recall@1. The largest gains are reported on the challenging cross-species gene disambiguation subtask, where BELXTR outperforms an LLM-powered retrieve-and-rerank pipeline and closely approaches a specialized rule-based system. Our results highlight multi-vector models as a practical alternative to hard-to-maintain rule-based systems or in scenarios where LLM-based reranking is too costly as in PubMed-scale mining. The code to reproduce our experiments can be found at: https://github.com/sg-wbi/belxtr.