Zero-shot relation extraction aims to identify relations between entity mentions using textual descriptions of novel types (i.e., previously unseen) instead of labeled training examples. Previous works often rely on unrealistic assumptions: (1) pairs of mentions are often encoded directly in the input, which prevents offline pre-computation for large scale document database querying; (2) no rejection mechanism is introduced, biasing the evaluation when using these models in a retrieval scenario where some (and often most) inputs are irrelevant and must be ignored. In this work, we study the robustness of existing zero-shot relation extraction models when adapting them to a realistic extraction scenario. To this end, we introduce a typology of existing models, and propose several strategies to build single pass models and models with a rejection mechanism. We adapt several state-of-the-art tools, and compare them in this challenging setting, showing that no existing work is really robust to realistic assumptions, but overall AlignRE (Li et al., 2024) performs best along all criteria.
Figures & tables
Figure 1: In relation extraction, the input is an utterance with two identified mentions, a head mention and a tail mention. We assume that the only targeted relation type is operating system . (left) Two input examples. In (1), the model must predict that there is a relation of this type between the two mentions. In (2), the model should reject the input, as the candidate input relation does not correspond to any type in the targeted ones. (right) Example of side-information used for zero-shot relation extraction.
Zs-Bert
LaVeEntail
Rcl
MatchPrompt
ReMatching
Cl&Cd
Emma
AlignRE
Ce-Da
GliRel
Chen and Li
Sainz et al.
Wang et al.
Wang et al.
Zhao et al.
Yang et al.
Li et al.
Li et al.
Zhang et al.
Boylan et al.
( 2021 )
( 2021 )
( 2022b )
( 2022a )
( 2023 )
( 2024 )
( 2024a )
( 2024b )
( 2025 )
( 2025 )
Neural architecture
SBert
✓
✓
✓
Single pass
✓
✓
Rej. mechanism
✓
Table 1: Typology of zero-shot relation extraction models, ordered chronologically. Row SBert indicates the use of SBert to encode side-information of relation types. The F1 row contains results as reported by original authors on FewRel using 5 unknown relation types, without rejection mechanism. (We do not include results for MatchPrompt and AlignRE as they have not been tested on this dataset) Note that there is also a late interaction variant of Emma , but it achieves significantly lower F1 scores.
Figure 2: Generic illustration of an encoder-only zero-shot RE model.
|T|
|D|
Dist. me.
Avg. len.
WikiZSL
113
94 383
77 623
24.85
FewRel
80
56 000
72 954
24.95
Table 2: Datasets statistics, where T is the set of relation types, D the set of relation quadruples including utterance, entities and relation type (s,e1,e2,t) , and the last two columns give the number of distinct entity mentions and the average length of utterances, respectively.
FewRel
WikiZSL
5
10
15
5
10
15
Emma
Off-the-shelf
98.4±1.2
84.5±4.5
79.5±6.2
88.2±7.0
67.9±8.8
62.8±6.6
Our efficient variants
↪ first
94.5±1.6
79.5±3.0
70.2±4.6
81.6±4.8
64.7±7.7
57.0±2.0
↪ projection
94.6±1.2
77.8±4.8
67.2±6.1
79.0±6.2
59.0±9.9
50.5±2.3
Table 3: Results in terms of macro F1, without rejection mechanism. We compare the off-the-shelf softwares with our efficient variants that allow on-the-fly zero-shot relation extraction with offline encoding.
FewRel
WikiZSL
5
10
15
5
10
15
Emma
Threshold
9.8±4.0
7.3±8.9
9.8±9.3
18.0±6.4
28.7±9.6
25.8±12.7
Description
42.8±16.6
42.5±28.5
24.9±15.3
61.4±14.2
49.8±16.0
47.3±9.5
Prototypes
100.0±0.0
99.8±0.3
100.0±0.0
37.1±33.4
36.0±54.5
67.5±26.4
Re-Matching
Table 4: Rejection accuracy in the rejection pass.
FewRel
WikiZSL
5
10
15
5
10
15
Emma
Threshold
52.4±1.9
31.8±6.5
29.0±11.0
54.6±6.0
47.1±8.4
47.0±8.1
Description
41.3±12.2
30.1±4.2
20.6±6.0
30.6±8.5
24.8±3.5
24.5±6.0
Prototypes
0.0±0.0
0.2±0.3
0.0±0.0
29.9±13.8
25.8±19.2
23.7±22.9
Re-Matching
Table 5: Macro F1 measure in the retention pass, computed for (non reject) relation types.
Despite the remarkable reasoning capabilities of large language models, they still struggle with one-shot relation extraction without predefined relation labels. We identify two pitfalls: models are often misled by irrelevant tokens instead of relation-conveying semantics, and they often fail to align with the abstraction level human annotators expect. We introduce a novel framework that closes this gap with two components: (1) COGRE, a cognitively-inspired reasoning framework that structures RE into a series of processes mimicking human text-processing; and (2) HIT@DICT, a reinforcement learning intermediate reward strategy that encourages reasoning to align with relational labels by rewarding relation-relevant phrases in reasoning. The reward is derived on a credit dictionary automatically extracted from correct predictions. Our experiments show that our framework improves both accuracy and explanation quality by addressing these two pitfalls. For example, COGRE with Qwen2.5-14B-Instruct on One-shot NYT29 achieves 24.65% F1, surpassing prior reasoning-based designs. Optimizing this approach with RL using HIT@DICT further improves performance by +23.46% points. Finally, human evaluation shows that our best model generates relational phrases closely aligned with gold labels, increasing human explanation quality ratings by 54% (relative).
Xinyu Guo, Zhengliang Shi, Minglai Yang +2
University of Arizona, Tucson, AZ, United States · Shandong University, Qingdao, China.
Joint named entity recognition (NER) and relation extraction (RE) is a fundamental task in natural language processing for constructing knowledge graphs from unstructured text. While recent approaches treat NER and RE as separate tasks requiring distinct models, we introduce GLiNER-Relex, a unified architecture that extends the GLiNER framework to perform both entity recognition and relation extraction in a single model. Our approach leverages a shared bidirectional transformer encoder to jointly represent text, entity type labels, and relation type labels, enabling zero-shot extraction of arbitrary entity and relation types specified at inference time. GLiNER-Relex constructs entity pair representations from recognized spans and scores them against relation type embeddings using a dedicated relation scoring module. We evaluate our model on four standard relation extraction benchmarks: CoNLL04, DocRED, FewRel, and CrossRE, and demonstrate competitive performance against both specialized relation extraction models and large language models, while maintaining the computational efficiency characteristic of the GLiNER family. The model is released as an open-source Python package with a simple inference API that allows users to specify arbitrary entity and relation type labels at inference time and obtain both entities and relation triplets in a single call. All models and code are publicly available.
Open Relation Extraction (OpenRE) requires a model to extract unseen relations between head and tail entities from unstructured text for real-world applications. The core challenge of OpenRE lies in achieving reliable generalization to unseen relation types. Current OpenRE approaches either employ clustering techniques, which cannot generate relation labels and suffer from poor generalization, or rely on direct relation label generation via Large Language Models (LLMs), which lack sufficient discriminative capacity to distinguish easily confused relations. To address these limitations, we propose Reasoning-guided progressive OpenRE (ReaORE), a framework for performing relation extraction through coarse-to-fine relation reasoning. Specifically, ReaORE consists of two key stages: (i) relation filtering, which reasons over multiple aspects to understand relations and instances, yielding an initial relation set, and further supplements and filters relations via embedding-based similarity to ensure the target relation is included; (ii) relation prediction, which aims to predict the target relations from the above set via fine-grained comparative reasoning to better distinguish easily confused relations. Extensive experiments on two widely used OpenRE datasets demonstrate that ReaORE outperforms existing baselines.
Xin Lin, Liang Zhang, Guoqi Ma +2
National Institute for Data Science in Health and Medicine, Xiamen University, China · School of Informatics, Xiamen University