The Answer Lies Within: Self-Derived Rewards Enable Explainable Relation Extraction

Xinyu Guo; Zhengliang Shi; Minglai Yang; Mahdi Rahimi; Mihai Surdeanu

cs.CLOct 7, 2025

The Answer Lies Within: Self-Derived Rewards Enable Explainable Relation Extraction

Authors: Xinyu Guo, Zhengliang Shi, Minglai Yang, Mahdi Rahimi, Mihai Surdeanu

Organizations: University of Arizona, Tucson, AZ, United States · Shandong University, Qingdao, China.

Abstract

Despite the remarkable reasoning capabilities of large language models, they still struggle with one-shot relation extraction without predefined relation labels. We identify two pitfalls: models are often misled by irrelevant tokens instead of relation-conveying semantics, and they often fail to align with the abstraction level human annotators expect. We introduce a novel framework that closes this gap with two components: (1) COGRE, a cognitively-inspired reasoning framework that structures RE into a series of processes mimicking human text-processing; and (2) HIT@DICT, a reinforcement learning intermediate reward strategy that encourages reasoning to align with relational labels by rewarding relation-relevant phrases in reasoning. The reward is derived on a credit dictionary automatically extracted from correct predictions. Our experiments show that our framework improves both accuracy and explanation quality by addressing these two pitfalls. For example, COGRE with Qwen2.5-14B-Instruct on One-shot NYT29 achieves 24.65% F1, surpassing prior reasoning-based designs. Optimizing this approach with RL using HIT@DICT further improves performance by +23.46% points. Finally, human evaluation shows that our best model generates relational phrases closely aligned with gold labels, increasing human explanation quality ratings by 54% (relative).

Figures & tables

Figure 1: Framework overview. \hlmarktab Part 1 - Dictionary Construction : We apply a base model to positive samples, retain true positive (TP) instances, then use GPT-4o or Qwen2.5-14B-Instruct to extract relational phrases from their explanations to construct a relational phrase dictionary. \hlmarktab Part 2 - Reinforcement Learning with Hit@Dict : During RL training, the model’s stepwise CogRE outputs on all training samples are scored by accuracy (answers) and Hit@Dict (explanations). To compute Hit@Dict , we first look up relational phrases from the dictionary for both the support and test relations. These phrases are then matched against the LLM outputs to obtain hit statistics (Hit Times Table), and Hit@Dict is computed as a normalized hit rate (Section 3.2 ).

Panel of Figure 2 — Figure 2: Training dynamics on the one-shot NYT29 dataset for Phi-4 and Qwen2.5-14B-Instruct. The X-axes show training time(Hours). The Y-axes show reward, KL penalty, and response length. We compare reinforcement learning with accuracy reward \hlyellowtab Only Acc and with the combined Hit@Dict reward \hlbluetab Hit@Dict +Acc . We use fixed hyperparameters with $N=5$ , $w_{\text{ent}}=0.4$ , and $w_{\text{rel}}=1.0$ .

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Illustration of the realistic few-shot relation extraction setting. Pairwise support sentence and test sentence matching results are aggregated to produce a multi-class prediction.

Phi – TACRED
Setting / Category	Stages
Setting / Category	_untrained	_rl_ACC._reward	rl Hit@Dict _reward
no_yes	24	26	29
yes_no	16	18	26
yes_yes	27	27	29
no_no	22	22	27

Appendix

Table 6: Human evaluation results across four settings (Phi/Qwen × TACRED/NYT29). Each cell reports the score of a category in one combination under human evaluation. The categories are defined as follows: no_yes : the ground truth is No but the model prediction is Yes; yes_no : the ground truth is Yes but the model prediction is No; yes_yes : both the ground truth and the model prediction are Yes; no_no : both the ground truth and the model prediction is No;

Qwen2.5-14B	GPT4o
/location/location/contains	/location/location/contains
“city”	“city”
“location”	“location”
“province”	“provincial”
“capital”	“capital”
“located”	“located”

Appendix

Table 7: The phrases for the label /location/location/contains separately by Qwen2.5-14B and GPT4o. The yellow means the same phrases generated by both open-source and close-source models. We list all the phrases for all the labels separately by Qwen2.5-14B and GPT4o in Appendix A.13 .

Dictionary Source	P	R	F1
GPT-4o	63.34	38.78	48.11
Qwen2.5-14B	45.35	51.28	48.13

Appendix

Table 8: Performance of our method with dictionaries built from different LLMs. Comparable F1 scores indicate robustness to dictionary replacement.

	Original	Sampled
Positive items ( $r,r$ )	7170	2583
Negative items with no_relation ( $r,\text{{no\_relation}}$ )	732075	14834
Negative items without no_relation ( $r,r^{\prime}$ )	28680	2583

Appendix

Table 10: Ratio of positives and negatives on the original training partition of one-shot TACRED and our sampled version. Positive items (

r,r

): pairs where both sentences express the same relation

r

. Negative items with no_relation (

r,\text{no\_relation}

): pairs where one sentence expresses a relation

r

and the other is labeled as no_relation. Negative items without no_relation (

r,r^{\prime}

): pairs where the two sentences express different relations

r

and

r^{\prime}

, neither being no_relation.

Relation	Original	Sampled
/business/company/founders	52,201	1,634
/business/company/place_founded	51,408	1,571
/business/person/company	66,753	2,226
/film/film_location/featured_in_films	50,425	1,590
/location/country/capital	70,537	2,351
/location/location/contains	135,244	4,934

Appendix

Table 11: Distribution of relation labels in the one-shot NYT29 training partition (original vs. sampled). Each row corresponds to a relation type, where the Original column reports the number of instances in the full dataset and the Sampled column reports the number of instances included in our one-shot sampled version. The relation no_relation indicates sentence pairs that do not express any annotated relation.

Relation	Original	Sampled
org:alternate_names	31,532	1,156
org:city_of_headquarters	31,016	997
org:dissolved	30,011	912
org:members	30,038	968
org:number_of_employees/members	30,236	929
org:political/religious_affiliation	30,633	938

Appendix

Table 12: Distribution of relation labels in the one-shot TACRED training partition (original vs. sampled). Each row corresponds to a relation type, where the Original column reports the number of instances in the full dataset and the Sampled column reports the number of instances included in our one-shot sampled version. The relation no_relation indicates sentence pairs that do not express any annotated relation.

Category	Original	Sampled
Positive items ( $r,r$ )	7,055	704
Negative items with no_relation ( $r,\text{{no\_relation}}$ )	114,725	11,480
Negative items without no_relation ( $r,r^{\prime}$ )	28,220	2,816

Appendix

Table 13: Ratio of positives and negatives on the original testing partition of one-shot NYT29 and our sampled version. Positive items (

r,r

): pairs where both sentences express the same relation

r

. Negative items with no_relation (

r,\text{no\_relation}

): pairs where one sentence expresses a relation

r

and the other is labeled as no_relation. Negative items without no_relation (

r,r^{\prime}

): pairs where the two sentences express different relations

r

and

r^{\prime}

, neither being no_relation.

Category	Original	Sampled
Positive items ( $r,r$ )	772	82
Negative items with no_relation ( $r,\text{{no\_relation}}$ )	146,140	14,590
Negative items without no_relation ( $r,r^{\prime}$ )	3,088	328

Appendix

Table 14: Ratio of positives and negatives on the original testing partition of one-shot TACRED and our sampled version. Positive items (

r,r

): pairs where both sentences express the same relation

r

. Negative items with no_relation (

r,\text{no\_relation}

): pairs where one sentence expresses a relation

r

and the other is labeled as no_relation. Negative items without no_relation (

r,r^{\prime}

): pairs where the two sentences express different relations

r

and

r^{\prime}

, neither being no_relation.

Category	Original	Sampled
Positive items ( $r,r$ )	526	51
Negative items with no_relation ( $r,\text{{no\_relation}}$ )	147,370	14,745
Negative items without no_relation ( $r,r^{\prime}$ )	2,104	204

Appendix

Table 15: Ratio of positives and negatives on the original testing partition of one-shot WIKIDATA and our sampled version. Positive items (

r,r

): pairs where both sentences express the same relation

r

. Negative items with no_relation (

r,\text{no\_relation}

): pairs where one sentence expresses a relation

r

and the other is labeled as no_relation. Negative items without no_relation (

r,r^{\prime}

): pairs where the two sentences express different relations

r

and

r^{\prime}

, neither being no_relation.

Relation	Original	Sampled
/business/company/major_shareholders	30,510	3,055
/location/administrative_division/country	40,690	4,130
/location/country/administrative_divisions	41,160	4,040
/location/neighborhood/neighborhood_of	39,520	3,960
/people/deceased_person/place_of_death	33,395	3,335
no_relation	114,725	11,480

Appendix

Table 16: Distribution of relation labels in the one-shot NYT29 testing partition (original vs. sampled). Each row corresponds to a relation type, where the Original column reports the number of instances in the full dataset and the Sampled column reports the number of instances included in our one-shot sampled version. The relation no_relation indicates sentence pairs that do not express any relation.

Relation	Original	Sampled
no_relation	146,140	14,590
org:founded_by	15,442	1,611
org:member_of	15,246	1,557
org:top_members/employees	16,431	1,648
per:children	14,977	1,413
per:city_of_death	14,944	1,535

Appendix

Table 17: Distribution of relation labels in the one-shot TACRED testing partition (original vs. sampled). Each row corresponds to a relation type, where the Original column reports the number of instances in the full dataset and the Sampled column reports the number of instances included in our one-shot sampled version. The relation no_relation indicates sentence pairs that do not express any annotated relation.

Relation	Original	Sampled
Fach	1743	171
IUCN conservation status	1653	165
Solid solution series with	1758	141
affiliation	1656	165
airline alliance	1617	150
airline hub	1790	174

Appendix

Table 18: Distribution of relation labels in the one-shot WIKIDATA testing partition (original vs. sampled). Each row corresponds to a relation type, where the Original column reports the number of instances in the full dataset and the Sampled column reports the number of instances included in our one-shot sampled version. The relation no_relation indicates sentence pairs that do not express any annotated relation. The number of relation labels in WIKIDATA is extremely large, so we split it into three parts. This is the first part.

Relation	Original	Sampled
brother	1685	183
canonization status	1575	162
capital	1717	185
carries	1698	189
cast member	1802	170
composer	1708	156

Appendix

Table 19: Distribution of relation labels in the one-shot WIKIDATA testing partition (original vs. sampled), part 2 of 3.

Relation	Original	Sampled
item operated	1776	171
killed by	1821	201
license	1700	147
list of episodes	1626	162
located next to body of water	1752	179
lyrics by	1775	213

Appendix

Table 20: Distribution of relation labels in the one-shot WIKIDATA testing partition (original vs. sampled), part 3 of 3.

Technique	One-shot TACRED
	P	R	F1
Qwen2.5-7B-Instruct
Direct Matching	22.58	9.21	13.08
Random Reasoning	3.42	43.42	6.35
Cognitive-Structured RE ( our )	10.24	50.00	17.00
Qwen2.5-3B-Instruct

Appendix

Table 21: Performance comparison of different model families and sizes on the one-shot TACRED task. We evaluate our CogRE framework against prompting-based baselines (Direct Matching and Random Reasoning).

Qwen2.5-14B	GPT4o
/location/country/capital	/location/country/capital
“city”	“city”
“capital”	“capital”
“location”	“location”
“government”	“government”
“center”

Appendix

Table 22: The phrases for the label /location/country/capital extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/location/location/contains	/location/location/contains
“city”	“city”
“location”	“location”
“province”	“provincial”
“state”	“capital”
“located”	“located”

Appendix

Table 23: The phrases for the label /location/location/contains extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/location/us_state/capital	/location/us_state/capital
“city”	“city”
“located”	“located”
“capital”	“capital”
“state”
	“part”

Appendix

Table 24: The phrases for the label /location/us_state/capital extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/business/company/place_founded	/business/company/place_founded
“based”	“based”
“located”	“located”
“headquarters”	“headquarters”
“offices”	“offices”
“building”	“building”

Appendix

Table 25: The phrases for the label /business/company/place_founded extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/film/film_location/featured_in_films	/film/film_location/featured_in_films
“setting”	“setting”
“set”	“set”
“place”	“place”
“associated”	“associated”

Appendix

Table 26: The phrases for the label /film/film_location/featured_in_films extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/business/company/founders	/business/company/founders
“chairman”	“chairman”
“founder”	“founder”
“associated”	“associated”
“chief”	“chief”
“officer”	“officer”

Appendix

Table 27: The phrases for the label /business/company/founders extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/people/person/nationality	/people/person/nationality
“President”	“president”
“Minister”	“minister”
“Prime”	“prime”
“chancellor”	“chancellor”

Appendix

Table 28: The phrases for the label /people/person/nationality extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/business/person/company	/business/person/company
“Secretary”	“secretary”
“President”	“president”
“program”	“program”
“executive”	“executive”
“chief”	“chief”

Appendix

Table 29: The phrases for the label /business/person/company extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/location/us_county/county_seat	/location/us_county/county_seat
“municipality”	“municipality”
“included”	“included”
“area”	“area”
“around”	“around”
“in”	“in”

Appendix

Table 30: The phrases for the label /location/us_county/county_seat extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/people/person/place_lived	/people/person/place_lived
“lives”	“lives”
“hometown”	“hometown”
“from”	“from”
“resides”	“resides”
“governor”	“governor”

Appendix

Table 31: The phrases for the label /people/person/place_lived extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/people/person/children	/people/person/children
“mother”	“mother”
“son”	“son”
“father”	“father”
“daughter”	“daughter”
“children”	“child”

Appendix

Table 32: The phrases for the label /people/person/children extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/people/ethnicity/geographic_distribution	/people/ethnicity/geographic_distribution
“energy”	“energy”
“gas”	“gas”
“loan”	“loan”
“reporter”	“reporter”
“national”	“national”

Appendix

Table 33: The phrases for the label /people/ethnicity/geographic_distribution extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/people/person/ethnicity	/people/person/ethnicity
“American”	“american”
“Italian”	“italian”
“Manchu”	“manchu”
“Russian”	“russian”
“Tejano”	“tejano”

Appendix

Table 34: The phrases for the label /people/person/ethnicity extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/people/deceased_person/place_of_burial	/people/deceased_person/place_of_burial
“home”	“home”
	“sites”
“buried”	“buried”
	“grave”,
“interred”	“interred”

Appendix

Table 35: The phrases for the label /people/deceased_person/place_of_burial extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Qwen2.5-14B	GPT4o
/people/place_of_interment/interred_here	/people/place_of_interment/interred_here
“home”	“home”
“tomb”	“tomb”
“buried”	“buried”
“site”	“sites”
“staging”	“staging”

Appendix

Table 36: The phrases for the label /people/place_of_interment/interred_here extracted separately by Qwen2.5-14B and GPT-4o. The yellow indicates the same phrases generated by both open-source and closed-source models.

Relation Extraction

Relation extraction identifies and classifies relationships between entities or events in text and multimodal data.

See all 82 papers

ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models
Jun 25, 2026Xin Lin, Liang Zhang, Guoqi Ma +2Relation Extraction
National Institute for Data Science in Health and Medicine, Xiamen University, China · School of Informatics, Xiamen University
When Does Consistency Pay Off? Relational Constraints for LLM-based Document-Level Relation Extraction under Incomplete Annotations
Jul 27, 2026Mingxuan SunRelation Extraction Document Understanding
Department of Computer Science, Université de Sherbrooke, Sherbrooke, Canada
Improving Implicit Discourse Relation Recognition with Natural Language Explanations from LLMs
Feb 25, 2026Heng Wang, Changxing WuNatural Language Explanation Generation
School of Information and Software Engineering, East China Jiaotong University, Nanchang, China

Jun 25, 2026J/K move · Enter open · S save

ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models

Xin Lin, Liang Zhang, Guoqi Ma, Hongyao Tu+1

National Institute for Data Science in Health and Medicine, Xiamen University, China · School of Informatics, Xiamen University

Relation Extraction

Open paper AskPDF arXiv

Open Relation Extraction (OpenRE) requires a model to extract unseen relations between head and tail entities from unstructured text for real-world applications. The core challenge of OpenRE lies in achieving reliable generalization to unseen relation types. Current OpenRE approaches either employ clustering techniques, which cannot generate relation labels and suffer from poor generalization, or rely on direct relation label generation via Large Language Models (LLMs), which lack sufficient discriminative capacity to distinguish easily confused relations. To address these limitations, we propose Reasoning-guided progressive OpenRE (ReaORE), a framework for performing relation extraction through coarse-to-fine relation reasoning. Specifically, ReaORE consists of two key stages: (i) relation filtering, which reasons over multiple aspects to understand relations and instances, yielding an initial relation set, and further supplements and filters relations via embedding-based similarity to ensure the target relation is included; (ii) relation prediction, which aims to predict the target relations from the above set via fine-grained comparative reasoning to better distinguish easily confused relations. Extensive experiments on two widely used OpenRE datasets demonstrate that ReaORE outperforms existing baselines.

Abstract

Relation Extraction

Explore similar work

ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models

When Does Consistency Pay Off? Relational Constraints for LLM-based Document-Level Relation Extraction under Incomplete Annotations

Improving Implicit Discourse Relation Recognition with Natural Language Explanations from LLMs