The Answer Lies Within: Self-Derived Rewards Enable Explainable Relation Extraction
Organizations: University of Arizona, Tucson, AZ, United States · Shandong University, Qingdao, China.
Abstract
Despite the remarkable reasoning capabilities of large language models, they still struggle with one-shot relation extraction without predefined relation labels. We identify two pitfalls: models are often misled by irrelevant tokens instead of relation-conveying semantics, and they often fail to align with the abstraction level human annotators expect. We introduce a novel framework that closes this gap with two components: (1) COGRE, a cognitively-inspired reasoning framework that structures RE into a series of processes mimicking human text-processing; and (2) HIT@DICT, a reinforcement learning intermediate reward strategy that encourages reasoning to align with relational labels by rewarding relation-relevant phrases in reasoning. The reward is derived on a credit dictionary automatically extracted from correct predictions. Our experiments show that our framework improves both accuracy and explanation quality by addressing these two pitfalls. For example, COGRE with Qwen2.5-14B-Instruct on One-shot NYT29 achieves 24.65% F1, surpassing prior reasoning-based designs. Optimizing this approach with RL using HIT@DICT further improves performance by +23.46% points. Finally, human evaluation shows that our best model generates relational phrases closely aligned with gold labels, increasing human explanation quality ratings by 54% (relative).
Figures & tables
Appendix figures & tables31 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting / Category | Stages | ||
|---|---|---|---|
| _untrained | _rl_ACC._reward | rl Hit@Dict _reward | |
| Phi – TACRED | |||
| no_yes | 24 | 26 | 29 |
| yes_no | 16 | 18 | 26 |
| yes_yes | 27 | 27 | 29 |
| no_no | 22 | 22 | 27 |
| Qwen2.5-14B | GPT4o |
|---|---|
| /location/location/contains | /location/location/contains |
| “city” | “city” |
| “location” | “location” |
| “province” | “provincial” |
| “capital” | “capital” |
| “located” | “located” |
| Dictionary Source | P | R | F1 |
|---|---|---|---|
| GPT-4o | 63.34 | 38.78 | 48.11 |
| Qwen2.5-14B | 45.35 | 51.28 | 48.13 |
| Original | Sampled | |
|---|---|---|
| Positive items ( ) | 7170 | 2583 |
| Negative items with no_relation ( ) | 732075 | 14834 |
| Negative items without no_relation ( ) | 28680 | 2583 |
| Relation | Original | Sampled |
|---|---|---|
| /business/company/founders | 52,201 | 1,634 |
| /business/company/place_founded | 51,408 | 1,571 |
| /business/person/company | 66,753 | 2,226 |
| /film/film_location/featured_in_films | 50,425 | 1,590 |
| /location/country/capital | 70,537 | 2,351 |
| /location/location/contains | 135,244 | 4,934 |
| Relation | Original | Sampled |
|---|---|---|
| org:alternate_names | 31,532 | 1,156 |
| org:city_of_headquarters | 31,016 | 997 |
| org:dissolved | 30,011 | 912 |
| org:members | 30,038 | 968 |
| org:number_of_employees/members | 30,236 | 929 |
| org:political/religious_affiliation | 30,633 | 938 |
| Category | Original | Sampled |
|---|---|---|
| Positive items ( ) | 7,055 | 704 |
| Negative items with no_relation ( ) | 114,725 | 11,480 |
| Negative items without no_relation ( ) | 28,220 | 2,816 |
| Category | Original | Sampled |
|---|---|---|
| Positive items ( ) | 772 | 82 |
| Negative items with no_relation ( ) | 146,140 | 14,590 |
| Negative items without no_relation ( ) | 3,088 | 328 |
| Category | Original | Sampled |
|---|---|---|
| Positive items ( ) | 526 | 51 |
| Negative items with no_relation ( ) | 147,370 | 14,745 |
| Negative items without no_relation ( ) | 2,104 | 204 |
| Relation | Original | Sampled |
|---|---|---|
| /business/company/major_shareholders | 30,510 | 3,055 |
| /location/administrative_division/country | 40,690 | 4,130 |
| /location/country/administrative_divisions | 41,160 | 4,040 |
| /location/neighborhood/neighborhood_of | 39,520 | 3,960 |
| /people/deceased_person/place_of_death | 33,395 | 3,335 |
| no_relation | 114,725 | 11,480 |
| Relation | Original | Sampled |
|---|---|---|
| no_relation | 146,140 | 14,590 |
| org:founded_by | 15,442 | 1,611 |
| org:member_of | 15,246 | 1,557 |
| org:top_members/employees | 16,431 | 1,648 |
| per:children | 14,977 | 1,413 |
| per:city_of_death | 14,944 | 1,535 |
| Relation | Original | Sampled |
|---|---|---|
| Fach | 1743 | 171 |
| IUCN conservation status | 1653 | 165 |
| Solid solution series with | 1758 | 141 |
| affiliation | 1656 | 165 |
| airline alliance | 1617 | 150 |
| airline hub | 1790 | 174 |
| Relation | Original | Sampled |
|---|---|---|
| brother | 1685 | 183 |
| canonization status | 1575 | 162 |
| capital | 1717 | 185 |
| carries | 1698 | 189 |
| cast member | 1802 | 170 |
| composer | 1708 | 156 |
| Relation | Original | Sampled |
|---|---|---|
| item operated | 1776 | 171 |
| killed by | 1821 | 201 |
| license | 1700 | 147 |
| list of episodes | 1626 | 162 |
| located next to body of water | 1752 | 179 |
| lyrics by | 1775 | 213 |
| Technique | One-shot TACRED | |||
| P | R | F1 | ||
| Qwen2.5-7B-Instruct | ||||
| Direct Matching | 22.58 | 9.21 | 13.08 | |
| Random Reasoning | 3.42 | 43.42 | 6.35 | |
| Cognitive-Structured RE ( our ) | 10.24 | 50.00 | 17.00 | |
| Qwen2.5-3B-Instruct | ||||
| Qwen2.5-14B | GPT4o |
|---|---|
| /location/country/capital | /location/country/capital |
| “city” | “city” |
| “capital” | “capital” |
| “location” | “location” |
| “government” | “government” |
| “center” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /location/location/contains | /location/location/contains |
| “city” | “city” |
| “location” | “location” |
| “province” | “provincial” |
| “state” | “capital” |
| “located” | “located” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /location/us_state/capital | /location/us_state/capital |
| “city” | “city” |
| “located” | “located” |
| “capital” | “capital” |
| “state” | |
| “part” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /business/company/place_founded | /business/company/place_founded |
| “based” | “based” |
| “located” | “located” |
| “headquarters” | “headquarters” |
| “offices” | “offices” |
| “building” | “building” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /film/film_location/featured_in_films | /film/film_location/featured_in_films |
| “setting” | “setting” |
| “set” | “set” |
| “place” | “place” |
| “associated” | “associated” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /business/company/founders | /business/company/founders |
| “chairman” | “chairman” |
| “founder” | “founder” |
| “associated” | “associated” |
| “chief” | “chief” |
| “officer” | “officer” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /people/person/nationality | /people/person/nationality |
| “President” | “president” |
| “Minister” | “minister” |
| “Prime” | “prime” |
| “chancellor” | “chancellor” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /business/person/company | /business/person/company |
| “Secretary” | “secretary” |
| “President” | “president” |
| “program” | “program” |
| “executive” | “executive” |
| “chief” | “chief” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /location/us_county/county_seat | /location/us_county/county_seat |
| “municipality” | “municipality” |
| “included” | “included” |
| “area” | “area” |
| “around” | “around” |
| “in” | “in” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /people/person/place_lived | /people/person/place_lived |
| “lives” | “lives” |
| “hometown” | “hometown” |
| “from” | “from” |
| “resides” | “resides” |
| “governor” | “governor” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /people/person/children | /people/person/children |
| “mother” | “mother” |
| “son” | “son” |
| “father” | “father” |
| “daughter” | “daughter” |
| “children” | “child” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /people/ethnicity/geographic_distribution | /people/ethnicity/geographic_distribution |
| “energy” | “energy” |
| “gas” | “gas” |
| “loan” | “loan” |
| “reporter” | “reporter” |
| “national” | “national” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /people/person/ethnicity | /people/person/ethnicity |
| “American” | “american” |
| “Italian” | “italian” |
| “Manchu” | “manchu” |
| “Russian” | “russian” |
| “Tejano” | “tejano” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /people/deceased_person/place_of_burial | /people/deceased_person/place_of_burial |
| “home” | “home” |
| “sites” | |
| “buried” | “buried” |
| “grave”, | |
| “interred” | “interred” |
| Qwen2.5-14B | GPT4o |
|---|---|
| /people/place_of_interment/interred_here | /people/place_of_interment/interred_here |
| “home” | “home” |
| “tomb” | “tomb” |
| “buried” | “buried” |
| “site” | “sites” |
| “staging” | “staging” |
Relation Extraction
Relation extraction identifies and classifies relationships between entities or events in text and multimodal data.
See all 82 papers