A Multi-granularity Interaction Network for Relational Triple Extraction
Authors: Xiaocheng Luo, Yanping Chen, Zaifu Zhan
Organizations: School of Computer Science and Technology, Soochow University, Suzhou, 215006, Jiangsu, PR China · State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, Guizhou, PR China · Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, 55455, MN, USA
Relational triple extraction identifies entity relations from raw sentences without depending on entity type information. The performance of traditional methods is seriously degenerated by unseen'' entities, because these methods can't build contextual dependencies relevant unseen'' entities. This paper addresses this challenge by proposing a multi-granularity interaction network that leverages word-word, entity-relation, and head-tail interactions to capture deep semantic dependencies. By incorporating multiple iterations, the multi-granularity interaction network is effective to refine the representations corresponding to head and tail entities. It enhances the global semantic features associated with relational triples, aiding in the remediation of triples that may have been overlooked or omitted in earlier steps. Strong and consistent performance is achieved by our method on two rearranged datasets, with a significant improvement in identifying relational triples, including those involving ``unseen'' named entities. For example, our model improves F1-score by 3.1%/20.5% over previous approaches on the NYT-R/WebNLG-R datasets in the Unseen metric. Code: https://github.com/xiaocen-luo/Interaction-Network.
Figures & tables
Dataset
Entire
Partial
Unseen
WebNLG
756 ( {\color[rgb]{1,0,0}\mathbf{85.23}\%} )
110 (12.40 % )
21 (2.37 % )
NYT
2672 ( {\color[rgb]{1,0,0}\mathbf{77.07\%}} )
663 (19.12 % )
132 (3.81 % )
WebNLG - R
439 (44.17 % )
338 (34.00 % )
217 (21.83 % )
NYT - R
823(13.81 % )
3985 (66.89 % )
1150 (19.30 % )
Table 1: The number (ratio) of triple types in the testing datasets.
Table 2: The F1 performance achieved by the current best-performing model across four datasets ( Lee et al., 2022 ) .
Figure 1: An example contains two types of relational triples.
Figure 2: Model Architecture.
Figure 3: The details of Entity-Relation Interaction.
Figure 4: An Example of entity-pair tagging.
Category
Details of the dataset
Type of triple
Train
Valid
Test
Relation
Entire
Partial
Unseen
Others
NYT - R
56196
5000
5000
24
348
3307
886
459
WebNLG - R
5019
500
703
216
155
178
174
196
NYT
56195
5000
5000
24
4292
473
88
147
WebNLG
5019
500
703
171
580
42
17
64
Table 3: Statistics of the datasets.
Dataset
Overlapping Pattern
Normal
SEO
EPO
SOO
N=1
N=2
N=3
N=4
N≥5
NYT - R
3965
709
408
110
3958
733
192
50
48
WebNLG - R
389
293
9
84
411
143
81
58
10
NYT
3266
1297
978
45
3,244
1,045
312
291
108
WebNLG
245
457
26
84
266
171
131
90
45
Table 4: Overlapping patterns in datasets. N denotes the total count of relational triples.
Model
NYT - R
WebNLG - R
Prec.
Rec.
F1
Entire
Partial
Unseen
Prec.
Rec.
F1
Entire
Partial
Unseen
CasRel† ( Wei et al., 2020 )
65.9
60.1
62.9
85.8
65.0
42.3
73.6
64.2
68.6
89.6
52.3
41.5
TPlinker† ( Wang et al., 2020 )
69.0
60.8
64.7
83.3
66.7
46.8
75.1
63.9
69.1
88.5
52.7
42.9
PRGC† ( Zheng et al., 2021 )
63.5
61.6
62.6
81.6
64.2
45.1
61.6
62.0
61.8
79.2
47.2
28.3
BiRTE\lx@sectionsign ( Ren et al., 2022 )
64.5
65.4
64.9
86.1
67.0
45.2
71.0
64.6
67.6
89.2
53.9
36.7
OneRel\lx@sectionsign ( Shang et al., 2022 )
70.8
58.2
63.9
82.5
66.5
44.2
75.9
66.5
70.9
90.8
57.2
39.2
Table 5: Main results.
Model
NYT - R
WebNLG - R
Prec.
Rec.
F1
Entire
Partial
Unseen
Prec.
Rec.
F1
Entire
Partial
Unseen
Fully Model
70.2
67.1
68.6
89.1
70.8
52.6
78.1
67.8
72.6
90.8
57.2
58.7
w/oWWI
70.3
65.0
67.6
88.4
69.5
50.7
77.1
63.5
69.7
87.0
55.9
50.0
w/oERI
69.4
65.0
67.1
89.3
69.2
51.7
76.8
67.0
71.6
90.0
54.7
55.8
w/oHTI
70.6
64.0
67.2
87.9
69.4
50.4
75.8
67.8
71.6
90.5
55.9
56.4
w/oALL
74.2
57.1
64.6
81.2
66.8
44.8
68.1
52.8
59.5
75.7
43.5
43.5
Table 6: Ablation study.
Dataset
Model
Normal
EPO
SEO
SOO
N=1
N=2
N=3
N=4
N≥5
NYT
CasRel
87.3
92.0
91.4
77.0
88.2
90.3
91.9
94.2
83.7
TPLinker
90.1
94.0
93.4
90.1
90.0
92.8
93.1
96.1
90.0
SPN
90.8
94.1
94.0
81.8
90.9
93.4
94.2
95.5
90.6
PRGC
91.0
94.5
94.0
90.8
91.1
93.0
93.5
95.5
93.0
DirectRel
91.7
94.8
90.0
90.0
91.7
94.1
93.5
96.3
92.7
BiRTE
91.4
94.2
94.7
-
91.5
93.7
93.9
95.8
92.1
Table 7: F1-score (%) on sentences with different overlapping patterns and different numbers of triples for the NYT and WebNLG datasets. N represents the number of triples within a sentence.
Figure 5: The performance of tensor-based operations with different vector spaces {3, 4, 5, 6, 8, 10, 12}.
Model
NYT - R
WebNLG - R
Prec.
Rec.
F1
Prec.
Rec.
F1
CCAOur(×1)
70.2
67.1
68.6
78.1
67.8
72.6
CCAOur(×2)
69.7
61.3
65.2
79.7
66.0
72.2
CCAOri(×1)
73.2
58.4
65.0
77.7
65.3
71.0
CCAOri(×2)
75.1
57.7
65.3
78.9
65.8
71.8
Table 8: The performance of CCA with different configurations on NYT - R and WebNLG - R .
Model
NYT - R
WebNLG - R
Prec.
Rec.
F1
Prec.
Rec.
F1
[MLP,CA]
71.5
59.7
65.1
78.5
64.0
70.6
[MHA,MLP]
73.8
57.4
64.6
79.5
66.4
72.4
[MLP,MLP]
73.9
57.0
64.3
78.1
67.8
72.6
[MHA,CA]
70.2
67.1
68.6
79.7
66.0
72.2
Table 9: The performance of Self Interaction and Multi-Object Interaction with different configurations on NYT - R and WebNLG - R .
Model
NYT - R
WebNLG - R
Prec.
Rec.
F1
Prec.
Rec.
F1
I=1
70.2
67.1
68.6
78.1
67.8
72.6
I=2
69.2
61.9
65.4
70.5
37.3
48.8
I=3
64.6
64.1
64.4
69.4
36.6
47.9
Table 10: The performance of interaction network with different number of iterations of on NYT - R and WebNLG - R .
Model
NYT
WebNLG
Prec.
Rec.
F1
Prec.
Rec.
F1
BiRTE
92.2
93.8
93.0
93.2
94.0
93.6
OneRel
92.8
92.9
92.8
94.1
94.4
94.3
BitCoin
92.9
92.8
93.1
94.4
94.5
94.4
SOIRP
93.3
92.9
93.1
94.3
94.1
94.2
RTF
93.2
93.4
93.3
94.0
95.3
94.6
Table 11: Performance on NYT and WebNLG .
Model
NYT - R
Training Time
GPU Memory
F1
SOIRP
6738 seconds
4278 MB
67.5
Ours
7113 seconds
5180 MB
68.6
Oursw/oWWI
6474 seconds
4612 MB
67.6
Oursw/oERI
5724 seconds
4338 MB
67.1
Oursw/oHTI
6418 seconds
5042 MB
67.2
Table 12: Comparison of model efficiency.
Figure 6: Examples of sentences of the Entire, Partial, and Unseen three triple types from OneRel and our models and the corresponding predictions. The orange mark indicates an error in the extracted relational triples. The blue mark indicates that the extracted relational triple is missing.
Figure 7: Visualization of different model components for two sentences.