In this paper, we adopt a relational view of analogies applied to Semantic Role Classification in FrameNet. We define analogies as formal relations over the Cartesian product of frame evoking lexical units (LUs) and frame element (FEs) pairs, which we use to construct a new dataset. Each element of this binary relation is labelled as a valid analogical instance if the frame elements share the same semantic role, or as invalid otherwise. This formulation allows us to transform Semantic Role Classification into binary classification and train a lightweight Artificial Neural Network (ANN) that exhibits rapid convergence with minimal parameters. Unconventionally, no Semantic Role information is introduced to the neural network during training. We recover semantic roles during inference by computing probability distributions over candidates of all semantic roles within a given frame through random sampling and analogical transfer. This approach allows us to surpass previous state-of-the-art results while maintaining computational efficiency and frugality.
Figures & tables
Figure 1: An example of an annotated sentence for the frame type triggered by decline.v .
Dataset
N. sents
N. frames
N. docs
All
27,228
796
101
train
18,772
754
70
dev
2,192
368
8
test
6,264
563
23
Table 1: The number of sentences, frames, and documents in the datasets
Dataset
all
positive
negative
train
7,638,128
3,036,470
4,601,658
train_balanced
6,072,940
3,036,470
3,036,470
dev
154,802
72,290
82,512
test
984,724
397,880
586,844
Table 2: Number of analogical instances
Figure 2: Architecture of our pipeline
Figure 3: Training statistics wrt checkpoints
Label
0 (negative)
1(positive)
Precision
91.80
86.03
Recall
90.18
88.23
F1 score
90.98
87.12
Accuracy
89.39
Table 3: Scores of the binary analogy model (in %)
Figure 4: Semantic parsing workflows with two distinct semantic role classifiers
Frame id.
Arg id.
SR cl.
Accuracy
Lin et al.
Lin et al.
Lin et al.
48.95
Lin et al.
Lin et al.
Ours
49.81
Table 4: Overall performance of state-of-the-art end-to-end parser vs ours on FrameNet 1.7
Frame id.
Arg id.
SR cl.
Accuracy
gold
Swayamdipta et al.
Swayamdipta et al.
58.46
gold
Swayamdipta et al.
Ours
59.35
Table 5: Overall performance of former state-of-the-art SRL approach vs ours on FrameNet 1.7
Precision
Recall
F1 score
Accuracy
Ours
80.77
79.26
79.17
79.30
Baseline 1 ( et )
-
-
-
9.14
Baseline 2 ( pt,et )
-
-
-
20.64
Table 6: Baseline evaluation results on FrameNet 1.7 test set with gold frame and spans, using the exact same architecture as our approach. Precision, recall, and F1 scores are weighted.
Analogical reasoning studies how systems transfer relations and solutions between situations with shared structure, enabling inference beyond direct examples.