Argumentation is an important cognitive activity that involves the scrutiny of arguments and counterarguments. When given an argument, it is desirable that its premises entail its claim, and that the premises and claim are consistent, and when given a pair of arguments, it is desirable to check whether one argument supports or attacks another. If the arguments are in natural language, then we could use methods for natural language inference, also known as recognizing textual entailment (RTE). The state-of-the-art RTE methods use machine learning and deep learning, but these are not explainable. An alternative approach which we investigate in this paper is to translate text into an abstract meaning representation (AMR) graph. For this we use a pre-trained AMR parser. We then translate the AMR graph into propositional logic and use a SAT solver for automated reasoning. In text, often common sense suggests that an entailment (or contradiction) relationship holds between a premise and a claim, but because different wordings are used, this is not identified from their logical representations. To address this, we introduce relaxation methods to allow replacement or forgetting of some propositions. Our experimental results show our approach performs well on three RTE datasets. The benefit of our approach is that we have a logical representation of premises and claims, and of the relaxations that are used to identify the entailment and contradiction relations.
Figure 1: AMR graphs for the sentences “The boy wants to go.” (left) and “The boy does not want to go.” (right).
Figure 2: Our sentence-to-label pipeline. The text input is premises and claim, and the output is a label.
Predicted class
ϕ′∧¬ψ′
ϕ∗∧ψ′
Entailment
False
True
Contradiction
True
False
Neutral
True
True
Table 1: Classification of relations. Let ϕ (resp. ψ ) be a premise (resp. claim), let ϕ′ (resp. ψ′ ) be the relaxed premise (resp. relaxed claim), and let ϕ∗ be FgtΓ(ϕ′) where Γ is the set of forgettable atoms of ϕ′ and ψ′ . For columns 2 and 3, PySAT returns true if the formula is consistent, and returns False if it is inconsistent. So for instance the entailment relation holds when ϕ′∧¬ψ′ is False and ϕ∗∧ψ′ is True.
Table 2: Results from the e-SNLI dataset
Figure 3: Results concerning the neuro-matching relaxation threshold. The y-axis represents the scores of precision, recall, F1-score, and accuracy, and the x-axis represents the threshold τ .
Class
Precision
Recall
F1
Support
Entailment
0.659
0.942
0.776
499
Contradiction
0.906
0.810
0.855
500
Neutral
0.826
0.560
0.667
500
Table 3: SICK dataset with forgetting (accuracy 0.771)
Class
Precision
Recall
F1
Support
Entailment
0.610
0.578
0.593
476
Contradiction
0.552
0.374
0.446
494
Neutral
0.459
0.629
0.531
496
Table 4: MultiNLI dataset with forgetting (accuracy 0.527)