Real-world arguments in text and dialogues are normally enthymemes (i.e. some of their premises and/or claims are implicit). Natural language processing (NLP) methods for handling enthymemes can potentially identify enthymemes in text but they do not decode their underlying logic, whereas logic-based approaches for handling them assume a knowledgebase with sufficient formulae that can be used to decode them via abduction. There is therefore a lack of a systematic method for translating textual components of an enthymeme into a logical argument and generating the logical formulae required for their decoding, and thereby showing logical entailment. To address this, we propose a pipeline that integrates: (1) a large language model (LLM) to generate intermediate implicit premises based on the explicit premise and claim; (2) another LLM to translate the natural language into logical formulas; and (3) a neuro-symbolic reasoner based on a SAT solver to determine entailment. We evaluate our pipeline on two enthymeme datasets, demonstrating promising performance in selecting the correct implicit premise, as measured by precision, recall, F1-score, and accuracy.
Figure 1: AMR for the sentence “The boy wants to go.” (left) and “The boy does not want to go.” (right).
Figure 2: Our neuro-symbolic pipeline where input is a set of natural language sentences (e.g. premise, implicit premises and claim), and the output is a label.
ARCT
ANLI
Premise
They add a lot to the piece and I look forward to reading comments.
Premise
Jane was a professor teaching piano to students.
Claim
Comment sections have not failed.
Claim
Jane spent the morning sipping coffee and reading a book.
Helpful
Comments sections are a welcome distraction from my work.
Helpful
None of Jane’s students had a lesson that day.
Unhelp.
Comments sections always distract me from my work.
Unhelp.
Two of Jane’s students were early for their lessons.
Table 1: Examples from the ARCT and ANLI datasets
ANLI
ARCT
Class
Step type
F1
τm
τc
Class
Step type
F1
τm
τc
0
original
0.67
0.8
80
0
original
0.67
0.5
90
1-step
0.72
0.6
80
1-step
0.72
0.7
90
2-step
0.74
0.5
80
2-step
0.70
0.75
80
3-step
0.75
0.6
80
3-step
0.72
0.7
90
1
original
0.46
0.55
100
1
original
0.21
0.55
80
Table 2: Best F1-scores and thresholds by dataset, class (0 non-entailment, 1 entailment), and step type.
Dataset
none
original
1-step
2-step
3-step
ANLI
0.496
0.520
0.624
0.660
0.716
ARCT
0.276
0.292
0.480
0.496
0.560
Table 3: Entailment accuracy by dataset and step type. None means no implicit premise; original means the dataset premise; 1-, 2-, and 3-step mean LLM-generated intermediate premises.
Figure 4: Overall accuracy for ANLI and ARCT by step type and thresholds. Original uses dataset premises; 1-, 2-, and 3-step use LLM-generated premises.
Figure 5: A structured argument graph for decoding an enthymeme. The claim that a large insect escaped recently is supported by Implicit Premise A (torn webs result from trapped prey escaping), contradicted by Implicit Premise B (a small insect fled), and neutral with respect to Implicit Premise C (wind tears a spiderweb) and the premise alone. Arc labels show the neuro-matching relations used for support and the neuro-contradict relation used for contradiction. Thus, adding the AMR formulae for Implicit Premise A to those for the premise, together with the neuro-matching relations, entails the claim. Black arrows denote combining the explicit premise with an implicit premise.
The logical reconstruction of argument graphs from natural language text is challenging because of the prevalence of enthymemes (i.e., arguments with implicit premises). There are natural language processing methods for identifying enthymemes in text, and there are symbolic methods based on abduction for identifying missing premises in a logical representation of enthymemes. However, there is a need for methods to generate implicit premises to logically show a known entailment or contradiction relationship between a pair of statements. To address this, we propose a neuro-symbolic pipeline that uses large language models (LLMs) to generate intermediate implicit premises that are translated into logical formulae and used with logical formulae representing explicit premises and explicit claims to show the logical relationships between them (entailment, contradiction, or neutrality). Our approach is evaluated on the Microtext Argumentative Corpus.
Xuyao Feng, Anthony Hunter
Department of Computer Science, University College London, United Kingdom
Argumentation is an important cognitive activity that involves the scrutiny of arguments and counterarguments. When given an argument, it is desirable that its premises entail its claim, and that the premises and claim are consistent, and when given a pair of arguments, it is desirable to check whether one argument supports or attacks another. If the arguments are in natural language, then we could use methods for natural language inference, also known as recognizing textual entailment (RTE). The state-of-the-art RTE methods use machine learning and deep learning, but these are not explainable. An alternative approach which we investigate in this paper is to translate text into an abstract meaning representation (AMR) graph. For this we use a pre-trained AMR parser. We then translate the AMR graph into propositional logic and use a SAT solver for automated reasoning. In text, often common sense suggests that an entailment (or contradiction) relationship holds between a premise and a claim, but because different wordings are used, this is not identified from their logical representations. To address this, we introduce relaxation methods to allow replacement or forgetting of some propositions. Our experimental results show our approach performs well on three RTE datasets. The benefit of our approach is that we have a logical representation of premises and claims, and of the relaxations that are used to identify the entailment and contradiction relations.
Xuyao Feng, Anthony Hunter
Department of Computer Science, University College London, London, UK
Arguments often omit premises or claims, forming enthymemes. We study pairwise logical selection between two candidates for the omitted component. Existing natural language methods can identify or generate candidates but often do not expose how the selected candidate completes the inference, while logic-based approaches usually assume that the required formulae and background knowledge are available. We extend a prior neuro-symbolic pipeline from missing-premise to missing-claim selection and replace binary entailment outcomes with logical-resistance scores. Top-Link uses weighted Partial MaxSAT under a single configuration of highest-confidence semantic links. We then introduce Possible-World Atom-Link Formalization (PWAL), which keeps translated formulae fixed and marginalizes logical resistance over alternative cross-formula semantic-link configurations. We evaluate PWAL on five tasks: ARCT and a CDED-derived task for missing-premise selection, iDebate- and AAE2-derived tasks for missing-claim selection, and alphaNLI for abductive hypothesis selection. Relative to Top-Link, PWAL raises strict accuracy by 2.95-30.86 percentage points and reduces tie rates by 4.57-58.00 percentage points on all five tasks. When ties receive half credit, accuracy still increases by 0.45-6.04 percentage points. PWAL also records the translated formulae, sampled link configurations, and resistance components for every comparison, providing a transparent trace of each score.
Xuyao Feng, Antonis Bikakis
Department of Computer Science, University College London · Department of Information Studies, University College London