Pairwise Logical Selection of Enthymeme Completions under Semantic-Link Uncertainty
Organizations: Department of Computer Science, University College London · Department of Information Studies, University College London
Abstract
Arguments often omit premises or claims, forming enthymemes. We study pairwise logical selection between two candidates for the omitted component. Existing natural language methods can identify or generate candidates but often do not expose how the selected candidate completes the inference, while logic-based approaches usually assume that the required formulae and background knowledge are available. We extend a prior neuro-symbolic pipeline from missing-premise to missing-claim selection and replace binary entailment outcomes with logical-resistance scores. Top-Link uses weighted Partial MaxSAT under a single configuration of highest-confidence semantic links. We then introduce Possible-World Atom-Link Formalization (PWAL), which keeps translated formulae fixed and marginalizes logical resistance over alternative cross-formula semantic-link configurations. We evaluate PWAL on five tasks: ARCT and a CDED-derived task for missing-premise selection, iDebate- and AAE2-derived tasks for missing-claim selection, and alphaNLI for abductive hypothesis selection. Relative to Top-Link, PWAL raises strict accuracy by 2.95-30.86 percentage points and reduces tie rates by 4.57-58.00 percentage points on all five tasks. When ties receive half credit, accuracy still increases by 0.45-6.04 percentage points. PWAL also records the translated formulae, sampled link configurations, and resistance components for every comparison, providing a transparent trace of each score.
Figures & tables
| Dataset ( ) | Metric | FH-HardSAT | Top-Link | PWAL | PWAL w/o no-link | PWAL w/o neg. guard | Direct NLI |
|---|---|---|---|---|---|---|---|
| NLI (400) | Acc | 11.00 | 36.50 | 57.75 | |||
| Tie | 55.75 | 27.25 | 0.00 | ||||
| EAcc | 38.88 | 50.13 | 57.75 | ||||
| ARCT (444) | Acc | 7.43 | 20.05 | 70.05 | |||
| Tie | 86.71 | 64.86 | 0.00 | ||||
| EAcc | 50.79 | 52.48 | 70.05 |
| Dataset | Metric | Ref. | ||
|---|---|---|---|---|
| NLI | Acc | |||
| Tie | ||||
| EAcc | ||||
| ARCT | Acc | |||
| Tie | ||||
| EAcc |
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Candidate pair | Candidate construction | Dev/Test; cluster |
|---|---|---|---|
| NLI ( Bhagavatula et al. 2020 ) | , | The two released hypotheses are retained in their released order. | 400/400; example |
| ARCT ( Habernal et al. 2018 ) | , | The two released warrants are retained. | 400/444; example |
| CDED ( Rinott et al. 2015 ) | , | Same-topic evidence annotated as evidence for another claim but not annotated as evidence for . | 200/400; topic |
| iDebate ( Wang and Ling 2016 ) | , | A different central claim from the same debate. | 400/400; debate |
| AAE2 ( Stab and Gurevych 2017 ) | , | An opposite-stance Claim from the same essay with a graph-disjoint support subtree. | 200/350; essay |
| Dataset | Clust. | Split ov. | Dup. | Ident. | Verb. | |
|---|---|---|---|---|---|---|
| CDED | 400 | 39 | 0 | 0 | 0 | 0 |
| iDebate | 400 | 128 | 0 | 0 | 0 | 1 |
| AAE2 | 350 | 127 | 0 | 0 | 0 | 0 |
| Dataset | Only gold | Both | Only distractor | Neither | Gold or both | |
|---|---|---|---|---|---|---|
| CDED | 50 | 39 | 9 | 0 | 2 | 48 (96%) |
| iDebate | 50 | 37 | 13 | 0 | 0 | 50 (100%) |
| AAE2 | 100 | 91 | 3 | 4 | 2 | 94 (94%) |
| Dataset | Dev Acc. (%) | Test | ||
|---|---|---|---|---|
| NLI | 90 | 0.55 | 11.50 | 44/133/223 |
| ARCT | 100 | 0.60 | 5.25 | 33/26/385 |
| CDED | 100 | 0.60 | 35.50 | 138/63/199 |
| iDebate | 100 | 0.70 | 29.25 | 132/48/220 |
| AAE2 | 100 | 0.60 | 17.50 | 66/67/217 |
| Dataset | TL ties | PWAL EAcc on TL ties | Repairs | Damages |
|---|---|---|---|---|
| NLI | 109 | |||
| ARCT | 288 | |||
| CDED | 54 | |||
| iDebate | 20 | |||
| AAE2 | 26 |
| Dataset | Wrong | Tension | Witness | Both |
|---|---|---|---|---|
| NLI | ||||
| ARCT | ||||
| CDED | ||||
| iDebate | ||||
| AAE2 |
| Dataset | Accuracy | Tie | EAcc | |
|---|---|---|---|---|
| NLI | 10 | 47.83 1.62 | 5.63 0.48 | 50.64 1.68 |
| 20 | 49.08 1.83 | 4.28 0.59 | 51.21 1.80 | |
| 50 | 49.30 0.86 | 3.18 0.51 | 50.89 0.76 | |
| 100 | 49.55 0.96 | 2.15 0.50 | 50.63 0.98 | |
| 200 | 49.92 1.16 | 1.48 0.25 | 50.66 1.17 | |
| Ref. | 49.75 | 3.25 | 51.38 |
| Dataset | Exact pairs | Capped pairs | Median | P95 |
|---|---|---|---|---|
| NLI | 357 | 43 | 294 | 205,920 |
| ARCT | 443 | 1 | 42 | 1,848 |
| CDED | 216 | 184 | 6,210 | |
| iDebate | 146 | 254 | 37,440 | |
| AAE2 | 93 | 257 | 170,586 |
| (a) Ten-seed mean-score approximation | ||||
|---|---|---|---|---|
| Dataset | Score MAE | Margin MAE | Decision agr. | Tie-status agr. |
| NLI | 0.0080 | 0.0117 | 95.25 | 97.25 |
| ARCT | 0.0078 | 0.0111 | 88.51 | 91.67 |
| CDED | 0.0069 | 0.0105 | 98.75 | 99.25 |
| iDebate | 0.0089 | 0.0122 | 99.00 | 100.00 |
| AAE2 | 0.0066 | 0.0094 | 99.71 | 100.00 |
| Dataset | Configuration | Mean | Median | P95 |
|---|---|---|---|---|
| NLI | Top-Link | 1.66 | 1.67 | 2.16 |
| PWAL | 14.58 | 14.37 | 21.71 | |
| PWAL | 27.31 | 26.19 | 41.30 | |
| PWAL | 68.19 | 66.75 | 101.80 | |
| PWAL | 135.59 | 133.95 | 203.65 | |
| PWAL | 280.89 | 283.08 | 411.32 |
| (a) Main-method contrasts | ||||
|---|---|---|---|---|
| Dataset | Contrast | Acc [95% CI] | EAcc [95% CI] | Tie reduction [95% CI] |
| NLI | PWAL Direct NLI | -8.20 [-14.52, -1.60] † | -7.12 [-13.46, -0.57] † | -2.15 [-3.38, -1.10] † |
| NLI | PWAL Top-Link | +13.05 [+8.45, +17.68] † | +0.50 [-3.70, +4.69] | +25.10 [+21.00, +29.38] † |
| NLI | PWAL FH-HardSAT | +38.55 [+33.27, +43.85] † | +11.75 [+6.41, +17.14] † | +53.60 [+48.62, +58.67] † |
| ARCT | PWAL Direct NLI | -19.14 [-25.18, -13.11] † | -15.71 [-21.69, -9.90] † | -6.87 [-9.14, -4.73] † |
| ARCT | PWAL Top-Link | +30.86 [+26.80, +34.91] † | +1.86 [-1.58, +5.24] | +58.00 [+53.33, +62.50] † |
| AMR record | Compiler action | Formula effect |
|---|---|---|
| and / or with :op | ordered branch traversal | / |
| multi-sentence with :snt or :rel | ordered sentence traversal | |
| :polarity - | local scope marker | |
| :condition | antecedent edge | |
| :name , :quant , :unit , and date/value fields | endpoint identity or lexical realization | no separate atom |
| :wiki | construction-only metadata | no separate atom |
| (a) PropBank function-tag resolution | |
| PropBank tag | Resolved relation class |
| PAG , CAU , SRC , EXT , PRP , TMP | agent, cause, source, extent, purpose, and time, respectively |
| PPT | refine to agent, property, instrument, accompaniment, path, source, location, topic, extent, attribute, active theme, or state; otherwise patient |
| GOL | refine to beneficiary, result, location, or instrument; otherwise goal |
| LOC | distinguish path, source, and goal; otherwise location |
| DIR | distinguish patient, source, goal, path, location, and beneficiary; otherwise direction |
| AMR role or atom kind | Verbalization template |
|---|---|
| purpose , beneficiary | |
| time , location | |
| direction | |
| domain | |
| mod | |
| manner |
| Join or resolved relation class | |
|---|---|
| ARG0+ARG1 | |
| accompanier ; beneficiary / purpose | ; |
| cause ; destination ; direction | ; ; |
| duration ; extent | ; |
| instrument ; location ; manner | ; ; |
| medium ; path ; source | ; ; |
| Premise | People learn a lot from comment sections. |
| Candidate A | too much learning hurts |
| Candidate B | learning never hurts |
| Claim | Comment sections have not failed |
| Candidate | Link | Source surface | Label | Soft clause | ||
|---|---|---|---|---|---|---|
| A | comment section | Con | 0.848 | 84 | ||
| A | person learn lot | Con | 0.982 | 98 | ||
| A | person learn toward section | Con | 0.971 | 97 | ||
| A | hurt much | Ent | 0.421 | 42 | ||
| B | comment section | Con | 0.848 | 84 | ||
| B | person learn lot | Con | 0.982 | 98 |
| World | Link | |||
|---|---|---|---|---|
| A0 | no-link | 22 | ||
| A1 | 17 | |||
| A2 | 17 | |||
| A3 | 17 | |||
| A4 | 27 |
| World | Link | |||
|---|---|---|---|---|
| B0 | no-link | 24 | ||
| B1 | 29 | |||
| B2 | 18 | |||
| B3 | 16 | |||
| B4 | 13 |
| Premise | Bill was poor. |
| Candidate A | His business went bankrupt and he had to take a new job. |
| Candidate B | His business is worth millions. |
| Outcome | Bill made less money as a computer scientist than he did before. |
| Candidate A source | Candidate B source | Target | |||
|---|---|---|---|---|---|
| Bill is poor | Bill is poor | computer scientist | |||
| business is bankrupted | multiple 1000000 dollars | Bill make money | |||
| job is new | he’s business | Bill make money | |||
| he’s business | business worth multiple | Bill make toward scientist | |||
| he take job | Bill make during before | ||||
| he obligate take | money have quant by less | ||||
| Cand. | Target | Source | Source surface | Label | ||
| A | Bill is poor | Con | 0.989 | 98 | ||
| A | business is bankrupted | Con | 0.973 | 97 | ||
| A | he’s business | Ent | 0.670 | 67 | ||
| A | he obligate take | Ent | 0.587 | 58 | ||
| A | job is new | Con | 0.983 | 98 | ||
| A | Bill is poor | Ent | 0.579 | 57 |
| Target | Candidate A choices | Candidate B choices |
|---|---|---|
| no-link (probability 1) | no-link (probability 1) | |
| no-link, Con , Con , Ent , Ent (each ) | no-link, Con , Ent , Ent , Ent (each ) | |
| same five A choices as (each ) | same five B choices as (each ) | |
| no-link (probability 1) | no-link (probability 1) | |
| no-link, Con (each ) | no-link (probability 1) | |
| no-link, Ent , Ent (each ) | no-link, Ent , Con , Con (each ) |
| Cand. | Draw | Active links | ||||
|---|---|---|---|---|---|---|
| A | 86 | Con | 97 | 97 | 0 | 1.000000 |
| A | 76 | Con ; Ent ; Con ; Ent ; Con | 287 | 387 | 0.455888 | |
| B | 12 | Ent ; Ent ; Con | 91 | 252 | 0.075397 | |
| B | 20 | Ent ; Ent ; Ent ; Ent | 0 | 322 | -0.571429 |