Abductive Reasoning with Syllogistic Forms in Large Language Models

Hirohiko Abe; Risako Ando; Takanobu Morishita Kentaro Ozeki; Koji Mineshima; Mitsuhiro Okada

cs.CLMar 6, 2026

Abductive Reasoning with Syllogistic Forms in Large Language Models

Authors: Hirohiko Abe, Risako Ando, Takanobu Morishita Kentaro Ozeki, Koji Mineshima, Mitsuhiro Okada

Organizations: Keio University, Tokyo, Japan · The University of Tokyo, Tokyo, Japan

Abstract

Research in AI using Large-Language Models (LLMs) is rapidly evolving, and the comparison of their performance with human reasoning has become a key concern. Prior studies have indicated that LLMs and humans share similar biases, such as dismissing logically valid inferences that contradict common beliefs. However, criticizing LLMs for these biases might be unfair, considering our reasoning not only involves formal deduction but also abduction, which draws tentative conclusions from limited information. Abduction can be regarded as the inverse form of syllogism in its basic structure, that is, a process of drawing a minor premise from a major premise and conclusion. This paper explores the accuracy of LLMs in abductive reasoning by converting a syllogistic dataset into one suitable for abduction. It aims to investigate whether the state-of-the-art LLMs exhibit biases in abduction and to identify potential areas for improvement, emphasizing the importance of contextualized reasoning beyond formal deduction. This investigation is vital for advancing the understanding and application of LLMs in complex reasoning tasks, offering insights into bridging the gap between machine and human cognition.

Figures & tables

R : All C are B O : These A are B H : These A are C	R : All B are C O : These A are B H : None	R : All C are B O : These A are not B H : None	R : All B are C O : These A are not B H : These A are not C
R : No C are B O : These A are B H : None	R : No B are C O : These A are B H : None	R : No C are B O : These A are not B H : These A are C	R : No B are C O : These A are not B H : These A are C

Table 1: Eight patterns of abduction. R : Rule , O : Observation , H : Hypothesis . Those in yellow are correct abductions, while those in grey are incorrect. Correct abductions are those in which H explains why O is the case, given the rule R .

Type	Example
Consistent (66)	Rule : All people that had a fun time are smiling.
	Observation : These people are smiling.
	Hypothesis : These people had a fun time.
Inconsistent (66)	Rule : All things that are made in the sweet restaurant are spicy.
	Observation : These cakes are made in the sweet restaurant.
	Hypothesis : These cakes are spicy.

Table 2: Examples abductive syllogisms labeled as Consistent , Inconsistent , and Neutral . The numbers in brackets show the number of each type.

P1 : All C are B P2 : These A are B C : None	P1 : All B are C P2 : These A are B C : These A are C	P1 : All C are B P2 : These A are not B C : These A are not C	P1 : All B are C P2 : These A are not B C : None
P1 : No C are B P2 : These A are B C : These A are not C	P1 : No B are C P2 : These A are B C : These A are not C	P1 : No C are B P2 : These A are not B C : None	P1 : No B are C P2 : These A are not B C : None

Table 3: Eight patterns of deduction. P1 : Major Premise , P2 : Minor Premise , C : Conclusion . Those in yellow are valid deductions, while those in grey are invalid.

Panel of Table 4 — Table 4: Example prompts for the Abduction task and Deduction task.

Condition	Model	Overall	Positive	Negative	Neither	Consistent	Inconsistent	Neutral
Zero-Shot	GPT-3.5	31.02	48.15	100.00	0.93	31.82	27.27	33.33
	GPT-4	41.67	80.25	92.59	0.00	46.97	34.85	42.86
	Llama3-8B	37.50	61.73	66.67	12.04	42.42	27.27	41.67
	Llama3-70B	37.04	64.20	100.00	0.93	45.45	31.82	34.52
Few-Shot	GPT-3.5	29.63	44.44	96.30	1.85	33.33	22.73	32.14
	GPT-4	28.70	65.43	22.22	2.78	31.82	19.70	33.33

Table 5: Accuracy (%) on the Abduction task (

n=216

Condition	Model	Overall	Positive	Negative	Neither	Consistent	Inconsistent	Neutral
Zero-Shot	GPT-3.5	33.80	100.00	54.32	1.85	39.39	28.79	33.33
	GPT-4	72.22	100.00	100.00	44.44	74.24	68.18	73.81
	Llama-3-8B	43.52	100.00	40.74	31.48	37.88	37.88	52.38
	Llama-3-70B	53.24	100.00	80.25	21.30	60.61	40.91	57.14
Few-Shot	GPT-3.5	46.30	85.19	91.36	2.78	48.48	42.42	47.62
	GPT-4	95.83	100.00	96.30	94.44	100.00	92.42	95.24

Table 6: Accuracy (%) on the Deduction task (

n=216

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Table 7 — Table 7: Examples of alternative prompts not adopted.

Panel of Table 8 — Table 8: An example few-shot prompt for the Abduction task.

Panel of Table 9 — Table 9: An example few-shot prompt for the Deduction task.

Explore similar work

Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs
Apr 9, 2026Moein Salimi, Shaygan Adim, Danial Parnian +3Abductive Reasoning
Department of Computer Engineering, Sharif University of Technology · Department of Mathematical Sciences, Sharif University of Technology
Graph of States: Solving Abductive Tasks with Large Language Models
Mar 22, 2026Yu Luo, Rongchen Gao, Lu Teng +9Abductive Reasoning LLM Reasoning with Graphs
Nankai University · Wenzhou Medical University · Alibaba Cloud +2
LLMs Can See the Smoke but not the Fire: Evaluating Abductive Reasoning with Elenchos
Jul 14, 2026Julius Steiglechner, Lucas Mahler, Gabriele LohmannLLM Evaluation LLM Reasoning
Dept. of Biomedical Magnetic Resonance, University Hospital Tübingen, Tübingen, Germany · Max-Planck-Institute for Biological Cybernetics, Magnetic Resonance Center, Tübingen, Germany

Apr 9, 2026J/K move · Enter open · S save

Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs

Moein Salimi, Shaygan Adim, Danial Parnian, Nima Alighardashi+2

Department of Computer Engineering, Sharif University of Technology · Department of Mathematical Sciences, Sharif University of Technology

Abductive Reasoning

Open paper AskPDF arXiv

Regardless of its foundational role in human discovery and sense-making, abductive reasoning--the inference of the most plausible explanation for an observation--has been relatively underexplored in Large Language Models (LLMs). Despite the rapid advancement of LLMs, the exploration of abductive reasoning and its diverse facets has thus far been disjointed rather than cohesive. This paper presents the first survey of abductive reasoning in LLMs, tracing its trajectory from philosophical foundations to contemporary AI implementations. To address the widespread conceptual confusion and disjointed task definitions prevalent in the field, we establish a unified two-stage definition that formally categorizes prior work. This definition disentangles abduction into Hypothesis Generation, where models bridge epistemic gaps to produce candidate explanations, and Hypothesis Selection, where the generated candidates are evaluated and the most plausible explanation is chosen. Building upon this foundation, we present a comprehensive taxonomy of the literature, categorizing prior work based on their abductive tasks, datasets, underlying methodologies, and evaluation strategies. In order to ground our framework empirically, we conduct a compact benchmark study of current LLMs on abductive tasks, together with targeted comparative analyses across model sizes, model families, evaluation styles, and the distinct generation-versus-selection task typologies. Moreover, by synthesizing recent empirical results, we examine how LLM performance on abductive reasoning relates to deductive and inductive tasks, providing insights into their broader reasoning capabilities. Our analysis reveals critical gaps in current approaches--from static benchmark design and narrow domain coverage to narrow training frameworks and limited mechanistic understanding of abductive processes...

Abstract

Explore similar work

Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs

Graph of States: Solving Abductive Tasks with Large Language Models

LLMs Can See the Smoke but not the Fire: Evaluating Abductive Reasoning with Elenchos