Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs
Authors: Moein Salimi, Shaygan Adim, Danial Parnian, Nima Alighardashi, Mahdi Jafari Siavoshani, Mohammad Hossein Rohban
Organizations: Department of Computer Engineering, Sharif University of Technology · Department of Mathematical Sciences, Sharif University of Technology
Regardless of its foundational role in human discovery and sense-making, abductive reasoning--the inference of the most plausible explanation for an observation--has been relatively underexplored in Large Language Models (LLMs). Despite the rapid advancement of LLMs, the exploration of abductive reasoning and its diverse facets has thus far been disjointed rather than cohesive. This paper presents the first survey of abductive reasoning in LLMs, tracing its trajectory from philosophical foundations to contemporary AI implementations. To address the widespread conceptual confusion and disjointed task definitions prevalent in the field, we establish a unified two-stage definition that formally categorizes prior work. This definition disentangles abduction into Hypothesis Generation, where models bridge epistemic gaps to produce candidate explanations, and Hypothesis Selection, where the generated candidates are evaluated and the most plausible explanation is chosen. Building upon this foundation, we present a comprehensive taxonomy of the literature, categorizing prior work based on their abductive tasks, datasets, underlying methodologies, and evaluation strategies. In order to ground our framework empirically, we conduct a compact benchmark study of current LLMs on abductive tasks, together with targeted comparative analyses across model sizes, model families, evaluation styles, and the distinct generation-versus-selection task typologies. Moreover, by synthesizing recent empirical results, we examine how LLM performance on abductive reasoning relates to deductive and inductive tasks, providing insights into their broader reasoning capabilities. Our analysis reveals critical gaps in current approaches--from static benchmark design and narrow domain coverage to narrow training frameworks and limited mechanistic understanding of abductive processes...
Figures & tables
Figure 1: Publication Trends of Abductive Reasoning Research in Computer Science. The stacked bar chart categorizes publications into LLM-based, NLP-based (Non-LLM), and other computer science approaches.
Figure 2: The Abductive Reasoning Pipeline. As defined in Section 2.2.1 , we conceptualize abduction as a two-stage process: (1) Hypothesis Generation , where an observation ( O ) triggers the retrieval or creation of a candidate set H to bridge an epistemic gap; and (2) Hypothesis Selection , where candidates are evaluated to identify the best explanation h∗ . This survey unites works that focus on either stage individually under this common framework.
Property
Deduction
Induction
Abduction
Defeasibility
Non-defeasible
Defeasible
Defeasible
Monotonicity
Monotonic
Non-monotonic
Non-monotonic
Ampliativeness
Non-ampliative
Ampliative
Ampliative
Table 1: Comparison of the three classical reasoning paradigms across key logical properties.
Figure 3: Categorization of literature on abductive reasoning in LLMs. The taxonomy is divided into four main dimensions: Task Formulation, Dataset Type, Methodology, and Evaluation Approach.
Paper
Task
Dataset
Method
Evaluation
Abductive Commonsense Reasoning (ART) ( Bhagavatula et al., 2020 )
A1, B1
A1
M2
A1, B1, C
UniADILR ( Sheng et al., 2025 )
A1
B2
M1, M2
A1
CauseJudger ( He & Lu, 2024 )
A2
B2
M1
A1
Thinking Like a Skeptic: Defeasible Inference in Natural Language ( Rudinger et al., 2020 )
A2
A1
M2
A1
DDXPlus ( Tchango et al., 2022 )
A1
B1
M2
A1
An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models ( Liu et al., 2024 )
B2 + A1 (full pipeline)
B3
M1
A1
Table 2: Unified categorization of abductive reasoning works.
Model
ART
e-CARE
DDXPlus
True Detective
MuSR
Top-1
Hit@3
Qwen2.5 3B
65.8
78.6
41.7
70.0
21.9
54.8
Qwen2.5 7B
75.8
83.0
54.5
86.2
26.1
58.0
Qwen2.5 72B
83.6
87.0
70.5
93.2
27.2
62.8
Qwen3 8B
73.0
83.8
60.5
89.2
34.5
54.8
Qwen3 32B
78.2
86.2
73.7
94.7
40.8
58.4
Table 3: Stage II (selection) results on the selected abductive benchmarks. DDXPlus is evaluated as a diagnosis-ranking task, so we report Top-1 and Hit@3 rather than simple accuracy. Reported human performance is unavailable for DDXPlus. For True Detective, the average human solve rate is 47%, although top human solvers exceed 80%.
Model
ART Val.
e-CARE Val.
DDXPlus Hit@3
UNcommonsense WR-C+L
ProofWriter Acc.
AbductionRules Acc.
NeuLR Acc.
Qwen2.5 3B
64.6
79.6
32.0
6.5
0.0
20.0
10.0
Qwen2.5 7B
73.0
87.6
40.0
16.0
1.2
51.2
15.2
Qwen2.5 72B
90.0
94.0
51.5
21.0
9.7
86.8
43.2
Qwen3 8B
75.6
86.0
45.5
13.0
0.0
57.2
16.8
Qwen3 32B
85.0
95.6
47.0
24.0
3.5
84.8
31.2
Llama3.1 8B
74.0
84.4
40.0
14.0
1.5
68.0
10.8
Table 4: Stage I (generation) results on the primary task metrics. Val. denotes format/task validity for ART and e-CARE. Hit@3 on DDXPlus is top-3 hit rate after strict judge-based disease matching. WR-C+L on UNcommonsense is pairwise win rate against enhanced crowd-written references. Acc. denotes task accuracy under the benchmark’s standard evaluation protocol.
Model
ART WR-H
e-CARE CEQ.
DDXPlus Set-F1@3
UNcommonsense WR-C
ProofWriter F1
AbductionRules Tok-F1
NeuLR CLS.
Qwen2.5 3B
37.6
0.045
17.1
16.5
0.3
73.6
61.7
Qwen2.5 7B
46.6
0.056
17.6
29.0
5.3
84.0
66.5
Qwen2.5 72B
69.3
0.051
23.3
47.5
26.5
95.3
79.9
Qwen3 8B
48.0
0.052
22.1
26.0
3.2
83.2
60.7
Qwen3 32B
63.6
0.056
23.0
43.5
30.5
95.2
73.7
Llama3.1 8B
46.0
0.038
19.1
31.0
17.8
90.9
61.5
Table 5: Complementary quality metrics for Stage I generation. WR-H is ART win rate against the human reference. CEQ. is causal explanation quality on e-CARE. Set-F1@3 on DDXPlus measures set overlap for the top-3 predicted diseases after strict judge-based matching. WR-C is win rate against crowd-written references on UNcommonsense. F1 for ProofWriter is proof-generation F1, Tok-F1 is token-level F1 on AbductionRules, and CLS. is mean character-level Levenshtein similarity on NeuLR.
Model
ART
e-CARE
UNcommonsense
BLEU-4
ROUGE-L
BERTScore
BLEU
ROUGE-L
BERTScore
Qwen2.5 3B
3.03
17.9
53.0
19.7
9.6
89.0
Qwen2.5 7B
1.83
21.7
55.2
26.4
15.9
89.2
Qwen2.5 72B
3.68
22.8
56.3
26.7
18.0
89.5
Qwen3 8B
2.43
19.2
53.8
26.2
16.5
89.5
Qwen3 32B
2.18
20.0
54.4
27.2
18.7
89.6
Table 6: Reference-based similarity scores for the open-ended generation tasks where such metrics are informative. For ART we report BLEU-4, ROUGE-L, and BERTScore; for e-CARE, BLEU and ROUGE-L; for UNcommonsense, BERTScore. We report these separately because lexical or embedding similarity does not always track judged explanatory quality.
Figure 4: Macro-average performance on the Stage I generation benchmarks. This figure summarizes overall model performance across the evaluated generation tasks.
Figure 5: Macro-average performance on the Stage II selection benchmarks. This figure summarizes overall model performance across the evaluated selection tasks.
Figure 6: Scaling trends within model families on the Stage I generation and Stage II selection benchmarks. The figure shows that performance generally improves with model scale within the Qwen and Llama families, although the magnitude of improvement varies across families and stages.
Figure 13
Benchmark
Subset used in this study
Stage II formulation
Primary metrics
ART
500 examples
Two-observation abductive selection with two candidate hypotheses; output only 1 or 2 .
Accuracy
e-CARE
500 examples from the cause split only
Event-to-cause selection with two candidate causes; the effect split is omitted because it does not instantiate the abductive direction targeted here.
Accuracy
DDXPlus
400 examples
Rank the provided candidate diagnoses from most to least likely; output diagnosis numbers only.
Top-1, Hit@3
True Detective
191 cases (entire dataset)
Solve the mystery from a long story and choose the single best answer option.
Accuracy
MuSR (murder)
250 cases (entire murder split)
Read a short murder mystery and select the most likely culprit.
Accuracy
Table 7: Stage II (hypothesis selection) benchmarks used in Section 4.1 .
Benchmark
Subset used in this study
Stage I formulation
Primary metrics
ART
300 examples
Generate one short intermediate event or state that bridges two observations.
Val., WR-H, BLEU-4, ROUGE-L, BERTScore
e-CARE
500 examples from the cause split; validity judged on 250 of them
Generate one short conceptual explanation for an explicit cause–effect pair derived from the abductive cause setting.
Val., CEQ, BLEU, ROUGE-L
DDXPlus
200 examples
Generate exactly three likely diagnoses directly, without a candidate list; this is our own open-ended formulation of the benchmark.
Hit@3, Set-F1@3
UNcommonsense
200 examples, with source proportions matched to the original dataset composition
Generate a plausible explanation for an unlikely outcome; the prompt template is selected by source (un-SocialIQA or un-RocStories).
WR-C, WR-C+L, BERTScore
ProofWriter
400 examples from the D5-Ab test set, with 80 examples from each depth level 1–5
Output all single missing facts that would each individually make the query provable.
Accuracy, F1
AbductionRules
250 examples from the Abduction-Animal test split, filtered to QCat = 0 , with 50 examples each from Q1, Q3, Q5, Q7, and Q9
Output exactly one missing fact that makes the query provable.
Accuracy, Tok-F1
Table 8: Stage I (hypothesis generation) benchmarks used in Section 4.1 .
Exact correctness under the task’s native answer key or verification procedure. On ProofWriter, this is exact-set accuracy: the full predicted set of missing facts must match the gold set.
Top-1
DDXPlus selection
Whether the gold primary diagnosis is ranked first among the provided candidate diagnoses.
Hit@3
DDXPlus selection and generation
In selection, whether the gold diagnosis appears in the model’s top three ranked candidate options. In generation, whether the gold primary diagnosis appears among the three generated diagnoses after strict judge-based disease matching.
Val.
ART and e-CARE generation
Percentage of outputs judged valid under the task-specific judge prompt. For e-CARE, validity was computed on 250 judged examples rather than the full 500-example generation subset.
WR-H / WR-C / WR-C+L
ART and UNcommonsense generation
Pairwise judge-based preference rates against a reference explanation: the human reference for ART (WR-H), crowd-written references for UNcommonsense (WR-C), and crowd-written references augmented with LLM-enhanced references for UNcommonsense (WR-C+L). The judge prompts also allow tie labels, which are tracked separately from outright wins.
CEQ
e-CARE generation
Causal Explanation Quality score, implemented to follow ( Du et al., 2022 ) as closely as possible: for generated explanation X , CEQ(X)=cs(C,E∣X)−cs(C,E) , where cs(C,E∣X)=max[cs(C+X,E),cs(C,E+X)] . In words, CEQ measures how much the explanation increases the estimated causal strength of the cause–effect pair.
Table 9: Metric glossary for Section 4.1 .
Paper
Test Dataset
Reasoning Type
Sheng et al. (2025)
UniADILR-PSy (abduction split)
Abduction
Sheng et al. (2025)
UniADILR-PSy (deduction split)
Deduction
Sheng et al. (2025)
UniADILR-PSy (induction split)
Induction
Sheng et al. (2025)
UniADILR-HGc (abduction split)
Abduction
Sheng et al. (2025)
UniADILR-HGc (deduction split)
Deduction
Sheng et al. (2025)
UniADILR-HGc (induction split)
Induction
Table 10: Datasets and corresponding reasoning types analyzed from the surveyed papers.
Research in AI using Large-Language Models (LLMs) is rapidly evolving, and the comparison of their performance with human reasoning has become a key concern. Prior studies have indicated that LLMs and humans share similar biases, such as dismissing logically valid inferences that contradict common beliefs. However, criticizing LLMs for these biases might be unfair, considering our reasoning not only involves formal deduction but also abduction, which draws tentative conclusions from limited information. Abduction can be regarded as the inverse form of syllogism in its basic structure, that is, a process of drawing a minor premise from a major premise and conclusion. This paper explores the accuracy of LLMs in abductive reasoning by converting a syllogistic dataset into one suitable for abduction. It aims to investigate whether the state-of-the-art LLMs exhibit biases in abduction and to identify potential areas for improvement, emphasizing the importance of contextualized reasoning beyond formal deduction. This investigation is vital for advancing the understanding and application of LLMs in complex reasoning tasks, offering insights into bridging the gap between machine and human cognition.
Large language models (LLMs) excel at pattern recognition and text generation, but their capacity for abductive inference - inferring latent hypotheses that explain observed behavior - remains poorly understood. Here, we introduce Elenchos (named after the Socratic method of cross-examination), a generative evaluation framework that measures abductive reasoning as a structural inverse problem. Given a reference formal system, such as the lambda-calculus, and a potentially mutated counterpart, agents must determine whether a mutation has occurred and infer the rule modifications responsible for the resulting behavioral differences. Evaluating frontier and mid-tier LLMs reveals a consistent detection-attribution dissociation: models often recognize that a system has been altered but struggle to identify the latent mutations causing the observed discrepancies. Performance degrades substantially under interacting mutations, where models frequently recover only a subset of the underlying mutations. Preliminary evidence also suggests diminishing returns from increased inference-time reasoning, with only modest improvements under larger reasoning budgets, though this finding requires further validation.
Julius Steiglechner, Lucas Mahler, Gabriele Lohmann
Dept. of Biomedical Magnetic Resonance, University Hospital Tübingen, Tübingen, Germany · Max-Planck-Institute for Biological Cybernetics, Magnetic Resonance Center, Tübingen, Germany
Abductive reasoning requires forming hypotheses that explain observed evidence and revising them as new evidence becomes available. While large language models (LLMs) are often evaluated on whether they solve abductive reasoning tasks correctly, less is known about how they acquire evidence, update their hypotheses, and decide when to stop. We introduce Alien Abduction game, an interactive probe for studying these behaviours under different interaction modes. The modes vary in whether evidence is provided upfront or across turns, and whether queries are selected by the model or examples are provided by the oracle. Across models, providing evidence upfront leads to higher success rates than distributing it across turns. In multi-turn settings, some models commit before using the available evidence, while others exhaust the turn budget without converging. Models also achieve higher success rates when examples are provided by the oracle than when they select their own queries, although their final hypotheses are more consistent with the evidence they selected. These findings suggest that models may form hypotheses that fit self-selected evidence without sufficiently distinguishing them from alternatives, and may struggle to validate and refine their hypotheses or determine when to stop.
Computational Linguistics, Department of Linguistics University of Potsdam, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany