cs.AIApr 9, 2026
SaveWiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs
Organizations: Department of Computer Engineering, Sharif University of Technology · Department of Mathematical Sciences, Sharif University of Technology
Abstract
Regardless of its foundational role in human discovery and sense-making, abductive reasoning--the inference of the most plausible explanation for an observation--has been relatively underexplored in Large Language Models (LLMs). Despite the rapid advancement of LLMs, the exploration of abductive reasoning and its diverse facets has thus far been disjointed rather than cohesive. This paper presents the first survey of abductive reasoning in LLMs, tracing its trajectory from philosophical foundations to contemporary AI implementations. To address the widespread conceptual confusion and disjointed task definitions prevalent in the field, we establish a unified two-stage definition that formally categorizes prior work. This definition disentangles abduction into Hypothesis Generation, where models bridge epistemic gaps to produce candidate explanations, and Hypothesis Selection, where the generated candidates are evaluated and the most plausible explanation is chosen. Building upon this foundation, we present a comprehensive taxonomy of the literature, categorizing prior work based on their abductive tasks, datasets, underlying methodologies, and evaluation strategies. In order to ground our framework empirically, we conduct a compact benchmark study of current LLMs on abductive tasks, together with targeted comparative analyses across model sizes, model families, evaluation styles, and the distinct generation-versus-selection task typologies. Moreover, by synthesizing recent empirical results, we examine how LLM performance on abductive reasoning relates to deductive and inductive tasks, providing insights into their broader reasoning capabilities. Our analysis reveals critical gaps in current approaches--from static benchmark design and narrow domain coverage to narrow training frameworks and limited mechanistic understanding of abductive processes...
Figures & tables
Figure 1: Publication Trends of Abductive Reasoning Research in Computer Science. The stacked bar chart categorizes publications into LLM-based, NLP-based (Non-LLM), and other computer science approaches.
Figure 2: The Abductive Reasoning Pipeline. As defined in Section 2.2.1 , we conceptualize abduction as a two-stage process: (1) Hypothesis Generation , where an observation ( ) triggers the retrieval or creation of a candidate set to bridge an epistemic gap; and (2) Hypothesis Selection , where candidates are evaluated to identify the best explanation . This survey unites works that focus on either stage individually under this common framework.
| Property | Deduction | Induction | Abduction |
| Defeasibility | Non-defeasible | Defeasible | Defeasible |
| Monotonicity | Monotonic | Non-monotonic | Non-monotonic |
| Ampliativeness | Non-ampliative | Ampliative | Ampliative |
Table 1: Comparison of the three classical reasoning paradigms across key logical properties.
Figure 3: Categorization of literature on abductive reasoning in LLMs. The taxonomy is divided into four main dimensions: Task Formulation, Dataset Type, Methodology, and Evaluation Approach.
| Paper | Task | Dataset | Method | Evaluation |
|---|---|---|---|---|
| Abductive Commonsense Reasoning (ART) ( Bhagavatula et al., 2020 ) | A1, B1 | A1 | M2 | A1, B1, C |
| UniADILR ( Sheng et al., 2025 ) | A1 | B2 | M1, M2 | A1 |
| CauseJudger ( He & Lu, 2024 ) | A2 | B2 | M1 | A1 |
| Thinking Like a Skeptic: Defeasible Inference in Natural Language ( Rudinger et al., 2020 ) | A2 | A1 | M2 | A1 |
| DDXPlus ( Tchango et al., 2022 ) | A1 | B1 | M2 | A1 |
| An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models ( Liu et al., 2024 ) | B2 + A1 (full pipeline) | B3 | M1 | A1 |
Table 2: Unified categorization of abductive reasoning works.
| Model | ART | e-CARE | DDXPlus | True Detective | MuSR | |
| Top-1 | Hit@3 | |||||
| Qwen2.5 3B | 65.8 | 78.6 | 41.7 | 70.0 | 21.9 | 54.8 |
| Qwen2.5 7B | 75.8 | 83.0 | 54.5 | 86.2 | 26.1 | 58.0 |
| Qwen2.5 72B | 83.6 | 87.0 | 70.5 | 93.2 | 27.2 | 62.8 |
| Qwen3 8B | 73.0 | 83.8 | 60.5 | 89.2 | 34.5 | 54.8 |
| Qwen3 32B | 78.2 | 86.2 | 73.7 | 94.7 | 40.8 | 58.4 |
Table 3: Stage II (selection) results on the selected abductive benchmarks. DDXPlus is evaluated as a diagnosis-ranking task, so we report Top-1 and Hit@3 rather than simple accuracy. Reported human performance is unavailable for DDXPlus. For True Detective, the average human solve rate is 47%, although top human solvers exceed 80%.
| Model | ART Val. | e-CARE Val. | DDXPlus Hit@3 | UNcommonsense WR-C+L | ProofWriter Acc. | AbductionRules Acc. | NeuLR Acc. |
| Qwen2.5 3B | 64.6 | 79.6 | 32.0 | 6.5 | 0.0 | 20.0 | 10.0 |
| Qwen2.5 7B | 73.0 | 87.6 | 40.0 | 16.0 | 1.2 | 51.2 | 15.2 |
| Qwen2.5 72B | 90.0 | 94.0 | 51.5 | 21.0 | 9.7 | 86.8 | 43.2 |
| Qwen3 8B | 75.6 | 86.0 | 45.5 | 13.0 | 0.0 | 57.2 | 16.8 |
| Qwen3 32B | 85.0 | 95.6 | 47.0 | 24.0 | 3.5 | 84.8 | 31.2 |
| Llama3.1 8B | 74.0 | 84.4 | 40.0 | 14.0 | 1.5 | 68.0 | 10.8 |
Table 4: Stage I (generation) results on the primary task metrics. Val. denotes format/task validity for ART and e-CARE. Hit@3 on DDXPlus is top-3 hit rate after strict judge-based disease matching. WR-C+L on UNcommonsense is pairwise win rate against enhanced crowd-written references. Acc. denotes task accuracy under the benchmark’s standard evaluation protocol.
| Model | ART WR-H | e-CARE CEQ. | DDXPlus Set-F1@3 | UNcommonsense WR-C | ProofWriter F1 | AbductionRules Tok-F1 | NeuLR CLS. |
| Qwen2.5 3B | 37.6 | 0.045 | 17.1 | 16.5 | 0.3 | 73.6 | 61.7 |
| Qwen2.5 7B | 46.6 | 0.056 | 17.6 | 29.0 | 5.3 | 84.0 | 66.5 |
| Qwen2.5 72B | 69.3 | 0.051 | 23.3 | 47.5 | 26.5 | 95.3 | 79.9 |
| Qwen3 8B | 48.0 | 0.052 | 22.1 | 26.0 | 3.2 | 83.2 | 60.7 |
| Qwen3 32B | 63.6 | 0.056 | 23.0 | 43.5 | 30.5 | 95.2 | 73.7 |
| Llama3.1 8B | 46.0 | 0.038 | 19.1 | 31.0 | 17.8 | 90.9 | 61.5 |
Table 5: Complementary quality metrics for Stage I generation. WR-H is ART win rate against the human reference. CEQ. is causal explanation quality on e-CARE. Set-F1@3 on DDXPlus measures set overlap for the top-3 predicted diseases after strict judge-based matching. WR-C is win rate against crowd-written references on UNcommonsense. F1 for ProofWriter is proof-generation F1, Tok-F1 is token-level F1 on AbductionRules, and CLS. is mean character-level Levenshtein similarity on NeuLR.
| Model | ART | e-CARE | UNcommonsense | |||
| BLEU-4 | ROUGE-L | BERTScore | BLEU | ROUGE-L | BERTScore | |
| Qwen2.5 3B | 3.03 | 17.9 | 53.0 | 19.7 | 9.6 | 89.0 |
| Qwen2.5 7B | 1.83 | 21.7 | 55.2 | 26.4 | 15.9 | 89.2 |
| Qwen2.5 72B | 3.68 | 22.8 | 56.3 | 26.7 | 18.0 | 89.5 |
| Qwen3 8B | 2.43 | 19.2 | 53.8 | 26.2 | 16.5 | 89.5 |
| Qwen3 32B | 2.18 | 20.0 | 54.4 | 27.2 | 18.7 | 89.6 |
Table 6: Reference-based similarity scores for the open-ended generation tasks where such metrics are informative. For ART we report BLEU-4, ROUGE-L, and BERTScore; for e-CARE, BLEU and ROUGE-L; for UNcommonsense, BERTScore. We report these separately because lexical or embedding similarity does not always track judged explanatory quality.
Figure 4: Macro-average performance on the Stage I generation benchmarks. This figure summarizes overall model performance across the evaluated generation tasks.
Figure 5: Macro-average performance on the Stage II selection benchmarks. This figure summarizes overall model performance across the evaluated selection tasks.
Figure 6: Scaling trends within model families on the Stage I generation and Stage II selection benchmarks. The figure shows that performance generally improves with model scale within the Qwen and Llama families, although the magnitude of improvement varies across families and stages.
| Benchmark | Subset used in this study | Stage II formulation | Primary metrics |
| ART | 500 examples | Two-observation abductive selection with two candidate hypotheses; output only 1 or 2 . | Accuracy |
| e-CARE | 500 examples from the cause split only | Event-to-cause selection with two candidate causes; the effect split is omitted because it does not instantiate the abductive direction targeted here. | Accuracy |
| DDXPlus | 400 examples | Rank the provided candidate diagnoses from most to least likely; output diagnosis numbers only. | Top-1, Hit@3 |
| True Detective | 191 cases (entire dataset) | Solve the mystery from a long story and choose the single best answer option. | Accuracy |
| MuSR (murder) | 250 cases (entire murder split) | Read a short murder mystery and select the most likely culprit. | Accuracy |
Table 7: Stage II (hypothesis selection) benchmarks used in Section 4.1 .
| Benchmark | Subset used in this study | Stage I formulation | Primary metrics |
| ART | 300 examples | Generate one short intermediate event or state that bridges two observations. | Val., WR-H, BLEU-4, ROUGE-L, BERTScore |
| e-CARE | 500 examples from the cause split; validity judged on 250 of them | Generate one short conceptual explanation for an explicit cause–effect pair derived from the abductive cause setting. | Val., CEQ, BLEU, ROUGE-L |
| DDXPlus | 200 examples | Generate exactly three likely diagnoses directly, without a candidate list; this is our own open-ended formulation of the benchmark. | Hit@3, Set-F1@3 |
| UNcommonsense | 200 examples, with source proportions matched to the original dataset composition | Generate a plausible explanation for an unlikely outcome; the prompt template is selected by source (un-SocialIQA or un-RocStories). | WR-C, WR-C+L, BERTScore |
| ProofWriter | 400 examples from the D5-Ab test set, with 80 examples from each depth level 1–5 | Output all single missing facts that would each individually make the query provable. | Accuracy, F1 |
| AbductionRules | 250 examples from the Abduction-Animal test split, filtered to QCat = 0 , with 50 examples each from Q1, Q3, Q5, Q7, and Q9 | Output exactly one missing fact that makes the query provable. | Accuracy, Tok-F1 |
Table 8: Stage I (hypothesis generation) benchmarks used in Section 4.1 .
| Metric | Used on | Operational definition in this study |
|---|---|---|
| Accuracy | ART, e-CARE, True Detective, MuSR, ProofWriter, AbductionRules, NeuLR | Exact correctness under the task’s native answer key or verification procedure. On ProofWriter, this is exact-set accuracy: the full predicted set of missing facts must match the gold set. |
| Top-1 | DDXPlus selection | Whether the gold primary diagnosis is ranked first among the provided candidate diagnoses. |
| Hit@3 | DDXPlus selection and generation | In selection, whether the gold diagnosis appears in the model’s top three ranked candidate options. In generation, whether the gold primary diagnosis appears among the three generated diagnoses after strict judge-based disease matching. |
| Val. | ART and e-CARE generation | Percentage of outputs judged valid under the task-specific judge prompt. For e-CARE, validity was computed on 250 judged examples rather than the full 500-example generation subset. |
| WR-H / WR-C / WR-C+L | ART and UNcommonsense generation | Pairwise judge-based preference rates against a reference explanation: the human reference for ART (WR-H), crowd-written references for UNcommonsense (WR-C), and crowd-written references augmented with LLM-enhanced references for UNcommonsense (WR-C+L). The judge prompts also allow tie labels, which are tracked separately from outright wins. |
| CEQ | e-CARE generation | Causal Explanation Quality score, implemented to follow ( Du et al., 2022 ) as closely as possible: for generated explanation , , where . In words, CEQ measures how much the explanation increases the estimated causal strength of the cause–effect pair. |
Table 9: Metric glossary for Section 4.1 .
| Paper | Test Dataset | Reasoning Type |
| Sheng et al. (2025) | UniADILR-PSy (abduction split) | Abduction |
| Sheng et al. (2025) | UniADILR-PSy (deduction split) | Deduction |
| Sheng et al. (2025) | UniADILR-PSy (induction split) | Induction |
| Sheng et al. (2025) | UniADILR-HGc (abduction split) | Abduction |
| Sheng et al. (2025) | UniADILR-HGc (deduction split) | Deduction |
| Sheng et al. (2025) | UniADILR-HGc (induction split) | Induction |
Table 10: Datasets and corresponding reasoning types analyzed from the surveyed papers.
| Paper | Evaluated Models |
| ( Sheng et al., 2025 ) | LLaMA2-7B, LLaMA2-13B, LLaMA2-70B, GPT-3.5, GPT-4, T5-780M, T5-3B |
| ( Dougrez-Lewis et al., 2025 ) | Claude V3 Sonnet, GPT-4, GPT-4o |
| ( Xu et al., 2025a ) | LLaMA3.1-Chat, Mistral-Ins-v0.3, Claude-3.5, GPT-4 |
Table 11: Models evaluated within each referenced paper.
| Method Category | Description |
| -shot Prompting | In-context learning using demonstration examples. |
| Chain-of-Thought (CoT) | Prompting models to generate intermediate reasoning steps. |
| Supervised Fine-Tuning (SFT) | Fine-tuning on specific datasets (not always matched to the test reasoning type). |
Table 12: Inference and training methods utilized across the aggregated experiments.