CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs

Moein Salimi; Danial Parnian; Shaygan Adim; Amirmohammad Ebrahiminasab; Nima Alighardashi; Parsa Gholami; Sahand Akramipour; Mahdi Jafari Siavoshani

cs.AIAug 14, 2026

CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs

Authors: Moein Salimi, Danial Parnian, Shaygan Adim, Amirmohammad Ebrahiminasab, Nima Alighardashi, Parsa Gholami, Sahand Akramipour, Mahdi Jafari Siavoshani, +1 more

Organizations: Sharif University of Technology · University of Tehran

Abstract

Abductive reasoning, often characterized as inference to the best explanation, is central to explanation under uncertainty, from everyday sense-making and investigation to scientific discovery. Yet LLM research has mostly studied abduction through narrow, task-specific benchmarks, making it unclear whether observed gains transfer beyond the benchmark family used for training or evaluation. We ask whether RL post-training can improve abduction as a transferable reasoning capability. We introduce CEDAR-GRPO, a process-aware framework that combines final-answer correctness with abductive rewards for evidence coverage and evidence-to-explanation directionality. Four open-weight LLMs are post-trained on a controlled, domain-neutral mixture of abductive hypothesis-generation and hypothesis-selection tasks. We evaluate them on 11 unseen tasks spanning hypothesis selection, missing-fact generation, defeasible inference, long-context investigation, clinical reasoning, code debugging, and non-abductive controls. CEDAR- GRPO improves every model on every held-out task over both base models and correctness-only GRPO, with average gains of 7.4 and 2.7 points, respectively, and a maximum gain of 30.8 points. Ablations confirm that RL, abductive reward design, and task diversity each contribute to transfer. Process-level metrics further show stronger abductive behavior, including exploration of alternatives, elimination of rivals, backtracking, and uncertainty marking.

Figures & tables

Panel of Figure 1 — Figure 1: CEDAR-GRPO grounds its answer in the observed evidence, while correctness-only GRPO relies on unsupported assumptions.

Figure 2: Overview of CEDAR-GRPO. The framework trains LLMs on abductive hypothesis generation and selection tasks using structured outputs and a composite reward for correctness, evidence coverage, and evidence–explanation directionality, then evaluates both task accuracy and reasoning behavior on held-out datasets.

Panel of Figure 3 — Figure 3: How CEDAR-GRPO’s composite reward shapes a BalancedCOPA reasoning trace over training. All three snapshots select the correct answer, so $r_{\mathrm{cor}}$ is already maximal. At Step 48, the trace reasons from an assumed explanation back to a generic consequence and accounts for none of the three observation details returned by the coverage judge. By Step 534, it starts from the observation and links two details to the hypothesis. By Step 1,018, it explains all three details while preserving the evidence-to-explanation direction, yielding the maximum reward.

Model

Method

ART

B-COPA

DefNLI

GoEmo.

MuSR-M

MuSR-O

MuSR-T

NeuLR

StratQA

MedQA

ML-Debug

Qwen3-4B

Base

65.25\%

84.40\%

80.75\%

27.75\%

20.40\%

19.20\%

51.60\%

35.00\%

38.25\%

35.50\%

25.25\%

Cor-GRPO

71.75\%

86.80\%

86.75\%

32.25\%

41.60\%

32.40\%

48.80\%

38.75\%

43.25\%

37.50\%

28.50\%

CEDAR-GRPO

\mathbf{72.25\%}

\mathbf{88.80\%}

\mathbf{88.50\%}

\mathbf{34.25\%}

\mathbf{48.40\%}

\mathbf{33.20\%}

\mathbf{55.60\%}

\mathbf{40.50\%}

\mathbf{44.75\%}

\mathbf{37.75\%}

\mathbf{29.00\%}

Qwen3-8B

Base

72.25\%

84.40\%

82.50\%

39.50\%

23.20\%

26.40\%

55.20\%

36.25\%

40.00\%

42.25\%

28.75\%

Cor-GRPO

74.00\%

86.00\%

87.75\%

45.50\%

43.60\%

34.00\%

58.80\%

39.50\%

44.50\%

46.25\%

29.75\%

CEDAR-GRPO

\mathbf{75.50\%}

\mathbf{89.20\%}

\mathbf{90.50\%}

\mathbf{47.00\%}

\mathbf{46.80\%}

\mathbf{36.80\%}

\mathbf{60.80\%}

\mathbf{43.75\%}

\mathbf{50.50\%}

\mathbf{49.25\%}

\mathbf{31.75\%}

Table 1: Held-out task performance across the evaluation suite. Scores represent task-native accuracy or exact-success metrics. CEDAR-GRPO is our composite-reward method; Cor-GRPO is the correctness-only baseline. The best score per backbone is bolded. Full dataset descriptions and abbreviations are provided in Appendix A .

Metric	Description
Branchiness	Exploring multiple distinct candidate explanations for the same observation.
Backtracking	Explicitly identifying an error or flaw in the reasoning and changing direction.
Differential Elimination	The active refutation of alternative hypotheses given the specific context.
Prior Invocation	Incorporating typicality or prior probability alongside case-specific evidence.
Evidence Coverage	The fraction of specific observation details explicitly accounted for by the chosen hypothesis.
Evidence–Expl. Directionality	Demonstrating awareness that reasoning must move from given evidence toward an explanatory conclusion.

Table 2: Overview of the process-level metrics used to evaluate intermediate reasoning behaviors.

	Held-out Metrics					Reward Metrics
Method	Backtracking	Branchiness	Diff. Elim.	Prior	Uncertainty	Coverage	Direction.
Baseline	0.69	1.22	0.79	0.59	0.87	33.1%	0.21
Cor-GRPO	0.93	1.16	0.97	0.53	0.92	39.1%	0.16
CEDAR-GRPO	1.09	1.53	1.29	0.72	1.37	52.9%	0.60

Table 3: Mean process-level metric scores averaged across ten held-out evaluation datasets for DeepSeek-R1-Distill-Qwen-7B. Evidence Coverage and Directionality were explicitly included as rewards in the CEDAR-GRPO training objective; the other five metrics are fully held-out metrics used to assess reasoning generalization. Shading indicates a relative increase ( green ) or decrease ( red ) against the Baseline. Full per-dataset breakdowns are provided in Appendix E , Table 10 .

Model

Ablation

Method

ART

B-COPA

DefNLI

GoEmo.

MuSR-M

MuSR-O

MuSR-T

NeuLR

StratQA

MedQA

ML-Debug

Avg.

\Delta

vs Base

Qwen3-4B

Reference

Base

65.25\%

84.40\%

80.75\%

27.75\%

20.40\%

19.20\%

51.60\%

35.00\%

38.25\%

35.50\%

25.25\%

0.00

Reference

Cor-GRPO

71.75\%

86.80\%

86.75\%

32.25\%

41.60\%

32.40\%

48.80\%

38.75\%

43.25\%

37.50\%

28.50\%

+5.91

Main

CEDAR-GRPO

\mathbf{72.25\%}

\mathbf{88.80\%}

\mathbf{88.50\%}

34.25\%

\mathbf{48.40\%}

\mathbf{33.20\%}

\mathbf{55.60\%}

\mathbf{40.50\%}

44.75\%

\mathbf{37.75\%}

\mathbf{29.00\%}

+8.15

RL vs SFT

SFT

66.75\%

86.00\%

80.75\%

28.50\%

22.80\%

17.60\%

48.40\%

33.25\%

37.25\%

33.25\%

24.25\%

-0.41

Reward

Cor+Cov-GRPO

69.50\%

88.00\%

87.00\%

32.00\%

43.20\%

\mathbf{33.20\%}

52.00\%

39.00\%

43.50\%

37.00\%

29.00\%

+6.37

Reward

Cor+Dir-GRPO

67.00\%

88.40\%

87.25\%

33.00\%

43.60\%

30.00\%

53.20\%

40.00\%

44.00\%

37.25\%

28.50\%

+6.26

Table 4: Unified ablation results on the two evaluated backbones, isolating the effects of supervised fine-tuning, reward composition, training stages, and generic reasoning post-training. Performance is compared against our main CEDAR-GRPO method and the Cor-GRPO baseline. All columns report task-native metrics on the same held-out evaluation datasets as Table 1 .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Relation to abduction	Samples	Training role
UniADILR-HGc (abductive) ( Sheng et al., 2025 )	Stage II	400	Abstract logical abductive selection with distractor premises; compact signal for evaluating candidate explanations.
Balanced COPA (cause) ( Kavumba et al., 2020 )	Stage II	400	Commonsense causal selection while reducing superficial lexical cues.
CauseLogics ( He and Lu, 2024 )	Stage II-adjacent	400	Binary candidate–cause verification with longer reasoning chains; deeper evaluative reasoning.
CLIMATE-FEVER ( Diggelmann et al., 2020 )	Stage II-adjacent	400	Four-way claim–evidence plausibility judgment over supports, refutes, not enough info, and disputed; broader verification-style reasoning.
AbductionRules ( Young et al., 2022 )	Stage I	400	Abductive generation of missing explanatory facts from rule contexts.
Crypto ( Li et al., 2025a )	Stage I	200	Cipher-rule learning (Caesar, Atbash) from examples.

Appendix

Table 5: Training and validation datasets used for GRPO fine-tuning. Each dataset contributes a fixed number of samples (400 for most datasets, 200 for Crypto and List Function), split 80/20 into train/validation; Which sums up to a total number of 2400 samples, consisting of 1920 training and 480 validation samples.

Dataset	Relation	Samples	Evaluation role
Classic abductive benchmarks
ART ( $\alpha$ NLI) ( Bhagavatula et al., 2019 )	Stage II	400	Conventional Stage II abductive selection between alternative hypotheses over narrative bridges.
NeuLR ( Xu et al., 2023 )	Stage I	400	Conventional Stage I abductive split; missing-fact generation in abstract logical contexts.
Stage II evidence-update evaluation
Defeasible NLI ( Rudinger et al., 2020 )	Stage II-adjacent	400	Determines whether an update strengthens or weakens a hypothesis; tests evidence-based plausibility assessment.
Abductive interpretations in neighboring domains

Appendix

Table 6: Held-out evaluation datasets used for model assessment. These datasets are not used for GRPO training or validation-based selection. The Samples column reports the number of held-out examples evaluated in this study, including full selected subsets for some datasets and fixed-size samples from larger source splits for others.

Hyperparameter	Value	Hyperparameter	Value
Base Models	DeepSeek-R1-Distill-Qwen-7B, Qwen3-4B, Qwen3-8B, Llama-3.1-8B-Instruct	Learning Rate	$1\times 10^{-5}$
Quantization	NF4 (4-bit)	LR Schedule	Cosine
LoRA Rank ( $r$ )	64	Warmup Steps	2
LoRA Alpha ( $\alpha$ )	64	Weight Decay	0.1
GRPO Group Size ( $G$ )	4	Optimizer	AdamW (Torch)
Max Sequence Length	4096	Adam $\beta_{1},\beta_{2}$	0.9, 0.99

Appendix

Table 7: Training and generation hyperparameters across model architectures.

Dataset	Verifier
BalancedCOPA	Parse an integer and compare it with the one-based gold label.
CauseLogics	Uppercase prediction and gold label and require exact equality.
ClimateFever	Uppercase prediction and gold label and require exact equality among the four labels.
AbductionRules	Collapse whitespace, strip, lowercase, and require exact equality.
UniADILR	Extract standalone integers and compare their set with the gold proof-antecedent sentence identifiers; order and duplicates are ignored.
Crypto	Execute transform(s) on every held-out test input and require exact string outputs for all tests.

Appendix

Table 8: Dataset-specific correctness verifiers used during training.

Panel of Table 9 — Table 9: Screening candidate process-level metrics as possible reward terms. Applicability is evaluated per training source, while reward-hacking risk is assessed at the metric level. This screening motivates our reward-composition choice: Evidence Coverage and Evidence–Explanation Directionality are selected because they are broadly applicable across datasets and comparatively resistant to superficial optimization.

		Datasets
		ART	B-COPA	GoEmo.	MedQA	DefNLI	MuSR-M	MuSR-O	MuSR-T	NeuLR	StratQA
Condition	Metric
Cor-GRPO	Backtracking	0.5	0.9	0.2	0.8	1.5	0.8	1.6	0.6	1.5	0.9
	Branchiness	1.2	0.4	0.9	1.5	1.6	1.2	1.3	1.5	1.4	0.6
	Coverage (%)	43.0	41.0	23.0	34.0	46.0	49.0	39.0	44.0	35.0	37.0
	Diff. Elim.	1.0	1.0	0.6	0.7	1.0	0.9	1.1	0.7	1.5	1.2

Appendix

Table 10: Process-level metric scores for DeepSeek-R1-Distill-Qwen-7B. Values are bolded if they are the maximum for that specific metric and dataset across all three conditions.

Dataset	Unc. Markers	Branch.	Back.	Diff. Elim.	Prior	Obs. Cov.	Direct.
ART	Focus on uncertainty while reasoning about which hypothesis better explains observations.	Do not count simple option selection; count only internal exploration.	Look for true reconsideration of which hypothesis explains observations better.	Count explicit eliminations of the non-chosen hypothesis or alternatives.	Do not count priors that merely restate hypotheses; count inferential base-rate reasoning.	Treat ‘Observation 1’ and ‘Observation 2’ as complete; extract all details.	Check reasoning starts from observations and seeks hypothesis, not back-fitting.
B-COPA	Focus on uncertainty in causal reasoning and option comparison.	Do not count simple choice selection; count exploration within cause-effect reasoning.	Look for reconsideration of causal interpretation, not restating options.	Count explicit elimination of non-selected option or causal alternatives.	Priors about typical everyday cause-effect relationships.	Treat ‘Cause:’ as observation; extract relevant details.	‘Cause’ is evidence; model must evaluate which option is most plausible effect.
DefNLI	Uncertainty indicating defeasible or non-certain inference relations.	Exploration of stronger vs. weaker inferences and defeasible conclusions.	Reconsideration when inference appears defeated or less robust.	Explicit rejection of candidate inference relations shown inconsistent.	Priors about typical premise-hypothesis relations and defaults.	Treat ‘Premise’, ‘Hypothesis’, ‘Update’ together as complete observation.	Reasoning evaluates logical impact of Update from given text.
GoEmo.	Uncertainty in label selection; avoid counting task-domain terms.	Count only if model explores multiple emotion labels.	Count only genuine revisions in emotion-label selection.	Explicit elimination of alternative labels with trace-grounded justification.	Priors about emotions typically associated with contexts.	Treat ‘Text’ as complete observation; extract salient spans.	Reasoning extracts cues before selecting emotion label.

Appendix

Table 11: Dataset-specific notes for all process-level metrics injected into the judge prompt. Each cell gives the exact text appended to the shared system prompt for that dataset.

Dataset	Unc. Markers	Branch.	Back.	Diff. Elim.	Prior	Obs. Cov.	Direct.
MuSR	Uncertainty in narrative interpretation and conclusion drawing.	Exploration of different narrative interpretations or solutions.	Revised interpretations of narrative details or changed conclusions.	Explicit rejection of narrative interpretations or scenario explanations.	Priors about typical narrative behavior, motives, patterns.	Treat ‘Context’ and ‘Problem’ as complete; extract narrative details (actors, timing, locations).	Reasoning builds explanations from details, not assumes conclusion and back-fits.
MedQA	Extract markers only from model’s own reasoning, not probabilistic symptoms.	Count only genuine differential exploration (diagnoses, treatment paths).	Corrections in diagnostic/treatment reasoning or missed details.	Each answer choice explicitly ruled out with clinical rationale.	Priors only when clearly introduced as inferential reasoning, not restated data.	Treat ‘Problem’ as observation; extract all clinical details.	Reasoning starts from clinical evidence and moves toward diagnosis/treatment.
NeuLR	Uncertainty while weighing competing abductive explanations.	Do not count superficial option comparison; genuine exploration.	True reconsideration of which explanation best fits observations.	Explicit elimination moves using concrete mismatch evidence.	Priors justifying why one hypothesis more plausible.	Treat ‘Logical Rules and Known Facts’ and ‘Target Conclusion’ as complete.	Reasoning uses Rules/Facts to find Missing Fact, not work backwards.
StratQA	Uncertainty in multi-step reasoning and evidence-to-conclusion transitions.	Exploration of multiple reasoning paths or inference chains.	Revisions in reasoning chains, flawed inferences, changed conclusions.	Explicit rejection of competing YES/NO reasoning lines.	World-knowledge priors and typicality assumptions.	Treat ‘Question’ and ‘Evidence’ as complete; extract relevant facts.	Reasoning builds upon facts toward YES/NO answer, not cherry-picks post-hoc.

Appendix

Table 11: Dataset-specific notes for all process-level metrics injected into the judge prompt. Each cell gives the exact text appended to the shared system prompt for that dataset. (continued)

Setting	SFT configuration
Data	1,920 training and 480 validation examples; sequence packing disabled
Quantization / LoRA	4-bit NF4; rank $r=64$ ; $\alpha=64$ ; Unsloth gradient checkpointing
LoRA targets	q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj
Sequence length	4,096 tokens
Optimizer	AdamW (Torch), learning rate $1\times 10^{-5}$ , $(\beta_{1},\beta_{2})=(0.9,0.99)$ , weight decay $0.1$
Schedule / clipping	Cosine decay, 2 warmup steps, maximum gradient norm $0.1$

Appendix

Table 12: Optimization and model-selection setup for the SFT ablation. Held-out evaluation tasks are not used for checkpoint selection.

Dataset	Reasoning profile	Samples	Role in the ablation mixture
GSM8K ( Cobbe et al., 2021 )	Arithmetic, multi-step reasoning	400	Introduces numerical and procedural reasoning through grade-school math word problems. Final supervision is based on the extracted numerical answer.
FOLIO ( Han et al., 2024 )	Formal logical inference	400	Provides premise–conclusion judgments grounded in first-order logical structure, adding a strongly symbolic component to the mixture.
CommonsenseQA ( Talmor et al., 2019 )	Commonsense multiple choice	400	Supplies broad everyday reasoning questions in a fixed five-choice format, complementing the more formal logical tasks.
VitaminC ( Schuster et al., 2021 )	Fact verification under contrastive evidence	400	Adds evidence-sensitive classification in which small textual changes can reverse the correct label, making it useful for testing fine-grained judgment.
MMLU ( Hendrycks et al., 2021 )	Academic multiple choice	400	Contributes subject-diverse question answering from a reasoning-oriented subset of academic domains.
BIG-Bench ( Srivastava et al., 2022 )	Heterogeneous multiple-choice reasoning	400	Adds task diversity through four reasoning-focused multiple-choice tasks spanning deduction, state tracking, date reasoning, and tabular reasoning.

Appendix

Table 13: Composition of the general-reasoning ablation mixture. Each source contributes 400 instances, producing a balanced 2,400-example corpus with an 80/20 train/validation split.

Explore similar work

Apr 9, 2026

Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMs

Regardless of its foundational role in human discovery and sense-making, abductive reasoning--the inference of the most plausible explanation for an observation--has been relatively underexplored in Large Language Models (LLMs). Despite the rapid advancement of LLMs, the exploration of abductive reasoning and its diverse facets has thus far been disjointed rather than cohesive. This paper presents the first survey of abductive reasoning in LLMs, tracing its trajectory from philosophical foundations to contemporary AI implementations. To address the widespread conceptual confusion and disjointed task definitions prevalent in the field, we establish a unified two-stage definition that formally categorizes prior work. This definition disentangles abduction into Hypothesis Generation, where models bridge epistemic gaps to produce candidate explanations, and Hypothesis Selection, where the generated candidates are evaluated and the most plausible explanation is chosen. Building upon this foundation, we present a comprehensive taxonomy of the literature, categorizing prior work based on their abductive tasks, datasets, underlying methodologies, and evaluation strategies. In order to ground our framework empirically, we conduct a compact benchmark study of current LLMs on abductive tasks, together with targeted comparative analyses across model sizes, model families, evaluation styles, and the distinct generation-versus-selection task typologies. Moreover, by synthesizing recent empirical results, we examine how LLM performance on abductive reasoning relates to deductive and inductive tasks, providing insights into their broader reasoning capabilities. Our analysis reveals critical gaps in current approaches--from static benchmark design and narrow domain coverage to narrow training frameworks and limited mechanistic understanding of abductive processes...

Moein Salimi, Shaygan Adim, Danial Parnian +3

Department of Computer Engineering, Sharif University of Technology · Department of Mathematical Sciences, Sharif University of Technology

Abductive Reasoning

Ask PDF

Aug 5, 2026

Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models

While modern large reasoning models (LRMs) excel at providing correct answers in many tasks, we provide additional evidence that they often struggle with a critical capability: knowing when to abstain from answering. We analyze this gap by comparing LRM behavior to results from an exploratory human study, revealing that human reasoning effort on unanswerable tasks isn't different from answerable tasks, whereas LRMs waste computational resources by generating longer Chains of Thought (CoTs) on unanswerable than on answerable prompts. To overcome this inefficiency, we take inspiration from a resource-rational perspective on human cognition and introduce a novel GRPO reward that encourages efficient reasoning about whether the task contains all the information needed to solve it. Fine-tuning several 4B and an 8B LRM with this reward leads to human-like abstention performance gains (+14.7% on average) while retaining answering capabilities and boosting the models' efficiency (44% shorter CoTs on average).

Polina Tsvilodub, Max Höth, Michael Franke +2

University of Tübingen · Aleph Alpha Research · Lab1141

LLM Abstention Large Reasoning Models Efficient Language Model Reasoning Question Answering RL for Language Model Reasoning

Ask PDF

Aug 4, 2026

Don't Let Me Ask for It: LLMs Show Deficiencies in Active Multi-Turn Information Acquisition for Abductive Inference

Abductive reasoning requires forming hypotheses that explain observed evidence and revising them as new evidence becomes available. While large language models (LLMs) are often evaluated on whether they solve abductive reasoning tasks correctly, less is known about how they acquire evidence, update their hypotheses, and decide when to stop. We introduce Alien Abduction game, an interactive probe for studying these behaviours under different interaction modes. The modes vary in whether evidence is provided upfront or across turns, and whether queries are selected by the model or examples are provided by the oracle. Across models, providing evidence upfront leads to higher success rates than distributing it across turns. In multi-turn settings, some models commit before using the available evidence, while others exhaust the turn budget without converging. Models also achieve higher success rates when examples are provided by the oracle than when they select their own queries, although their final hypotheses are more consistent with the evidence they selected. These findings suggest that models may form hypotheses that fit self-selected evidence without sufficiently distinguishing them from alternatives, and may struggle to validate and refine their hypotheses or determine when to stop.

Shahrukh Mohiuddin, Chalamalasetti Kranti, Sherzod Hakimov +1

Computational Linguistics, Department of Linguistics University of Potsdam, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany

LLM Evaluation Language Model Probing

Ask PDF