cs.AIAug 14, 2026
SaveCEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs
Organizations: Sharif University of Technology · University of Tehran
Abstract
Abductive reasoning, often characterized as inference to the best explanation, is central to explanation under uncertainty, from everyday sense-making and investigation to scientific discovery. Yet LLM research has mostly studied abduction through narrow, task-specific benchmarks, making it unclear whether observed gains transfer beyond the benchmark family used for training or evaluation. We ask whether RL post-training can improve abduction as a transferable reasoning capability. We introduce CEDAR-GRPO, a process-aware framework that combines final-answer correctness with abductive rewards for evidence coverage and evidence-to-explanation directionality. Four open-weight LLMs are post-trained on a controlled, domain-neutral mixture of abductive hypothesis-generation and hypothesis-selection tasks. We evaluate them on 11 unseen tasks spanning hypothesis selection, missing-fact generation, defeasible inference, long-context investigation, clinical reasoning, code debugging, and non-abductive controls. CEDAR- GRPO improves every model on every held-out task over both base models and correctness-only GRPO, with average gains of 7.4 and 2.7 points, respectively, and a maximum gain of 30.8 points. Ablations confirm that RL, abductive reward design, and task diversity each contribute to transfer. Process-level metrics further show stronger abductive behavior, including exploration of alternatives, elimination of rivals, backtracking, and uncertainty marking.
Figures & tables
Figure 1: CEDAR-GRPO grounds its answer in the observed evidence, while correctness-only GRPO relies on unsupported assumptions.
Figure 2: Overview of CEDAR-GRPO. The framework trains LLMs on abductive hypothesis generation and selection tasks using structured outputs and a composite reward for correctness, evidence coverage, and evidence–explanation directionality, then evaluates both task accuracy and reasoning behavior on held-out datasets.
Figure 3: How CEDAR-GRPO’s composite reward shapes a BalancedCOPA reasoning trace over training. All three snapshots select the correct answer, so is already maximal. At Step 48, the trace reasons from an assumed explanation back to a generic consequence and accounts for none of the three observation details returned by the coverage judge. By Step 534, it starts from the observation and links two details to the hypothesis. By Step 1,018, it explains all three details while preserving the evidence-to-explanation direction, yielding the maximum reward.
| Model | Method | ART | B-COPA | DefNLI | GoEmo. | MuSR-M | MuSR-O | MuSR-T | NeuLR | StratQA | MedQA | ML-Debug |
| Qwen3-4B | Base | |||||||||||
| Cor-GRPO | ||||||||||||
| CEDAR-GRPO | ||||||||||||
| Qwen3-8B | Base | |||||||||||
| Cor-GRPO | ||||||||||||
| CEDAR-GRPO |
Table 1: Held-out task performance across the evaluation suite. Scores represent task-native accuracy or exact-success metrics. CEDAR-GRPO is our composite-reward method; Cor-GRPO is the correctness-only baseline. The best score per backbone is bolded. Full dataset descriptions and abbreviations are provided in Appendix A .
| Metric | Description |
| Branchiness | Exploring multiple distinct candidate explanations for the same observation. |
| Backtracking | Explicitly identifying an error or flaw in the reasoning and changing direction. |
| Differential Elimination | The active refutation of alternative hypotheses given the specific context. |
| Prior Invocation | Incorporating typicality or prior probability alongside case-specific evidence. |
| Evidence Coverage | The fraction of specific observation details explicitly accounted for by the chosen hypothesis. |
| Evidence–Expl. Directionality | Demonstrating awareness that reasoning must move from given evidence toward an explanatory conclusion. |
Table 2: Overview of the process-level metrics used to evaluate intermediate reasoning behaviors.
| Held-out Metrics | Reward Metrics | ||||||
| Method | Backtracking | Branchiness | Diff. Elim. | Prior | Uncertainty | Coverage | Direction. |
| Baseline | 0.69 | 1.22 | 0.79 | 0.59 | 0.87 | 33.1% | 0.21 |
| Cor-GRPO | 0.93 | 1.16 | 0.97 | 0.53 | 0.92 | 39.1% | 0.16 |
| CEDAR-GRPO | 1.09 | 1.53 | 1.29 | 0.72 | 1.37 | 52.9% | 0.60 |
Table 3: Mean process-level metric scores averaged across ten held-out evaluation datasets for DeepSeek-R1-Distill-Qwen-7B. Evidence Coverage and Directionality were explicitly included as rewards in the CEDAR-GRPO training objective; the other five metrics are fully held-out metrics used to assess reasoning generalization. Shading indicates a relative increase ( green ) or decrease ( red ) against the Baseline. Full per-dataset breakdowns are provided in Appendix E , Table 10 .
| Model | Ablation | Method | ART | B-COPA | DefNLI | GoEmo. | MuSR-M | MuSR-O | MuSR-T | NeuLR | StratQA | MedQA | ML-Debug | Avg. vs Base |
| Qwen3-4B | Reference | Base | ||||||||||||
| Reference | Cor-GRPO | |||||||||||||
| Main | CEDAR-GRPO | |||||||||||||
| RL vs SFT | SFT | |||||||||||||
| Reward | Cor+Cov-GRPO | |||||||||||||
| Reward | Cor+Dir-GRPO |
Table 4: Unified ablation results on the two evaluated backbones, isolating the effects of supervised fine-tuning, reward composition, training stages, and generic reasoning post-training. Performance is compared against our main CEDAR-GRPO method and the Cor-GRPO baseline. All columns report task-native metrics on the same held-out evaluation datasets as Table 1 .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Relation to abduction | Samples | Training role |
| UniADILR-HGc (abductive) ( Sheng et al., 2025 ) | Stage II | 400 | Abstract logical abductive selection with distractor premises; compact signal for evaluating candidate explanations. |
| Balanced COPA (cause) ( Kavumba et al., 2020 ) | Stage II | 400 | Commonsense causal selection while reducing superficial lexical cues. |
| CauseLogics ( He and Lu, 2024 ) | Stage II-adjacent | 400 | Binary candidate–cause verification with longer reasoning chains; deeper evaluative reasoning. |
| CLIMATE-FEVER ( Diggelmann et al., 2020 ) | Stage II-adjacent | 400 | Four-way claim–evidence plausibility judgment over supports, refutes, not enough info, and disputed; broader verification-style reasoning. |
| AbductionRules ( Young et al., 2022 ) | Stage I | 400 | Abductive generation of missing explanatory facts from rule contexts. |
| Crypto ( Li et al., 2025a ) | Stage I | 200 | Cipher-rule learning (Caesar, Atbash) from examples. |
Table 5: Training and validation datasets used for GRPO fine-tuning. Each dataset contributes a fixed number of samples (400 for most datasets, 200 for Crypto and List Function), split 80/20 into train/validation; Which sums up to a total number of 2400 samples, consisting of 1920 training and 480 validation samples.
| Dataset | Relation | Samples | Evaluation role |
| Classic abductive benchmarks | |||
| ART ( NLI) ( Bhagavatula et al., 2019 ) | Stage II | 400 | Conventional Stage II abductive selection between alternative hypotheses over narrative bridges. |
| NeuLR ( Xu et al., 2023 ) | Stage I | 400 | Conventional Stage I abductive split; missing-fact generation in abstract logical contexts. |
| Stage II evidence-update evaluation | |||
| Defeasible NLI ( Rudinger et al., 2020 ) | Stage II-adjacent | 400 | Determines whether an update strengthens or weakens a hypothesis; tests evidence-based plausibility assessment. |
| Abductive interpretations in neighboring domains | |||
Table 6: Held-out evaluation datasets used for model assessment. These datasets are not used for GRPO training or validation-based selection. The Samples column reports the number of held-out examples evaluated in this study, including full selected subsets for some datasets and fixed-size samples from larger source splits for others.
| Hyperparameter | Value | Hyperparameter | Value |
| Base Models | DeepSeek-R1-Distill-Qwen-7B, Qwen3-4B, Qwen3-8B, Llama-3.1-8B-Instruct | Learning Rate | |
| Quantization | NF4 (4-bit) | LR Schedule | Cosine |
| LoRA Rank ( ) | 64 | Warmup Steps | 2 |
| LoRA Alpha ( ) | 64 | Weight Decay | 0.1 |
| GRPO Group Size ( ) | 4 | Optimizer | AdamW (Torch) |
| Max Sequence Length | 4096 | Adam | 0.9, 0.99 |
Table 7: Training and generation hyperparameters across model architectures.
| Dataset | Verifier |
| BalancedCOPA | Parse an integer and compare it with the one-based gold label. |
| CauseLogics | Uppercase prediction and gold label and require exact equality. |
| ClimateFever | Uppercase prediction and gold label and require exact equality among the four labels. |
| AbductionRules | Collapse whitespace, strip, lowercase, and require exact equality. |
| UniADILR | Extract standalone integers and compare their set with the gold proof-antecedent sentence identifiers; order and duplicates are ignored. |
| Crypto | Execute transform(s) on every held-out test input and require exact string outputs for all tests. |
Table 8: Dataset-specific correctness verifiers used during training.
Table 9: Screening candidate process-level metrics as possible reward terms. Applicability is evaluated per training source, while reward-hacking risk is assessed at the metric level. This screening motivates our reward-composition choice: Evidence Coverage and Evidence–Explanation Directionality are selected because they are broadly applicable across datasets and comparatively resistant to superficial optimization.
| Datasets | |||||||||||
| ART | B-COPA | GoEmo. | MedQA | DefNLI | MuSR-M | MuSR-O | MuSR-T | NeuLR | StratQA | ||
| Condition | Metric | ||||||||||
| Cor-GRPO | Backtracking | 0.5 | 0.9 | 0.2 | 0.8 | 1.5 | 0.8 | 1.6 | 0.6 | 1.5 | 0.9 |
| Branchiness | 1.2 | 0.4 | 0.9 | 1.5 | 1.6 | 1.2 | 1.3 | 1.5 | 1.4 | 0.6 | |
| Coverage (%) | 43.0 | 41.0 | 23.0 | 34.0 | 46.0 | 49.0 | 39.0 | 44.0 | 35.0 | 37.0 | |
| Diff. Elim. | 1.0 | 1.0 | 0.6 | 0.7 | 1.0 | 0.9 | 1.1 | 0.7 | 1.5 | 1.2 | |
Table 10: Process-level metric scores for DeepSeek-R1-Distill-Qwen-7B. Values are bolded if they are the maximum for that specific metric and dataset across all three conditions.
| Dataset | Unc. Markers | Branch. | Back. | Diff. Elim. | Prior | Obs. Cov. | Direct. |
| ART | Focus on uncertainty while reasoning about which hypothesis better explains observations. | Do not count simple option selection; count only internal exploration. | Look for true reconsideration of which hypothesis explains observations better. | Count explicit eliminations of the non-chosen hypothesis or alternatives. | Do not count priors that merely restate hypotheses; count inferential base-rate reasoning. | Treat ‘Observation 1’ and ‘Observation 2’ as complete; extract all details. | Check reasoning starts from observations and seeks hypothesis, not back-fitting. |
| B-COPA | Focus on uncertainty in causal reasoning and option comparison. | Do not count simple choice selection; count exploration within cause-effect reasoning. | Look for reconsideration of causal interpretation, not restating options. | Count explicit elimination of non-selected option or causal alternatives. | Priors about typical everyday cause-effect relationships. | Treat ‘Cause:’ as observation; extract relevant details. | ‘Cause’ is evidence; model must evaluate which option is most plausible effect. |
| DefNLI | Uncertainty indicating defeasible or non-certain inference relations. | Exploration of stronger vs. weaker inferences and defeasible conclusions. | Reconsideration when inference appears defeated or less robust. | Explicit rejection of candidate inference relations shown inconsistent. | Priors about typical premise-hypothesis relations and defaults. | Treat ‘Premise’, ‘Hypothesis’, ‘Update’ together as complete observation. | Reasoning evaluates logical impact of Update from given text. |
| GoEmo. | Uncertainty in label selection; avoid counting task-domain terms. | Count only if model explores multiple emotion labels. | Count only genuine revisions in emotion-label selection. | Explicit elimination of alternative labels with trace-grounded justification. | Priors about emotions typically associated with contexts. | Treat ‘Text’ as complete observation; extract salient spans. | Reasoning extracts cues before selecting emotion label. |
Table 11: Dataset-specific notes for all process-level metrics injected into the judge prompt. Each cell gives the exact text appended to the shared system prompt for that dataset.
| Dataset | Unc. Markers | Branch. | Back. | Diff. Elim. | Prior | Obs. Cov. | Direct. |
| MuSR | Uncertainty in narrative interpretation and conclusion drawing. | Exploration of different narrative interpretations or solutions. | Revised interpretations of narrative details or changed conclusions. | Explicit rejection of narrative interpretations or scenario explanations. | Priors about typical narrative behavior, motives, patterns. | Treat ‘Context’ and ‘Problem’ as complete; extract narrative details (actors, timing, locations). | Reasoning builds explanations from details, not assumes conclusion and back-fits. |
| MedQA | Extract markers only from model’s own reasoning, not probabilistic symptoms. | Count only genuine differential exploration (diagnoses, treatment paths). | Corrections in diagnostic/treatment reasoning or missed details. | Each answer choice explicitly ruled out with clinical rationale. | Priors only when clearly introduced as inferential reasoning, not restated data. | Treat ‘Problem’ as observation; extract all clinical details. | Reasoning starts from clinical evidence and moves toward diagnosis/treatment. |
| NeuLR | Uncertainty while weighing competing abductive explanations. | Do not count superficial option comparison; genuine exploration. | True reconsideration of which explanation best fits observations. | Explicit elimination moves using concrete mismatch evidence. | Priors justifying why one hypothesis more plausible. | Treat ‘Logical Rules and Known Facts’ and ‘Target Conclusion’ as complete. | Reasoning uses Rules/Facts to find Missing Fact, not work backwards. |
| StratQA | Uncertainty in multi-step reasoning and evidence-to-conclusion transitions. | Exploration of multiple reasoning paths or inference chains. | Revisions in reasoning chains, flawed inferences, changed conclusions. | Explicit rejection of competing YES/NO reasoning lines. | World-knowledge priors and typicality assumptions. | Treat ‘Question’ and ‘Evidence’ as complete; extract relevant facts. | Reasoning builds upon facts toward YES/NO answer, not cherry-picks post-hoc. |
Table 11: Dataset-specific notes for all process-level metrics injected into the judge prompt. Each cell gives the exact text appended to the shared system prompt for that dataset. (continued)
| Setting | SFT configuration |
| Data | 1,920 training and 480 validation examples; sequence packing disabled |
| Quantization / LoRA | 4-bit NF4; rank ; ; Unsloth gradient checkpointing |
| LoRA targets | q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj |
| Sequence length | 4,096 tokens |
| Optimizer | AdamW (Torch), learning rate , , weight decay |
| Schedule / clipping | Cosine decay, 2 warmup steps, maximum gradient norm |
Table 12: Optimization and model-selection setup for the SFT ablation. Held-out evaluation tasks are not used for checkpoint selection.
| Dataset | Reasoning profile | Samples | Role in the ablation mixture |
| GSM8K ( Cobbe et al., 2021 ) | Arithmetic, multi-step reasoning | 400 | Introduces numerical and procedural reasoning through grade-school math word problems. Final supervision is based on the extracted numerical answer. |
| FOLIO ( Han et al., 2024 ) | Formal logical inference | 400 | Provides premise–conclusion judgments grounded in first-order logical structure, adding a strongly symbolic component to the mixture. |
| CommonsenseQA ( Talmor et al., 2019 ) | Commonsense multiple choice | 400 | Supplies broad everyday reasoning questions in a fixed five-choice format, complementing the more formal logical tasks. |
| VitaminC ( Schuster et al., 2021 ) | Fact verification under contrastive evidence | 400 | Adds evidence-sensitive classification in which small textual changes can reverse the correct label, making it useful for testing fine-grained judgment. |
| MMLU ( Hendrycks et al., 2021 ) | Academic multiple choice | 400 | Contributes subject-diverse question answering from a reasoning-oriented subset of academic domains. |
| BIG-Bench ( Srivastava et al., 2022 ) | Heterogeneous multiple-choice reasoning | 400 | Adds task diversity through four reasoning-focused multiple-choice tasks spanning deduction, state tracking, date reasoning, and tabular reasoning. |
Table 13: Composition of the general-reasoning ablation mixture. Each source contributes 400 instances, producing a balanced 2,400-example corpus with an 80/20 train/validation split.