cs.CLOct 21, 2026

Accept or Repair: Preserving Passage-Supported Answers in Contrastive Decoding

Authors: Jungseob Lee, Seungyoon Lee, Hyeonseok Moon, Seongtae Hong, Suhyune Son, Sugyeong Eo, Minhyuk Kim, Chanjun Park, 

Organizations: Korea University · Sookmyung Women’s University · Yonsei University Mirae Campus · Soongsil University · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models often answer from knowledge learned during training, even when a provided passage contradicts that knowledge. To address this problem, contrastive decoding contrasts the predictions under the full and a modified prompt at every token, but it also alters correct answers that the passage supports. We show that checking answers against the passage removes this trade-off. Based on this observation, we propose Accept-or-Repair (AoR), a training-free procedure. AoR keeps a sampled answer if the passage supports it. Otherwise, it tries up to two replacements with contrastive decoding, returning one as soon as it passes this check. It rejects replacements that repeat the answer generated without the passage when the passage favors another answer. Our experiments show that AoR raises the score over sampling by 12.5 points on counterfactual passages and 2.3 on ordinary ones, surpassing eight contrastive and prompting baselines on both, including when each receives the same rejection rule. Replacing only initial answers that the passage does not support improves the average score of every decoder. These results suggest that models can follow passages that contradict their knowledge while preserving accuracy on passages that agree with it. Code will be released at https://github.com/js-lee-AI/AoR.

Figures & tables

Figure 1: Gain over plain sampling on counterfactual and ordinary passages, at the setting that each method chose on the development split. Test split, averaged over six models.
Figure 2: Two test questions. (a) CAD contrasts every answer. (b) AoR accepts supported answers (Supp). When the passage favors another answer, the memory veto rejects repairs that repeat mm , generated without the passage. Gray steps are skipped.
 CounterfactualOrdinary  
MethodNQ-SwapFaithEvalAvg.NQ-Orig.SQuADHotpotQAAvg.MeanCost
Sampling56.963.660.376.084.563.574.767.51.00
CAD ( Shi et al., 2024 )62.473.768.1 (+7.8)72.584.660.872.6 ( −- 2.1)70.31.18
AdaCAD ( Wang et al., 2025a )61.465.563.4 (+3.2)75.685.563.274.8 (+0.1)69.11.18
COIECD ( Yuan et al., 2024 )62.273.768.0 (+7.7)72.784.561.172.8 ( −- 1.9)70.41.18
NWCAD ( Tao and Agrawal, 2026 )57.363.860.6 (+0.3)76.084.563.774.8 (+0.1)67.71.18
Table 1: Test scores averaged over six models: F1 for free-form tasks, accuracy for FaithEval. Mean averages both groups. Parentheses show differences from sampling, colored when their 95% intervals exclude zero. Cost is the token count relative to sampling.
Figure 3: (a) How AoR ends on each task over six models: the plain answer is accepted, a repair is supported, or no draw is supported. (b) Mean on Qwen3 models of five sizes.
 CounterfactualOrdinaryRepaired part
MethodSS¬S\neg SFixed (%)SS¬S\neg SBroken (%)Lowered (%)CFOrd.
Sampling–––––––37.235.5
CAD−- 0.2+8.022.0−- 3.4+1.312.212.952.743.7
AdaCAD+0.3+2.99.4−- 0.9+0.93.95.344.741.1
COIECD−- 0.2+7.921.6−- 3.3+1.411.912.652.543.9
NWCAD−- 0.1+0.41.2−- 0.1+0.20.91.438.236.6
Table 2: Where the gains come from. SS , ¬S\neg S : share-weighted difference to sampling on supported and unsupported plain answers; Fixed, Broken: wrong plain answers made right, correct ones made wrong; Lowered: F1 below the plain answer; Repaired part: scores where AoR repairs.
Wrapped decoderCFOrd.Mean
CAD68.376.0 (+3.4)72.2 (+1.8)
AdaCAD63.275.6 (+0.9)69.4 (+0.3)
COIECD68.276.1 (+3.3)72.1 (+1.7)
NWCAD60.774.9 (+0.1)67.8 (+0.1)
RPCD66.876.5 (+2.1)71.6 (+1.4)
ID66.276.7 (+2.1)71.5 (+1.1)
Table 3: Each decoder DD wrapped by AoR as its only repair, without the veto. On ordinary passages and in Mean, parentheses give the difference to DD alone, colored when its 95% interval excludes zero.
Figure 4: F1 change on the supported plain answers of the ordinary tasks, binned by the probability that the model gives the same answer without the passage.
Figure 5: Counterfactual and ordinary averages of CAD, ID and RPCD at four strengths, alone and wrapped by AoR as the only repair. Dotted lines mark sampling.
VariantCFOrd.MeanCost
AoR72.877.074.91.71
Components
Repair every answer+0.7−- 1.3−- 0.32.71
No memory veto−- 3.20.0−- 1.61.63
Any occurring option supported−- 0.90.0−- 0.51.65
Repairs
Table 4: Variants of AoR and majority votes. Score differences are relative to AoR; color indicates a 95% interval excluding zero. Cost is the token count relative to sampling.
GateNQ-SwapOrd.Mean
CAD on every answer+5.5−- 2.1+1.7
Same as closed-book answer+1.5−- 0.8+0.3
Answer probability+4.2+0.2+2.2
Probability gain from passage+4.8−- 0.8+2.0
Self-check+3.0−- 0.3+1.4
Entailment model+3.7−- 0.2+1.7
Table 5: Gain over sampling when CAD replaces selected answers on free-form tasks. Four ranked gates match the support test’s selection count. Mean averages NQ-Swap and Ord. Color indicates 95% intervals excluding zero.
 Main tasksHeld-out dataFurther modelsOther settingsLLM judge
MethodNo FaithEvalWith vetoCFOrd.OLMoYi32B70BGreedyDistractorsQwenGemma
Sampling65.870.263.183.768.667.972.368.966.663.3––
CAD67.571.766.681.960.164.274.468.967.363.6−- 1.5+1.1
AdaCAD68.171.665.783.768.567.574.071.667.664.9+0.6+1.6
COIECD67.571.866.782.060.064.674.369.666.563.5−- 1.2+1.3
NWCAD66.068.663.483.768.767.872.369.166.663.50.0+0.1
Table 6: Beyond the main setting: Mean, or group average for held-out data. No FaithEval: NQ-Swap, ordinary average; With veto: veto for all; Greedy: four models, retuned, first row greedy; Distractors: passages of four other questions; Judge: free-form Mean gain over sampling.
Task, modelPassage, excerptQuestionSamplingCADAoR
NQ-Orig. Qwen3-8BUConn owns the two longest winning streaks …The longest streak, 111 straight wins, started …how many games in a row have the uconn women’s basketball team won111 ✓two ✗111 ✓ accepted
NQ-Swap Qwen3-8BIn 1971, after the Irish War of Independence …most of Ireland seceded from the United Kingdom …when did ireland gain its independence from england1922 ✗1971 ✓1971 ✓ CAD repair
NQ-Swap Llama-3.1-8BNeal Dahlen as Hiccup Horrendous Haddock III, the son of the Viking chief Stoick the Vast and Valka.who voices hiccup in how to train your dragon 2Jay Baruchel ✗Jay Baruchel ✗Neal Dahlen ✓ ID repair
FaithEval Qwen3-14B…recent studies have shown that minerals hold a pivotal role in the fossilization process. …Which of the following would most likely form a fossil? A. a rock, B. a crystal, C. a mineral, D. a boneD ✗D ✗C ✓ D vetoed
Table 7: Test answers from three models, with passage excerpts, illustrating acceptance, CAD and ID repairs, and the memory veto. A check marks a correct answer; labels below AoR’s answers indicate the action taken.
Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

PromptTextUsed by
StandardAnswer the question using the passage. Reply with only the answer, a few words at most. + passage + questionsampling, AoR, every contrastive decoder except Strict CAD
Closed-bookAnswer the question. Reply with only the answer, a few words at most. + questionthe memory veto of AoR, contrast of CAD, AdaCAD, COIECD, NWCAD and Strict CAD
Other passagestandard instruction + passage of another example + questioncontrast of RPCD
Opposite directiveAnswer the question using your own knowledge and ignore the passage. Reply with only the answer, a few words at most. + passage + questioncontrast of ID
StrictAnswer the question using only the passage, even if the passage contradicts what you know. Reply with only the answer, a few words at most. + passage + questionStrict prompt, Strict CAD
CopyAnswer the question using the passage. Reply with only the answer, copied from the passage in as few words as possible. + passage + questioncopy prompt, alone and with CAD
Appendix
Table 8: Prompts of the free-form tasks. A plus sign joins the parts of a prompt. Shading marks the two prompts that AoR samples from. Its repairs use the prompts of CAD and ID.
TaskMetricDev.TestUsed
Counterfactual passages
NQ-SwapF1200800724
FaithEvalaccuracy200800800
Ordinary passages
NQ-OriginalF1200800724
SQuADF1200800724
Appendix
Table 9: Counts of development and test examples. Used excludes test examples whose question, or for SQuAD whose passage, occurs in the development split. ConFiQA rows represent each subset.
ModelTemperatureTop- ppTop- kk
Qwen3 (all sizes)0.70.820
Llama-3.1 (8B, 70B)0.60.9off
Mistral-7B0.70.9off
OLMo-2-13B0.70.9off
Yi-1.5-9B0.70.9off
Appendix
Table 10: Sampling settings of every model, shared by all methods.
MethodParameterSettings triedScores
CADstrength0.25, 0.5, 1, 21.81, 2.67, 2.66 , −- 0.49
AdaCADstrength0.5, 1, 2, 41.26, 1.67 , 1.54, 0.35
RPCDstrength0.25, 0.5, 1, 21.85, 2.75, 3.11 , 1.00
IDstrength0.25, 0.5, 1, 2, 4, 80.92, 1.73, 2.82, 3.10 , 0.76, −- 10.89
NWCADdivergence threshold0.05, 0.1, 0.2, 0.3, 0.50.17, 0.23 , 0.20, −- 0.16, −- 0.76
COIECDstrength, constraint scale(0.5, 0.25), (1, 0.25), (2, 0.25), (1, 0.1), (1, 0.5)−- 12.55, 1.19, 2.60 , 1.54, 0.51
Appendix
Table 11: Settings tried on the development split and their scores in the same order. Scores average the counterfactual and ordinary gains over sampling, in points. Bold marks the chosen setting.
MethodSettingsCounterfactualOrdinary
CAD0.25, 0.5, 1, 23.7, 5.9, 7.8 , 8.00.2, −- 0.1, −- 2.1 , −- 7.7
AdaCAD0.5, 1, 2, 42.3, 3.2 , 3.7, 3.20.2, 0.1 , −- 0.5, −- 2.5
RPCD0.25, 0.5, 1, 22.8, 4.5, 5.8 , 5.70.8, 0.7, −- 0.3 , −- 4.2
ID0.25, 0.5, 1, 2, 4, 81.5, 2.7, 4.5, 5.8 , 4.2, −- 7.70.4, 0.5, 0.8, 0.0 , −- 4.3, −- 15.3
NWCAD0.05, 0.1, 0.2, 0.3, 0.50.2, 0.3 , 0.1, −- 0.1, −- 0.70.0, 0.1 , −- 0.3, −- 0.6, −- 1.2
COIECD(0.5, 0.25), (1, 0.25), (2, 0.25), (1, 0.1), (1, 0.5)−- 22.8, 1.9, 7.7 , 2.7, 1.0−- 4.3, 0.2, −- 1.9 , 0.1, 0.1
Appendix
Table 12: Gain over sampling at each setting of Table 11 , on the counterfactual and the ordinary average of the test split. Bold marks the chosen setting.
 Closed-book probability p(a∣q)p(a\mid q)
 <0.01<0.010.010.01 – 0.10.10.10.1 – 0.50.5≥0.5\geq 0.5
Share (%)72.78.87.211.3
Plain F180.585.186.389.3
F1 change
CAD−- 3.0−- 5.4−- 7.1−- 7.4
AdaCAD−- 1.0−- 1.3−- 1.2−- 0.7
Appendix
Table 13: Supported plain answers on ordinary tasks, grouped by their closed-book probability p(a∣q)p(a\mid q) . Rows give the share, plain F1 and decoder F1 changes. Test split, six models.
VariantNQ-SwapFaithEvalNQ-Orig.SQuADHotpotQACFOrd.Mean
Sampling56.963.676.084.563.560.374.767.5
AoR65.280.478.287.565.272.877.074.9
Repair every answer66.680.476.586.863.873.575.774.6
No memory veto65.274.078.287.565.269.677.073.3
CAD only62.879.576.986.764.571.276.073.6
ID only64.276.578.186.965.270.476.873.6
Appendix
Table 14: Variants of AoR on each task, test split, six models. The plain answer as fallback returns the plain answer, instead of the first answer not rejected by the veto, when no draw is supported.
DecoderWithout vetoWith veto
Veto score as a solver61.4–
Sampling63.674.5
ID67.876.1
CAD73.779.2
AoR74.080.4
Appendix
Table 15: FaithEval accuracy of each decoder without and with the memory veto, which masks the letter of a vetoed closed-book answer. The first row picks the option with the highest veto score. Test split, six models.
MethodCFOrd.Mean
Sampling + veto65.774.770.2
CAD + veto70.872.671.7
AdaCAD + veto68.474.871.6
COIECD + veto70.872.871.8
NWCAD + veto62.474.868.6
RPCD + veto69.974.472.2
Appendix
Table 16: Each baseline with the memory veto of AoR, which on FaithEval masks the letter of a vetoed closed-book answer. Test split, six models.
MethodNQ-SwapOrd.MeanCost
Sampling56.974.765.81.00
Span decoding61.476.368.81.00
Copy prompt59.673.466.51.02
Copy prompt, CAD59.766.263.01.20
AoR with one repair
Feedback58.574.966.71.28
Appendix
Table 17: Span and copy methods and AoR variants on free-form tasks. Ord. averages NQ-Original, SQuAD and HotpotQA, and Mean averages NQ-Swap and Ord. Cost counts processed tokens relative to sampling. Test split, six models.
MethodNQ-SwapNQ-Orig.Mean
Sampling53.872.763.3
CAD58.868.563.6
AdaCAD57.971.964.9
COIECD58.468.663.5
NWCAD54.372.763.5
RPCD59.172.665.8
Appendix
Table 18: NQ-Swap and NQ-Original with the passage of each question placed among four passages of other questions, at a random position. Test split, six models.
MethodCFOrd.Mean
Sampling64.276.670.4
CAD72.875.574.1
AdaCAD67.476.872.1
COIECD72.775.674.1
NWCAD64.676.670.6
RPCD70.776.573.6
Appendix
Table 19: Scores when a free-form answer counts as correct if it contains a gold answer as whole words, which does not penalize longer answers. FaithEval keeps accuracy. Test split, six models.
MethodNQ-Orig.SQuADHotpotQAOrd.NQ-Swap
CAD−- 4.6−- 1.8−- 4.9−- 3.8+0.9
AdaCAD−- 0.60.0−- 1.2−- 0.6+1.8
COIECD−- 4.2−- 1.8−- 4.5−- 3.5+1.0
NWCAD−- 0.1−- 0.20.0−- 0.10.0
RPCD−- 2.3−- 1.7−- 3.0−- 2.3+0.8
ID−- 3.4−- 1.0−- 1.0−- 1.8+2.8
Appendix
Table 20: Accuracy differences from sampling on free-form answers, judged by Qwen2.5-32B-Instruct. Test split, six models.
MethodNQ-SwapNQ-Orig.SQuADOrd.Mean
Sampling64.986.390.988.676.7
AoR (ours)73.386.791.989.381.3
Difference+8.4 ∗+0.4+1.0+0.7 ∗+4.6 ∗
Appendix
Table 21: Answers given as one sentence, correct when they contain a gold answer. Ord. averages NQ-Original and SQuAD, and Mean averages NQ-Swap and Ord. ∗ : the 95% interval of the difference excludes zero. Test split, six models.
    Mean
MethodCFOrd.MeanLlamaMistralOLMoYi
Greedy58.474.866.670.258.069.468.8
CAD63.371.467.374.162.167.765.4
AdaCAD61.773.567.672.361.169.068.1
COIECD60.872.266.572.560.567.865.3
NWCAD58.574.766.670.558.169.368.3
Appendix
Table 22: Greedy decoding of every method on Llama-3.1-8B, Mistral-7B, OLMo-2-13B and Yi-1.5-9B, test split, with every setting chosen again under greedy decoding on the development split. The memory veto of AoR uses the sampled closed-book answer of the main runs.
MethodQwen3-1.7BQwen3-4BQwen3-8BQwen3-14BLlama-3.1-8BMistral-7B
Sampling65.168.172.272.269.557.7
CAD65.370.676.275.572.362.3
AdaCAD65.969.473.373.271.960.9
COIECD65.570.676.175.572.462.2
NWCAD65.468.372.372.269.857.9
RPCD65.870.474.374.671.265.1
Appendix
Table 23: Mean of the counterfactual and ordinary averages for each model in the main comparison. Test split.
More in this topic

Evidence-Grounded QA

See all 42 papers

Explore similar work

CardsList

  1. Detecting and Repairing Hallucinations in Retrieval-Augmented Generation

    Aug 29, 2026Sai Krishna Reddy Mulakkayala, Niki van Stein, Aske PlaatHallucination in RAGLLM Hallucination Mitigation

    LIACS, Leiden University, Leiden, The Netherlands

  2. Shaping the Document, Not the Model: Optimized Suffixes for Context Following

    Oct 19, 2026Gia-Bao Dinh Ho, Sekh Mainul Islam, Masahiro Kaneko +2Knowledge Conflicts in Language ModelsAutomatic Prompt Optimization

    Mohamed bin Zayed University of Artificial Intelligence · University of Copenhagen

  3. Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding

    Oct 21, 2026Shin'ya YamaguchiCoT ReasoningFaithfulness of Language Model Reasoning

    NTT