Don't Pick a Winner: Knowledge Conflicts Need a Prior, Not a Rule

Minh Pham Van; Cong Tran

cs.AIOct 26, 2026

Don't Pick a Winner: Knowledge Conflicts Need a Prior, Not a Rule

Authors: Minh Pham Van, Cong Tran

Organizations: Posts and Telecommunications Institute of Technology, Hanoi, Vietnam

Abstract

A retrieval-augmented language model answers from two sources, the retrieved context and its own knowledge, and the two disagree whenever the context is incomplete or wrong. Existing methods settle the disagreement with a rule fixed at design time: follow the context, amplify it in proportion to how far it moves the model, or trust the model's memory. Each rule is confidently wrong in some deployment. In a decision-theoretic model of two sources, the score of any fixed rule is linear in the share of reliable contexts while the best attainable score is convex, so every fixed rule is beaten wherever it is not exactly optimal; and a symmetric divergence between the two sources, the signal adaptive decoding uses, carries no information about which one is right. Resolving a conflict therefore needs two inputs that no existing method keeps: evidence about each source separately, and a prior on how often the context is right. We test these claims on GRACE, a benchmark that intervenes on the knowledge-graph context of a fixed question, so that what the model knows, what the context says and what is true are measured separately. There, every baseline from four families is strong on right contexts or on wrong ones, never on both. We then build the two inputs: the model states its knowledge as notes, a small policy voices each note with a calibrated confidence, a reader answers in two voices, and a selector commits or abstains under a prior $π$ applied as an exact prior shift. With $π$ set to the true share of right contexts, the system is above the best baseline at every mixture of GRACE contexts, by up to 26 points. Its overall score is 45.3 and 35.0 (Qwen3.5-35B-A3B, Llama-3.1-8B), against 29.4 and 27.5 for the best baseline. On preregistered ClashEval it beats same-backbone CK-PLUG and TCR by 12 and 26 points. Where the model lacks the knowledge, the gains vanish, as the theory predicts.

Figures & tables

Figure 1: (a) GRACE dev, Qwen: deployments with a share

f

of right contexts (clean and wrong_branch versions of the same questions, mixed). Every existing method is a line in

f

(grey; dashed: the best at each

f

); ours with

\pi=f

(orange) is above all of them. (b) A wrong context (real outputs): the plain reader repeats it; our answer states both sources and commits to the right one.

Figure 2: C $-$ W on GRACE development data by context state, for every method and both backbones. Every method that reads the context scores between $-36$ and $-74$ on wrong_branch contexts; methods that ignore it lose the clean column. Ours ( Section 6 ) is the only row without a strongly negative cell. CK-PLUG, TCR, CAD and AdaCAD are run on Llama only.

Figure 3: The divergence detects a wrong context but not which source is right. (a, b) AdaCAD’s weight

\alpha

at the first answer token: higher when the context of the same question is wrong (filled), no higher when the model’s own answer is right (open). (c) Where the model is right and the context wrong, amplifying the context turns abstentions into wrong answers. Per-state values in Section D.2 .

Figure 4: Method overview on one real GRACE dev record with a wrong context (Qwen). (a) Six elicitation runs yield notes

n_{j}

with support

k_{j}

; a frozen encoder and the learned two-layer policy give

p_{j},u_{j}

and the spoken confidence

c_{j}^{*}

( Equation 3 ). (b) The reader receives the question, the unedited context, all voiced notes and the plain reader’s reading, and answers in two voices that keep

a_{c}

and

a_{m}

. The selector scores four candidates, shifts the context-sourced ones from

\pi_{0}

to the stated

\pi

( Equation 4 ), and commits if the best exceeds

1/2

	Qwen3.5-35B-A3B		Llama-3.1-8B
Method	C $-$ W	$\Delta$	C $-$ W	$\Delta$
Ours	45.3		35.0
USC ( Chen et al., 2024b )	29.4	$+15.9$ [12.6, 19.1]	26.5	$+8.5$ [4.8, 12.3]
plain reader	22.6	$+22.7$ [19.8, 25.7]	27.5	$+7.5$ [4.4, 10.6]
InstructRAG-ICL ( Wei et al., 2025 )	23.7	$+21.6$ [18.4, 24.8]	20.1	$+14.9$ [11.6, 18.2]
Astute RAG ( Wang et al., 2025a )	21.7	$+24.0$ [19.8, 28.4]	10.1	$+24.9$ [20.4, 29.5]

Table 1: GRACE development data, C

-

\pi=0.25

\Delta

: ours minus method, 95% paired interval. –: Llama only (official code or logit access; Appendix C ).

Figure 5: Testing the theory. (a, b) Deployments with a share

f

of right contexts ( GRACE dev): every baseline is a line; dashed, the best at each

f

; orange, ours with

\pi=f

(stars: significantly above). (c) Calibration error against the prior used: lowest at the true share on GRACE (0.25) and ClashEval (0.114), but high on ClashEval, where label shift fails.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

	Definition	Measured by
Acc1 (knows)	correct without any context	closed-book arm
Acc2 (reads)	answers what the context says, right or wrong	context-only arm, scored against the context's answer
Acc3 (is right)	answers the real-world answer	every method; the final objective

Appendix

Table 7

Split	Size	Use
Pool	1,000 LC-QuAD-train questions × 4 states	training the note policy and the selector
Dev	3 disjoint slices of 225 questions × 4 states = 2,700 records	development and all GRACE tables
GRACE-QT	1,000 LC-QuAD-train questions of types absent from the pool (rank, qualified statement, boolean, string); 500 train / 500 dev; real Wikidata 1-hop context, no perturbation	question-type generalisation

Appendix

Table 8

Method	Uses model knowledge explicitly	Knowledge tied to individual context facts	Calibrated confidence	Both sources visible in the answer	Abstains	Interpretable control
FaithfulRAG, TruthfulRAG	partly (to find conflicts)	yes	no	no	no	no
Self-Route	only when context is silent	no	no	no	no	no
CK-PLUG, CAD, AdaCAD	implicitly (logits)	no	no	no	no	α (fixed, entropy- or divergence-based)
TCR, Knowledgeable-R1	implicitly (trained)	no	no	no	no	no
InstructRAG, USC	implicitly	no	no	no	no	no
GenRead, Astute RAG	yes	no	self-reported	no	no	no

Appendix

Table 4: Design properties of prior work and of our method.

Baseline	Venue	Source used
FaithfulRAG	ACL 2025	official code
TruthfulRAG	AAAI 2026	official code
InstructRAG-ICL	ICLR 2025	official code
GenRead	ICLR 2023	official code
TCR	AAAI 2026	official code, retrained with the authors' recipe (three fixes were needed to run it)
CK-PLUG	ICLR 2026	official code, adaptive α (one line changed: a hard-coded GPU index)

Appendix

Table 10

Baseline	Code	What is unchanged	What had to be adapted
FaithfulRAG	official, commit 9181b11	the authors' pipeline ( get_self_facts , get_topk_chunks , get_predictions , normal CoT), no line changed	the OpenAI-compatible backend points at our server
InstructRAG-ICL	official, commit 8147bbc	the authors' prompt builder and template; rationales generated on 200 GRACE-pool records; two random in-context examples as in the paper	chat tokens of the template changed to the backbone's
GenRead	official, commit 49828ba	the original prompts and add_prompt ; step 1 at most 300 tokens, step 2 at most 10, greedy	called through the chat template instead of the completion API. GenRead does not read the retrieved context
TruthfulRAG	official	the steps of TruthfulRAG.run() ; MiniLM on CPU as in the README	backend points at our server
CK-PLUG	official	eval_rag.py , mode ck , adaptive α, the "Q: … A:" prompt	one line: a hard-coded GPU index; the first output line is taken as the answer
CAD, AdaCAD	official arithmetic, commits 1281d7f (CAD) and 57508c3 (AdaCAD)	get_jsd and filter_logits_top_p taken verbatim from the AdaCAD file; per-step combination as in the code (CAD: weights 2 and −1, the authors' NQ-Swap setting; AdaCAD: α = JSD of the two distributions); greedy, 32 new tokens, the AdaCAD question-answering prompts, first output line as the answer (as in eval_qa.py )	the official code holds one model copy per input in two processes on two GPUs and adds logits through torch.distributed ; we compute both passes in one process with one copy (the arithmetic is the same)

Appendix

Table 11

	correct	abstain	wrong
plain reader	12.0	68.2	19.9
CAD (fixed $\alpha=1$ )	11.2	40.8	47.9
AdaCAD ( $\alpha$ = divergence)	12.7	17.6	69.7
ours ( Section 6 )	32.2	51.3	16.5

Appendix

Table 7: Records of the wrong_branch state whose question the model answers correctly on its own (Llama,

n=267

): the model is right and the context is wrong. Percent of records.

Figure 6: What AdaCAD’s weight measures (Llama-3.1-8B, GRACE development data, 95% intervals). (a)

\alpha

is highest when the context is wrong. (b) Within wrong contexts it is no higher when the model is right. (c) On wrong contexts that the model can answer, amplification converts abstentions into wrong answers.

Stance	Meaning
confirm	context fact Fi matches what the model knows
contradict	context fact Fi is wrong according to the model; the object is the value the model believes
add	additional knowledge about a related entity
bridge	knowledge that connects the question to the answer

Appendix

Table 14

$f=\pi$	0.1	0.25	0.5	0.75	0.9	0.99
Qwen: best baseline	GenRead	Astute RAG	Astute RAG	USC	USC	USC
ours $-$ best	$+25.7$	$+25.8$	$+21.3$	$+11.4$	$+3.5$	$+1.5$
	[19.3, 32.1]	[20.2, 31.5]	[16.3, 26.4]	[7.7, 15.1]	[ $-0.4$ , 7.3]	[ $-2.5$ , 5.5]
Llama: best baseline	Astute RAG	plain reader	plain reader	USC	AdaCAD	AdaCAD
ours $-$ best	$+10.7$	$+11.7$	$+4.4$	$+4.1$	$+2.1$	$+0.7$
	[4.9, 16.9]	[7.2, 16.2]	[0.4, 8.4]	[ $-0.6$ , 8.8]	[ $-1.8$ , 6.1]	[ $-3.6$ , 4.9]

Appendix

Table 9: Ours with

\pi=f

minus the best baseline at each share

f

of right contexts ( GRACE development data, 95% intervals). The best baseline is chosen after seeing the scores.

	Qwen3.5-35B-A3B					Llama-3.1-8B
π	All	clean	broken	wrong	comp.	All	clean	broken	wrong	comp.
0.1	44.8 (56.9 / 12.0)	61.2	43.0	20.4	54.7	33.3 (45.4 / 12.0)	53.5	34.7	−0.6	45.8
0.25	45.3 (58.4 / 13.1)	64.3	44.1	17.9	55.0	35.0 (48.5 / 13.6)	57.5	37.6	−4.1	48.9
0.5	43.4 (59.7 / 16.3)	70.2	44.9	4.0	54.4	34.1 (54.0 / 19.8)	64.9	39.1	−19.6	52.1
0.75	38.8 (59.2 / 20.4)	72.6	45.6	−14.5	51.6	30.0 (55.1 / 25.1)	66.8	38.2	−34.4	49.5
0.9	30.7 (55.9 / 25.2)	73.3	45.9	−37.9	41.5	26.4 (54.6 / 28.3)	67.4	38.2	−45.6	45.5

Appendix

Table 16

Figure 7: Clean against wrong_branch score on GRACE dev (protocol v3). Each marker is one method; the line is our system as the context-trust prior π is swept from 0.1 to 0.99 (star: the trained setting, 0.25). Baselines of every family lie in the lower right (good on clean contexts, confidently wrong on wrong_branch ones) or on the left (safe but uninformative). Our curve covers the region above them and reaches the context-following corner at π = 0.99. Inset: the dense region of the Qwen panel, enlarged.

Figure 8: Change in C−W when one component is removed (GRACE dev, protocol v3, π = 0.25), with 95% question-clustered bootstrap intervals. Every interval excludes zero on both backbones.

Figure 9: Accuracy of the knowledge part of the two-voiced answer by the hedge the reader wrote (GRACE dev; bars show 95% binomial intervals; the number above a bar is the share of that backbone's answers). Accuracy falls with the stated certainty on both backbones. Right of the dashed line: answers in which the reader wrote no hedge word.

Figure 10: Records on which both the closed-book arm and the context-only arm fail (GRACE dev; percentages are of all records). Baselines recover a similar share of these records but are wrong on about twice as many. Ours is the only method on or above the line where correct equals wrong.

Figure 11: Wrong answers on GRACE dev, split by whether the answer shown to the user states a conflict between the context and the model's knowledge. Solid bars are wrong answers that reach the user with no warning (number above each bar of ours); the hatched part is wrong answers whose text states the conflict. The plain reader returns a short answer, so all its wrong answers are unwarned.

Lockbox	Data	System tested	Status
1	WebQSP test Yih et al., 2016 , CWQ test Talmor and Berant,2018	first version (first selector, lenient matching rule)	mixed; scored with a rule later found lenient, so not used as evidence ( Section G.1 )
2	Mintaka test ( Sen et al., 2022 ; entity answers, 2,432 questions), QALD-10 test ( Usbeck et al., 2024 ; 171) with Wikidata 1-hop context	v2 (earlier selector)	failed; reported in Section G.2
3	ClashEval ( Wu et al., 2024 ; 10,942 records: 1,247 original contexts, 9,695 perturbed, all perturbation levels); Housing Statute QA ( Zheng et al., 2025 ; 1,000 yes/no questions, BM25 retrieval within the state)	v2 as preregistered; v3 added by an amendment written before any output was read	Section 7.2

Appendix

Table 22

Lockbox	File (first 16 hex digits of its SHA-256)	Frozen at
1	PREREGISTRATION_LOCKBOX.json ( 5847cc03f7a51f05 )	2026-10-06
2	PREREGISTRATION_LOCKBOX2_SELECTION.json ( 22afe8ac578b90d5 ), PREREGISTRATION_LOCKBOX2_SYSTEM.json ( 4a9155d8dfb18bc4 )	2026-10-06 (selection), 2026-10-07 (system)
3	PREREGISTRATION_LOCKBOX3.json ( c10d51bf9086bea1 )	2026-10-08 01:28
3, amendment 1	PREREGISTRATION_LOCKBOX3_AMENDMENT1.json ( 2c07407ff7150c2e )	2026-10-08 23:02

Appendix

Table 23

Frozen artefact	SHA-256 (first 16 hex digits)
note policy, Qwen / Llama	51e21f66c4754f5a / b96f56c4b6232bd4
selector v2, Qwen / Llama	53c8ccb97d90374d / cbf7f684c7b6fd68
selector v3, Qwen / Llama	60831c3e75732a59 / c2121e95254bd208
selector v4, Qwen / Llama	3358115d6a4d9766 / b9a0b3a7d5a4ecff

Appendix

Table 24

Figure 12: Accuracy against wall-clock time per question ( GRACE development data; time measured on 40 questions, one request at a time, idle GPU). Ours (star) is the most accurate method on both backbones at a cost comparable to the other methods that call the model several times; dropping the sampled elicitation runs (circle) trades 3 to 4 points for 1 to 1.5 s.

Setting	Model knowledge (closed-book)	Context	Ours against the best alternative
GRACE broken / wrong_branch / competing	useful (about 40% correct)	missing, wrong or contradictory	large gain
ClashEval names, years, locations, perturbed	useful	wrong	gain
GRACE clean; ClashEval original	useful	right	small cost at low π; parity at high π (Qwen)
GRACE-QT (rank, qualified, string)	poor (C−W about −33)	right, unperturbed	parity
Housing statutes	harmful (60% wrong)	partly retrieved, right	Qwen: below the plain reader at the registered π, near it at high π. Llama: the frozen reader does not answer yes/no; ties the plain reader, 15–23 points below free-text methods
ClashEval drug dosages	poor (closed-book wrong on 56–58% of answered questions)	wrong	reduced error rate with Qwen, no low-risk operating point

Appendix

Table 26

Situation in a deployed system	What goes wrong today	Evidence in this paper
Knowledge graph or index with missing facts (incomplete extraction, an entity not yet linked)	the reader abstains or answers from a partial path although the model knows the fact	GRACE broken: +6.9 (Qwen) and +5.9 (Llama) over the plain reader; the correct answer is visible in the output on 78.8% and 72.9% of records against 44.4% and 51.4%
Retrieved fact is wrong (extraction error, outdated or edited document, poisoned source)	the reader repeats the wrong fact with no sign of doubt	GRACE wrong_branch: unwarned wrong answers fall from 69.2% to 11.1% (Qwen) and from 26.4% to 14.1% (Llama); ClashEval perturbed documents: +35.6 (Qwen) and +16.4 (Llama) over the plain reader
Sources disagree with each other (two documents, two graph branches)	the reader picks one silently	GRACE competing: +8.1 (Qwen) and +7.1 (Llama); the conflict is stated on 38.8% and 30.5% of records
Mixed traffic, reliability of the context unknown per query	a context-following system and a memory-first system each fail on one side	one system, one parameter: the total is flat for π between 0.1 and 0.5 on GRACE (Section 7.1 ), and π = 0.99 returns to context-following behaviour

Appendix

Table 27

Task	A real question, wrong fact in the context	Plain reader	Ours	Wrong fact in context: ours − plain reader (Qwen / Llama)	Missing fact: ours − plain reader (Qwen / Llama)
Biographical lookup (father, education, award, adviser; 139 questions)	"What award did John Steinbeck receive?" The context says Alabama Hall of Fame .	"Alabama Hall of Fame"	"According to the provided information, … the Alabama Hall of Fame. As far as I know, John Steinbeck received the Nobel Prize in Literature"	+77.0 [62.6, 92.1] / +14.4 [3.6, 25.2]	+8.6 [0.0, 17.3] / +8.6 [−2.9, 20.1]
Geographic and administrative facts (capital, continent, located in, borders; 71)	"Which state of Australia has Melbourne as its capital?" The context says Western Australia .	"Western Australia"	"… As far as I know, this is incorrect; Melbourne is the capital of the state of Victoria"	+76.1 [53.5, 98.6] / +21.1 [5.6, 36.6]	+21.1 [9.9, 32.4] / +4.2 [−8.5, 18.3]
Attribution of works and discoveries (author, composer, inventor, developer; 71)	"Who is the writer of The Double Helix?" The context says Vladimir Lenin .	"Vladimir Lenin"	"… As far as I am confident, James Watson is the actual author of The Double Helix"	+95.8 [74.6, 116.9] / +21.1 [5.6, 36.6]	+5.6 [−7.0, 16.9] / −1.4 [−18.3, 15.5]
Organisations: ownership, membership, leadership (owner, founder, member of, chief executive; 56)	"Who is the business owner of Barbie the doll maker?" The context says United Parcel Service .	"United Parcel Service"	"… As far as I am confident, Mattel is the actual owner of Barbie, while United Parcel Service is a logistics company"	+87.5 [67.9, 107.1] / +10.7 [−5.4, 26.8]	−3.6 [−17.9, 8.9] / +17.9 [1.8, 35.7]

Appendix

Table 28

Problem	What the literature reports	Relation to our results
Answers on search engines (AI overviews, answer engines)	With a commercial search engine about 70% of retrieved passages lack the answer, and in conflicts the model is right 47.4% of the time (Astute RAG; Wang et al., ACL 2025). 57.2% of real questions retrieve contradicting web paragraphs Yuan et al., 2026 . A deployed system repeated a joke and a satire page as advice (Section 1 ).	Directly tested: ClashEval is built from web documents with perturbed answers (Section 7.2 ); misinformation, insufficient context and conflicting sources are the wrong_branch, broken and competing states of GRACE
Assistants and question answering over knowledge graphs	Knowledge graphs are incomplete and contain wrong statements: over 70% of people in Freebase had no place of birth West et al., 2014 ; Wikidata has removed 76.5 million statements and deprecated 10 million Shenoy et al., 2022 . On CRAG, with mock web and knowledge-graph search, industry RAG systems answer 63% of questions without hallucination Yang et al., 2024 .	Directly tested: GRACE is this setting (Sections 4 and 7)
GraphRAG over graphs extracted by a language model	Extraction loses triples and context (Huang et al., IJCAI 2025).	Appendix S4 measures deployed GraphRAG-family systems on GRACE
Corpora and agent memories that others can write to	Five injected passages per question give about 90% attack success Zou et al., 2025 ; injecting records into an agent's memory through queries alone succeeds for more than 95% of injections (Dong et al., NeurIPS 2025).	Not a defence against a targeted attack; when the attacked fact is known to the model the conflict is stated (76.9% of wrong_branch records with Qwen)

Appendix

Table 29

Figure S1: ClashEval (preregistered external test, whole-answer scoring): C−W as a function of the fraction f of documents left unmodified. Left, Llama: among the methods that read the document, ours is best when most documents are perturbed and falls behind from f ≈ 0.4, because its score on unmodified documents is capped by the candidate defect of Appendix I . GenRead ignores the document and abstains on 62% of the questions, so its line is flat near −9: it is highest for f below about 0.35 and lowest above 0.6. Right, Qwen: lowering π helps when documents are unreliable, and at π = 0.99 the system coincides with the plain reader, so no setting of f makes it worse than the reader it is built on. Closed-book is flat by construction.

Figure S2: Risk against coverage on ClashEval (lockbox 3, whole-answer scoring), separately for unmodified and perturbed documents. Points are methods; the curve is our system at π = 0.25 when it answers only above a threshold on the selector probability, and the star is the frozen operating point (threshold 0.5). On perturbed documents the curve does not fall as the threshold rises: the selector probability does not rank answers by correctness there.

Direction	What was tried	Result	Lesson
Graph-edit policy trained with reinforcement learning	a teacher LLM proposes edits to the context graph; a small policy accepts, rejects or stops	Acc3 50.3% against 51.1% for applying every edit	the reward gave nothing for a correct added edge that was not a gold edge, so stopping early was always safe; the policy discarded useful knowledge
Self-verification as a filter	each proposed triple is scored alone by the LLM (AUC 0.875 on 800 held-out facts); rejected triples are deleted	50.4%, equal to deleting the same number of triples at random (50.6%)	the model knows which facts are false, but deleting a false fact with no replacement removes the relation and the reader answers "insufficient" (27 of 29 harmed wrong_branch cases)
Verify, then repair	a rejected object is refilled from the model's knowledge and re-verified; never delete without a replacement	+4.8 to +6.8 Acc3 over the plain reader on two fresh slices	useful, but the context is edited, and stale knowledge can overwrite a correct, newer fact
Confidence tags on context facts	each context fact annotated with the model's belief (three wordings)	−1.4 to −4.4 Acc3 on every slice	14% of correct facts receive a low score; a low score mixes "unfamiliar" with "false", and the reader then ignores correct facts
Gates on a "does the model know" signal (WebQSP / CWQ validation)	answer from memory when a signal passes a threshold	C−W stayed near 27	the best single signal reaches AUC 0.78 (agreement over five samples 0.73, minimum token probability 0.78, linear probe on the last hidden layer 0.76); a gate also hides the knowledge
Attention-based selector	candidates, notes and facts as tokens; softmax over candidates and "none"	C−W 37.6 against 48.2 for gradient boosting (Qwen, earlier scoring rule)	with 3,600 records the transformer does not learn which note supports which candidate; the softmax spreads probability so the cautious mode abstains too often

Appendix

Table 32

Knowledge Conflicts in Language Models

Knowledge conflicts in language models concern how models reconcile stored knowledge with conflicting context, retrieved evidence, tools, or memory.

See all 142 papers

Jun 9, 2026

From Context-Aware to Conflict-Aware: Generalizing Contrastive Decoding for Knowledge Conflict in LLMs

When large language models generate from retrieved or augmented contexts, conflicts between external context and parametric priors remain a central reliability bottleneck. Existing contrastive decoding methods follow a context-aware paradigm that unilaterally amplifies context over parametric priors, overwriting correct priors when the context is erroneous. We generalize this to the conflict-aware paradigm that dynamically allocates authority between prior and context based on conflict signals, rather than presupposing context trustworthiness. We show that the affine combination of prior and context logits yields a power family with an inherent regime asymmetry: extrapolation amplifies errors unboundedly when the prior is correct, interpolation under-corrects when the context is correct, and no static regime covers both. Existing contrastive decoding methods are instances of this family, mostly extrapolative. To evaluate both conflict directions, we propose TriState-Bench, a model-aware evaluation protocol that calibrates per-model prior knowledge to measure three conflict states: correction, resistance, and agreement. To resolve the asymmetry, we propose Adaptive Regime Routing (ARR), which routes between regimes at each step, lifting resistance EM from below 6 to 16--33 without sacrificing correction or agreement. Our code is available at https://github.com/keith-Jiang/conflict-aware-decoding.

Runze Jiang, Taiqiang Wu, Yan Wang +2

1Peking University · 2Alibaba Group · 3The University of Hong Kong

LLM Evaluation Knowledge Conflicts in Language Models Contrastive Decoding

Ask PDF

Sep 2, 2026

Large Language Models in Resolving Contextual Knowledge Conflicts

Most prior works focused on conflicts between an LLM's internal parametric knowledge and externally provided context. In contrast, we investigate how LLMs handle conflicts that arise within contextual knowledge itself. We introduce a taxonomy of six types of contextual conflicts (factual, inferential, temporal, granularity, perspective, and ambiguity) and contribute a comprehensive dataset ContextConflict for this setting. The dataset contains 5,781 samples, covers both reasoning and summarization tasks, and includes both explicit contradictions and implicit conflicts that require multi-step reasoning. Experiments on nine LLMs show that current models still fall short in resolving contextual knowledge conflicts. We further provide mechanistic interpretability insights into how LLMs process such conflicts, revealing their latent awareness of conflicts and the representational geometry underlying conflict processing. In addition, our analysis uncovers a consistent model bias towards earlier evidence, and this positional preference serves as a key obstacle to effective conflict resolution. Motivated by these findings, we further propose a simple training-free, label-free steering method that steers activations to encourage a more comprehensive incorporation of evidences for better conflict resolution. On our dataset, the method consistently improves accuracy on reasoning tasks and generates higher-quality, more balanced summaries for summarization tasks.

Xinye Yang, Zhenyang Liu, Ruisi Li +1

1Northwestern University, Evanston, IL · 2New York University, New York, NY · 3Computer & Information Science and Engineering, University of Florida, Gainesville, FL

Knowledge Conflicts in Language Models LLM Interpretability

Ask PDF

May 12, 2026

Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validation

The literature on how large language models handle conflict between their training knowledge and a contradicting document presents a persistent empirical contradiction: some studies find models stubbornly retain their trained answers, ignoring provided documents nearly half the time, while others find models readily defer to the document, following context approximately 96% of the time. We argue these contradictions dissolve once one recognises that prior experiments have studied three qualitatively distinct processing situations without distinguishing them. We propose a three-regime framework: Regime 1 (single-source updating, dominant predictor: evidence coherence), Regime 2 (competitive integration, dominant predictor: parametric certainty), and Regime 3 (task-appropriate selection, dominant predictor: task knowledge requirement). We formalise a distinction between parametric strength (exposure frequency) and parametric uniqueness (encoding consistency), showing empirically that these are orthogonal dimensions (r = -0.002, p = .97) with strength as the operative predictor in stable factual domains. We validate the framework across Claude Sonnet 4.6, GPT-5.5, Gemini 2.5 Flash, Llama 4 Maverick, and DeepSeek V3 using 9,970 API calls in three experimental phases. GEE logistic regression confirms the predicted Regime 2 certainty gradient for all five models (beta = -0.38 to -0.50, all p <= .013, BH-FDR corrected). A Regime 3 ablation shows task framing alone flips context-following from near-100% (contextual knowledge condition) to 6-71% (parametric knowledge condition), with all five models significant (p < .001). The certainty gradient is robust to multinomial outcome modeling, sensitivity analyses for hedging responses, and FDR correction.

Pruthvinath Jeripity Venkata

Independent Researcher

LLM Grounding Knowledge Conflicts in Language Models Factual Knowledge in Language Models

Ask PDF

Abstract

Knowledge Conflicts in Language Models

Explore similar work

From Context-Aware to Conflict-Aware: Generalizing Contrastive Decoding for Knowledge Conflict in LLMs

Large Language Models in Resolving Contextual Knowledge Conflicts

Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validation