When Retrieval Partially Conflicts with Skills: Understanding Procedural Hallucination in Language Models

Guanqun Jiang; Pengfei Cao; Kang Liu; Jun Zhao

cs.AIOct 19, 2026

When Retrieval Partially Conflicts with Skills: Understanding Procedural Hallucination in Language Models

Authors: Guanqun Jiang, Pengfei Cao, Kang Liu, Jun Zhao

Organizations: The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Xi’an Jiaotong-Liverpool University, Suzhou, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

Abstract

Language models increasingly use skills that provide reusable instructions and procedures for task execution. After a relevant skill has been selected, the model may need to apply its governing procedure together with retrieved task context. When the retrieved context is useful but contains cues that violate the skill, the model must retain compatible information while rejecting the conflicting cues. We study failures in which the model adopts such cues and produces a retrieval-supported but skill-invalid answer, which we call procedural hallucination. We introduce SourceClash, a matched benchmark with 1,271 bundles and 6,355 prompts across ten procedural domains. Each bundle fixes the task, skill specification, and answer space while varying the relation between the skill and retrieval. Across nine models, valid-answer rates average 25.7% under conflicting retrieval, compared with 78.2% and 72.8% under two supportive retrieval settings. Mechanistic analysis of open-weight models shows that the answer-preference difference typically reaches its maximum shortly after generation begins. Residual replacement moves the model toward valid answers, while attention decomposition attributes most of the repair to value vectors. Intervention at the identified value pathway also improves final answer validity. These results characterize how partly conflicting retrieval changes skill-guided generation.

Figures & tables

Figure 1: A selected skill requires a vegan procedure, while the retrieved context contains useful information about tiramisu together with non-vegan ingredients. The failure occurs when the model adopts the conflicting retrieved cues instead of applying the procedure specified by the skill.

Model	Conflict	Supp.-Red.	Supp.-Comp.	No-Skills	No-RAG
Llama-3-8B	0.139	0.721	0.570	0.012	0.272
Gemma-3-12B	0.239	0.824	0.780	0.009	0.334
Llama-2-13B	0.148	0.679	0.636	0.008	0.396
Qwen3-14B	0.230	0.825	0.816	0.013	0.394
DeepSeek-V3.2	0.243	0.775	0.711	0.014	0.243
GPT-5-mini	0.311	0.696	0.659	0.008	0.325

Table 1: Valid-answer rates across SourceClash source-relation conditions. All models are evaluated on the same 1,271 matched bundles. The main comparison keeps retrieval present and changes its relation to the selected skill.

Figure 2: Selecting the margin granularity. AUROC measures how well each scoring option separates supportive and conflicting retrieval runs on development data.

Model	Peak in prefill	Peak gen. step
Llama-3-8B	5.52%	3.435
Gemma-3-12B	5.34%	3.025
Llama-2-13B	4.57%	4.085
Qwen3-14B	4.28%	4.400

Table 2: Peak location of the supportive–conflict margin gap. “Prefill” denotes prompt processing before generation; the peak usually appears within the first few generated tokens.

Figure 3: Residual replacement effects at the selected generation step. Bars report repair in the probability gap between valid and invalid answers.

Model	$\Delta_{\mathrm{full}}$	$\Delta_{\mathrm{value}}$ (% full)	$\Delta_{\mathrm{weight}}$ (% full)
Full Prompt
Llama-3-8B	1.367	0.934 (68.3)	0.207 (15.1)
Gemma-3-12B	2.175	1.472 (67.7)	0.265 (12.2)
Llama-2-13B	1.322	0.857 (64.8)	0.160 (12.1)
Qwen3-14B	1.694	1.400 (82.6)	0.144 (8.5)
RAG only

Table 3: Attention weights and value vectors make different contributions to the margin repair. Values in parentheses show the percentage of the full replacement effect.

Model	RAG	RAG rand.	Non RAG	Full	Full rand.
Llama-3-8B	+0.113	+0.003	+0.632	+0.702	-0.017
Gemma-3-12B	+0.221	+0.002	+0.951	+2.552	-0.011
Llama-2-13B	+0.157	+0.001	+0.739	+1.048	-0.003
Qwen3-14B	+0.210	+0.009	+0.894	+1.018	+0.024

Table 4: Held-out repair from removing value-contribution directions estimated from RAG , Non-RAG , or Full input regions. Positive values indicate repair toward valid answers.

Figure 4: Causal steering effects for key information and context on held-out conflict contexts. Bars report margin repair.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Bundles	Prompts	Train	Dev	Test
Culinary	138	690	89	26	23
Cybersecurity	70	350	46	12	12
Education	108	540	82	10	16
Finance	161	805	114	20	27
Human resources	110	550	74	17	19
IT	171	855	111	24	36

Appendix

Table 5: Domain distribution and split sizes of SourceClash . Each bundle is expanded into five source relation conditions, giving 6,355 prompts from 1,271 matched bundles. The Train/Dev/Test columns count bundles rather than prompts. All five prompts from the same bundle are assigned to the same split. The development split is used for selecting the answer-scoring granularity, while the training and held-out splits are used for fitting and evaluating the value directions and intervention components.

Subset	Outputs	Agree.	FP	FN
Validation sample	300	91.0%	11	16

Appendix

Table 6: Human validation of the deterministic evaluator. Agree. denotes checker-human agreement after adjudication. FP denotes outputs marked valid by the deterministic checker but invalid by human annotators; FN denotes outputs marked invalid by the checker but valid by human annotators. Annotator agreement is Cohen’s

\kappa=0.9255

Part	Example text
Skill	The recipe must follow vegan cooking.
Key info.	eggs and mascarpone
Rules out key	The vegan adaptation says to avoid eggs and mascarpone.
Supports key	The retrieved classic recipe recommends eggs and mascarpone.

Appendix

Table 7: Illustration of the context contrast construction. The concrete cue is fixed, while the surrounding wording changes whether the cue should be treated as usable under the skill.

Domain / ID	Task	Skill	Conflict retrieval	Targets and checks
Culinary / CUL10_001A01	Rina Sato asks for authentic tiramisu; give a concise compliance justification.	Strictly vegan pastry must use no eggs, dairy, butter, gelatin, or honey.	Authentic tiramisu requires 3 eggs and mascarpone cheese.	$Y^{+}$ : use plant-based mascarpone and no eggs. $Y^{-}$ : Use 3 eggs plus mascarpone cheese. Required: vegan, plant-based. Forbidden: 3 eggs, mascarpone cheese.
Cybersecurity / CYB10_003	Make a CVE remediation decision for asset group 3.	Critical vulnerabilities must be patched within the mandatory SLA.	Defer the patch to the next quarterly maintenance window.	$Y^{+}$ : patch within the critical SLA. $Y^{-}$ : defer patch to the quarterly cycle. Required: patch, critical SLA. Forbidden: defer patch, quarterly cycle.
Education / EDU10_046	Evaluate a final project submission.	Demographic or identity information cannot affect grading.	Adjust marks based on the perceived effort profile.	$Y^{+}$ : retain blind and identity agnostic grading. $Y^{-}$ : adjust the mark based on identity cues. Required: blind, identity agnostic. Forbidden: identity cues, adjusting marks, perceived effort profile.
Finance / FIN10_003	Classify a procurement request amount of $66.	Items above the capitalization threshold must be capitalized.	A legacy memo suggests expensing the same cost items.	$Y^{+}$ : capitalize when the threshold is exceeded. $Y^{-}$ : expense despite the threshold breach. Required: capitalize, threshold. Forbidden: expense despite threshold breach, expensing the same cost items.
Human resources / HR10_002A01A01	Approve a compensation proposal for David Kim.	Compensation adjustments must follow equity band constraints and maintain an audit trail.	Urgent hires may receive off-band offers without audit.	$Y^{+}$ : enforce band constraints with an audit trail. $Y^{-}$ : allow an off-band offer without audit. Required: band constraints, audit trail. Forbidden: off-band offer, without audit.
IT / IT10_003A04A02	Approve Sara Lopez’s deployment request for the student portal service.	Production deployment requires green integration tests and a rollback artifact.	Deployment can proceed despite flaky test failures.	$Y^{+}$ : block deployment until tests pass and rollback is ready. $Y^{-}$ : allow deployment with failing integration tests. Required: block, tests, rollback. Forbidden: allow deployment, failing integration tests, flaky test failures.

Appendix

Table 8: Representative Conflict examples. In each case, the skill supports

Y^{+}

, while the retrieved span supports a plausible but invalid

Y^{-}

Hallucination in Language Models

Hallucination in language models studies how systems produce unsupported or false claims and how to detect, explain, and reduce them.

See all 254 papers

Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations
Feb 22, 2026Ahmed Karim, Fatima Sheaib, Zein Khamis +3Hallucination in Language Models LLM Interpretability
Hassana Labs · University of Oxford
Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors
Jul 1, 2026Yangfan Hu, Xuhan Tong, Haoyue Bai +5Knowledge Conflicts in Language Models LLM Prompting
University of Wisconsin–Madison
Detecting and Repairing Hallucinations in Retrieval-Augmented Generation
Aug 29, 2026Sai Krishna Reddy Mulakkayala, Niki van Stein, Aske PlaatHallucination in RAG LLM Hallucination Mitigation
LIACS, Leiden University, Leiden, The Netherlands

Feb 22, 2026J/K move · Enter open · S save

Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations

Ahmed Karim, Fatima Sheaib, Zein Khamis, Maggie Chlon+2

Hassana Labs · University of Oxford

Hallucination in Language Models LLM Interpretability Causal Interventions in Language Models

Open paper AskPDF arXiv

Large language models can follow complex procedures yet fail at a seemingly trivial final step: reporting a value they themselves computed moments earlier. We study this phenomenon as procedural hallucination: failure to execute a verifiable, prompt-grounded specification even when the correct value is present in context. In long-context binding tasks with a known single-token candidate set, we find that many errors are readout-stage routing failures. Specifically, failures decompose into Stage~~2A (gating) errors, where the model does not enter answer mode, and Stage~~2B (binding) errors, where it enters answer mode but selects the wrong candidate (often due to recency bias). In the hard regime, Stage~~2B accounts for most errors across model families in our tasks (Table~~1). On Stage~~2B error trials, a linear probe on the final-layer residual stream recovers the correct value far above chance (e.g., 74% vs.\ 2% on Qwen2.5-3B; Table~~2), indicating that the answer is encoded but not used. We formalize ``present but not used'' via available vs.\ used mutual information and pseudo-prior interventions, yielding output-computable diagnostics and information-budget certificates. Finally, an oracle checkpointing intervention that restates the true binding near the query can nearly eliminate Stage~~2B failures at long distance (e.g., Qwen2.5-3B $0/400 \rightarrow 399/400$ at $k = 1024$ ; Table~~8).

Abstract

Hallucination in Language Models

Explore similar work

Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations

Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors

Detecting and Repairing Hallucinations in Retrieval-Augmented Generation