When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation

Haolin Jin; Pengyue Yang; Huaming Chen

cs.CLAug 17, 2026

When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation

Authors: Haolin Jin, Pengyue Yang, Huaming Chen

Organizations: School of Electrical and Computer Engineering The University of Sydney Sydney, NSW, Australia

Abstract

Retrieval-augmented generation (RAG) improves large language models by grounding generation in external evidence, but it also introduces a source trust problem: retrieved context may be useful, irrelevant, or even misleading. Existing RAG systems often apply a fixed trust policy toward retrieved evidence, which can either over-trust incorrect context or underuse context when the user explicitly asks for context-following behavior. Therefore, we propose Intent-Guided Decoding (IGD), a framework that arbitrates between retrieved context and parametric memory according to user intent. IGD uses answer-level filtering and token-level correction to steer the final decoding trajectory between retrieved context and parametric memory. We evaluate IGD on three faithful QA benchmarks and three factual-conflict benchmarks across five LLMs, IGD substantially improves factual recovery, achieving gains of up to 65.4 percentage points on factual-conflict benchmarks over Direct RAG, while preserving or improving strict context-following behavior, this findings highlight the importance of balancing factuality and faithfulness in RAG.

Figures & tables

Fig. 1: The figure illustrates two truth mode examples: a faithful QA case where the retrieved context correctly supports the answer, and a factual-conflict case where the retrieved context is misleading. For each case, IGD first obtains branch-level previews and top- $K$ token distributions, then applies token-level correction only when the context and memory branches exhibit distributional conflict. The correction direction is determined by entropy-based branch confidence and the prompt-mode prior, while the intervention strength is scaled by the reliability of the selected source.

Model	Method	IA	Faithful benchmarks			Factual-conflict benchmarks
Model	Method	IA	KILT-NQ	TriviaQA	SQuAD	ConflictBank	NQ-Swap	CounterFact
Qwen3-32B	Closed-book Q-only †	56.8	35.4	62.0	32.4	56.4	70.0	84.6
	Direct RAG	51.0	74.8	88.8	90.4	15.0	27.0	10.0
	ExplicitSCR	12.9	16.0	18.4	15.8	4.0	3.0	20.2
	RCR-InternalEval	62.1	58.2	74.6	47.8	47.8	70.5	73.6
	RCR-ContextEval	53.6	68.8	80.0	83.2	21.0	37.0	31.4

TABLE I: Truth-mode comparison across three faithful benchmarks and three factual-conflict benchmarks. IA is the macro-average over the six benchmark columns. Bold marks the best context-using method in each column, and underline marks the second-best. IGD also reports deltas against Direct RAG under the same model and benchmark.

Fig. 2: Parametric recovery rate (PRR) of IGD on factual-conflict benchmarks, the dashed line denotes full recovery relative to the closed-book reference.

Model	Method	IA	Faithful benchmarks			Factual-conflict benchmarks
Model	Method	IA	KILT-NQ	TriviaQA	SQuAD	ConflictBank	NQ-Swap	CounterFact
Qwen3-32B	Direct RAG	83.0	72.6	89.4	91.0	83.2	63.0	99.0
	IGD	84.3	69.4 $\downarrow$ 3.2	86.8 $\downarrow$ 2.6	90.0 $\downarrow$ 1.0	87.2 $\uparrow$ 4.0	72.5 $\uparrow$ 9.5	99.6 $\uparrow$ 0.6
Qwen2.5-14B	Direct RAG	84.2	74.6	89.2	94.0	84.4	63.0	100.0
	IGD	88.1	76.4 $\uparrow$ 1.8	88.6 $\downarrow$ 0.6	93.6 $\downarrow$ 0.4	89.2 $\uparrow$ 4.8	81.0 $\uparrow$ 18.0	100.0 $\leftrightarrow$ 0.0
Llama-3-8B	Direct RAG	80.8	73.0	88.0	88.4	85.4	50.0	99.8

TABLE II: Strict context following prompt. Factual-conflict benchmarks are evaluated by context accuracy, testing whether IGD preserves user intent when the user explicitly asks to follow the provided context.

Fig. 3: Sensitivity of truth mode performance to token-level steering hyperparameters on Qwen2.5-14B-Instruct.

Fig. 4: Answer distribution on factual-conflict benchmarks under truth mode.

Fig. 5: Snippet localization accuracy versus downstream IGD performance.

RAG Citation Faithfulness

See all 19 papers

Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAG
May 29, 2026Ibne Farabi Shihab, Fariya Afrin, Sanjeda Akter +1Retrieval-Augmented Generation Language Model Decoding
Department of Computer Science, Iowa State University · Department of Computer Science, Kalinga Institute of Industrial Technology · Department of Civil, Construction & Environmental Engineering, Iowa State University
Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation
Apr 9, 2026Zhengyi Zhao, Shubo Zhang, Zezhong Wang +7Hallucination in RAG Retrieval-Augmented Generation
The Chinese University of Hong Kong · Ministry of Education Key Laboratory of High Confidence Software Technologies, CUHK · University of International Relations +2
Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction
May 4, 2026Florian Geissler, Francesco Carella, Laura Fieback +1Retrieval-Augmented Generation Factual Consistency Evaluation
Fraunhofer Institute for Cognitive Systems (IKS), Munich, Germany · Volkswagen AG, Wolfsburg, Germany

May 29, 2026J/K move · Enter open · S save

Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAG

Ibne Farabi Shihab, Fariya Afrin, Sanjeda Akter, Anuj Sharma

Department of Computer Science, Iowa State University · Department of Computer Science, Kalinga Institute of Industrial Technology · Department of Civil, Construction & Environmental Engineering, Iowa State University

Retrieval-Augmented Generation Language Model Decoding Knowledge Conflicts in Language Models Evidence-Grounded Generation

Open paper AskPDF arXiv

As retrieval-augmented generation (RAG) systems scale, it becomes increasingly challenging to ensure faithful grounding in external evidence. Large language models may still prioritize parametric knowledge over retrieved information when conflicts arise. We propose a novel training-free decoding framework, Grounded Decoding, designed to improve factual consistency in RAG without modifying model parameters. Unlike standard approaches that rely on a single conditional distribution, our method constructs two matched-prompt distributions at every generation step: (1) a full RAG distribution conditioned on the query, retrieved documents, and generated prefix, and (2) a retrieval-only distribution conditioned solely on retrieved evidence and the same prefix. The final next-token distribution is derived as the unique solution to a KL-barycenter objective over the probability simplex, yielding a normalized geometric fusion of the two distributions.This formulation naturally recovers standard RAG when the grounding weight is zero and smoothly shifts probability mass toward retrieved evidence as grounding strength increases. We further introduce a conflict-aware adaptive weighting scheme that dynamically adjusts grounding based on distributional disagreement and retriever confidence. Experiments on ALCE, Natural Questions, and FActScore demonstrate consistent improvements in factual accuracy and citation quality over standard RAG and competitive decoding-time baselines, while maintaining fluency. Our results indicate that probability-level fusion provides a strong and efficient alternative to logit-level intervention methods for faithful RAG decoding.

Abstract

RAG Citation Faithfulness

Explore similar work

Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAG

Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation

Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction