Context or Knowledge: How LLMs Attribute Their Own Answers

Kirill Afendulev; Alexey Dontsov; Alla Chepurova; Elena Tutubalina; Anton Korznikov

cs.CLOct 23, 2026

Context or Knowledge: How LLMs Attribute Their Own Answers

Authors: Kirill Afendulev, Alexey Dontsov, Alla Chepurova, Elena Tutubalina, Anton Korznikov

Abstract

Language models can answer a question from a supplied document or from their parametric knowledge. When later asked where the answer came from, what determines their source report? We study this by intervening on the model's key-value (KV) cache after the answer is generated and before the source question is asked. On a controlled factual task with four models (7B-32B), changing or removing the document's cache never changes the report. In contrast, transplanting the question-and-answer (Q+A) cache between runs with identical question and answer tokens transfers it: every eligible report follows the donor, while the recipient's document is unchanged. On the two smaller models, steering Q+A values along directions fitted before any source question reverses 98.9-100% of reports across four document conditions at a fixed strength, while norm-matched random directions flip at most 1.1%. The components of keys and values along fitted directions carry the transplant effect: copying them alone nearly reproduces the report change; holding them fixed nearly prevents it. Applied during answer generation, the same value directions also change which source supplies the answer, though more weakly and depending on the prompt. Source reporting in this setting thus depends causally on cached state produced while answering. We do not establish general reporting accuracy or a complete interpretation of this state.

Figures & tables

Panel of Figure 1 — Figure 1: Same answer, different source reports. Each row is a separate dialogue. After answering, the model reports context for the document or knowledge for its own knowledge. We ask how this report is produced. Schematic examples; source-question wording shortened.

Hypothesis	Proposed mechanism	Evidence
H1: Re-read the document	The model re-inspects the document to produce the source report.	✗ Not supported as a sufficient explanation. Changing or removing the document after answering does not change the report (§ 3 ).
H2: Judge the answer against knowledge	The model determines the report by evaluating the answer against its parametric knowledge.	✗ Not supported as a sufficient explanation. Transplanting the Q+A cache changes the report while keeping the answer tokens and recipient document unchanged (§ 4 ).
H3: Read a cached trace	The report depends on source-related state produced while answering.	✓ Supported by our interventions. Q+A cache transplants transfer the report, and steering Q+A values changes it (§ 4 , § 5 ). Clamping fitted K+V projections nearly blocks transfer; transferring those projections alone nearly reproduces it (§ 6 ).

Table 1: Hypotheses for the source report and evidence from our interventions. Blue ✓ marks evidence supporting the hypothesis; orange ✗ marks evidence against it. H1 and H2 are tested as sufficient explanations of the report. H3 represents the hypothesis that the report depends on source-related state produced while answering.

Panel of Figure 2 — Figure 2: The document changes; the source report does not. The LLM answers “Tokyo” with a counterfactual document. Each edited branch keeps the original question-and-answer cache, and either removes the document entries or replaces them with an irrelevant fact, which remains readable. All three branches then report context to the source question, and the answer shown was generated before either edit. Schematic example; Table 2 gives results for all four models.

Document edit	OLMo-3-7B	Qwen3-8B	OLMo-3.1-32B	Qwen3-32B
Drop counterfactual	0%	0%	0%	0%
Counterfactual $\to$ irrelevant	0%	0%	0%	0%
Irrelevant $\to$ true fact	0%	0%	0%	0%

Table 2: Zero source-report flips after document edits. Cells are report-flip rates (%), with Q+A state retained. The first two edits try to remove a context report; the third tries to create one.

Panel of Figure 3 — Figure 3: Source report follows the transplanted cache. Both runs answer “Paris”, but report different sources. We copy the donor’s Q+A cache into the recipient after answering, retaining the recipient’s document. Asked anew, the source question now elicits context . Colours identify the origin of cached state; the original question and its answer are not regenerated.

Recipient $\leftarrow$ Q+A donor	OLMo-3-7B	Qwen3-8B	OLMo-3.1-32B	Qwen3-32B
True fact $\leftarrow$ neutral	100%	100%	100%	100%
Neutral $\leftarrow$ true fact	100%	100%	100%	100%
True fact $\leftarrow$ irrelevant	100%	100%	100%	100%
Irrelevant $\leftarrow$ true fact	100%	100%	100%	100%

Table 3: Every eligible report follows the Q+A donor. Cells are report-flip rates (%) among eligible facts with identical answer tokens. The document stays unchanged; only Q+A KV entries are replaced.

Panel of Figure 4 — Figure 4: Steering Q+A values changes the source report. Left: addition makes irrelevant-document runs report context ; both reach 100% at $\alpha=2$ . Right: subtraction makes counterfactual runs report knowledge ; both reach 100% by $\alpha=3$ . The answer tokens remain fixed. Dotted curves show mean random-control rates over three draws, at most 1.1% on the left and zero on the right. Rates are over eligible item–split pairs. Other document conditions and span ablations appear in Tables 11 – 12 .

Panel of Figure 5 — Figure 5: Which part of the transplanted cache carries the report? Each row independently edits the same recipient’s Q+A cache. Projections are components along the fitted directions. Schematic example; Table 4 gives measured rates in both directions.

Intervention	OLMo-3-7B	Qwen3-8B
Full Q+A KV transplant	100%	100%
+ restore recipient K+V projections	1.1%	0%
+ clamp energy-matched subspace	100%	94.7%
No transplant: donor K+V projections only	97.9%	100%

Table 4: Projections onto fitted directions block and reproduce report transfer. Report-flip rates (%) over directional evaluations for true-fact

\leftrightarrow

irrelevant-fact pairs, pooled over two held-out relation splits. Every row tests the same pairs in both directions.

Panel of Figure 6 — Figure 6: The value intervention also changes the answer’s source. Value steering at $\alpha=2$ . Left: instructed to use documents, but steering favours memory. Right: the instruction favours knowledge when the document may be wrong; steering favours the document. Coloured bars are switches; grey are answers containing neither value; diamonds are mean random switches over three draws. Switch percentages are on the right. Rates are over eligible item–split pairs. The residual-stream comparison is in Table 15 .

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Relation	Phrase	Candidates
capital	capital	30
symbol	chemical symbol	26
author	author	22
continent	continent	21
language	official language	20
painter	painter	12

Appendix

Table 5: The eight relations in the candidate pool. Counts precede the length-matching and eligibility requirements of the reported experiments.

document	example	control variant	role
true fact	The capital of France is Paris.	—	relevant, true value
counterfactual	The capital of France is Tokyo.	… is Lima.	relevant, false value
irrelevant fact	The capital of Canada is Ottawa.	… of Egypt is Cairo.	true fact, other subject
neutral sentence	Snow melted slowly on the hills.	The library was open until late.	no fact at all

Appendix

Table 6: The seven document variants for the question What is the capital of France? Each control variant is a second document of the same condition.

Counterfactual run (read “Tokyo”, answered “Tokyo”): reports context
	OLMo-3-7B	Qwen3-8B	OLMo-3.1-32B	Qwen3-32B
intact	100%	100%	100%	100%
Document drop	100%	100%	100%	100%
Document swap $\to$ irrelevant fact	100%	100%	100%	100%
model quotes the new document	100%	100%	100%	100%
Irrelevant-fact run (answered “Paris” from memory): reports context

Appendix

Table 7: Document edits leave the report fixed; question transplants often move it. Percentages of eligible items with the stated report. Bold: the weak direction on each 32B model (Section 4 ).

The same true answer “Paris”; Question swap
recipient $\leftarrow$ donor: report becomes	OLMo-3-7B	Qwen3-8B	OLMo-3.1-32B	Qwen3-32B
true-fact $\leftarrow$ neutral: knowledge	80.2%	100%	100%	95.8%
neutral $\leftarrow$ true-fact: context	97.4%	100%	73.3%	100%
A false answer “Tokyo” in plain sight; Question swap
counterfactual $\leftarrow$ neutral: knowledge	90.4%	100%	68.3%	25.8%
still reads “Tokyo” in the document	83.3%	97.2%	68.3%	25.8%

Appendix

Table 8: The report follows the cache even when the answer, or all visible text, is fixed. Percentages of eligible items whose report became the stated one. Bold: Qwen3-32B with a false answer, where question-only transplants are weakest.

Panel of Figure 7 — Figure 7: The report moves with question state, not with the edited document. Question swaps exchange counterfactual and irrelevant-fact runs; coloured dots show the fraction whose report follows the donor. Document swaps replace a counterfactual with an irrelevant fact (left), or an irrelevant fact with a true fact stating the answer (right): zero flips on every model. Document deletion also produces zero flips. Question swaps are asymmetric at 32B, motivating the answer-cache tests. Controls: Table 7 .

swap of the question span	keys+values	keys only	values only
counterfactual $\leftarrow$ irrelevant (remove)	97.9% $\,\cdot\,$ 100%	0% $\,\cdot\,$ 0%	6.2% $\,\cdot\,$ 0%
true-fact $\leftarrow$ irrelevant (remove)	92.9% $\,\cdot\,$ 100%	0% $\,\cdot\,$ 0%	0% $\,\cdot\,$ 0%
irrelevant $\leftarrow$ counterfactual (create)	86.6% $\,\cdot\,$ 90.6%	40.2% $\,\cdot\,$ 0.9%	99% $\,\cdot\,$ 97.2%
irrelevant $\leftarrow$ true-fact (create)	90.9% $\,\cdot\,$ 98.3%	54.5% $\,\cdot\,$ 5.2%	100% $\,\cdot\,$ 100%

Appendix

Table 9: The question span swapped as keys+values, keys only or values only. Cells: OLMo-3-7B

\,\cdot\,

Qwen3-8B, percentages of eligible items whose report flipped (remove or create the context report). Value-only transplants create the report in most cases; removing it is more effective with keys and values together.

	question only	answer only	question $+$ answer
OLMo-3-7B, Qwen3-8B, Qwen3-32B (all four directions)	75–100%	0–3%	100%
OLMo-3.1-32B, irrelevant-fact $\leftarrow$ true-fact	30%	0%	100%
OLMo-3.1-32B, true-fact $\leftarrow$ irrelevant-fact	100%	59%	100%

Appendix

Table 10: Percentages of eligible items whose report moved, by swapped span.

Document / direction	Model	$\alpha=1$	$\alpha=2$	$\alpha=3$	$\alpha=4$
Irrelevant fact: create	OLMo-3-7B	42.6%	100%	100%	100%
	Qwen3-8B	4.9%	100%	100%	100%
Neutral sentence: create	OLMo-3-7B	12.8%	88.5%	100%	100%
	Qwen3-8B	0%	2.4%	100%	100%
Counterfactual: remove	OLMo-3-7B	0%	60.5%	100%	100%
	Qwen3-8B	12.5%	100%	100%	100%

Appendix

Table 11: Joint Q+A steering across all four document conditions. Report-flip rates (%) over eligible item–split pairs pooled across the two retained relation splits. Every cell reaches 100% at

\alpha=4

; at

\alpha=3

, the only exception is OLMo on true facts (98.9%). Bold marks complete flipping. Random means are given below.

Document / direction	Model	Q only	A only	Q+A
Irrelevant fact: create	OLMo-3-7B	62.3%	67.2%	100%
	Qwen3-8B	97.5%	3.7%	100%
Neutral sentence: create	OLMo-3-7B	42.3%	62.8%	100%
	Qwen3-8B	8.4%	0%	100%
Counterfactual: remove	OLMo-3-7B	0%	72.1%	100%
	Qwen3-8B	58%	85.2%	100%

Appendix

Table 12: Joint editing is more effective than either span alone. Report-flip rates (%) over eligible item–split pairs at the same strength,

\alpha=3

, for all three edits. This descriptive comparison does not establish that Q and A encode independent source variables.

Document / direction	$\alpha$	OLMo-3-7B	Qwen3-8B
Irrelevant fact: create	2	100%	100%
Counterfactual: remove	3	100%	100%

Appendix

Table 13: Report flips with successful answer repetition at the operating points. Joint success rates (%) over eligible item–split pairs for Q+A steering. Every evaluation flips and passes repetition; document quotation also succeeds throughout.

Document / direction	Model	Supporting vs. irrelevant	Fixed document
True fact: remove	OLMo-3-7B	37.5%	30.7%
	Qwen3-8B	23.9%	77.3%
Counterfactual: remove	OLMo-3-7B	72.1%	72.1%
	Qwen3-8B	85.2%	100%
Irrelevant fact: create	OLMo-3-7B	67.2%	37.7%
	Qwen3-8B	3.7%	2.5%

Appendix

Table 14: Two fitting contrasts, the same answer-only intervention. Report-flip rates (%) over eligible evaluations at

\alpha=3

; remove/create refer to context . Supporting versus irrelevant is the original contrast; fixed document uses a prompt correction. This is a descriptive comparison at a common multiplier, not at matched edit norms or a separately optimised strength.

		OLMo-3-7B			Qwen3-8B
Steer toward	Edit	Switched	Random	Damaged	Switched	Random	Damaged
Memory	Values	70.9	1.6	9.3	51.2	4.1	1.2
	Residual	16.3	0.2	0.6	75.6	3.7	2.4
Document	Values	85.3	23.6	0	72.8	3.2	0
	Residual	52.3	5.5	1.8	98	1.2	0

Appendix

Table 15: Source steering in the two main settings at

\alpha=2

. All rates are percentages over eligible item–split pairs. Switched : the answer changes to the target source’s value. Random : source switches under random directions, averaged over three draws. Damaged : the answer contains neither the document nor the memory value.

section	script	artifacts
items	90_clean_items.py	items.jsonl
3 – 4 , document and question swaps	91_clean_kv_patching.py	kv_patching.json , kv_patching_items.jsonl
4 , Identical-text swap	93_clean_answer_patching.py	answer_patching.json , answer_patching_items.jsonl
tables for 3 – 4	92_clean_tables.py	tables.md
5 , report-steering sweep	author-supplied aggregate table	data/report_steering_update.csv
Appendix C.7	author-supplied block 3i (script 105)	data/report_steering_checks.json

Appendix

Table 16: Scripts and result artifacts behind each section (per model, under a directory named by the model tag).

	OLMo-3-7B		Qwen3-8B
	Create	Remove	Create	Remove
K+V transplant	100%	100%	100%	100%
V transplant	100%	10.6%	100%	47.4%
V transplant + V clamp	0%	0%	0%	0%
V transplant + random V clamp	100%	10.6%	100%	47.4%
K transplant	78.7%	0%	39.5%	0%

Appendix

Table 17: Value-coordinate interventions. Author block 3j. Report-flip rates (%) over directional evaluations on true-fact–irrelevant-fact pairs. Create/remove refer to context . Random-control rates are means over three draws.

	OLMo-3-7B		Qwen3-8B
	Create	Remove	Create	Remove
K+V transplant + V clamp	100%	0%	68.4%	0%
K+V transplant + K+V clamp	2.1%	0%	0%	0%
K+V transplant + V clamp + random K clamp	100%	0%	67.6%	0%
K transplant + K clamp	0%	0%	0%	0%
K transplant + random K clamp	76%	0%	38.7%	0%

Appendix

Table 18: Joint key/value interventions. Author block 3k. Report-flip rates (%) over directional evaluations on true-fact–irrelevant-fact pairs. Create/remove refer to context . Random-control rates are means over three draws.

Intervention	OLMo-3-7B	Qwen3-8B
Full K+V transplant	100%	100%
+ selected-coordinate clamp	1.1%	0%
+ top-PC clamp	100%	100%
+ energy-matched clamp	100%	94.7%
+ random clamp	100%	100%
V transplant + selected-coordinate clamp	0%	0%

Appendix

Table 19: Controls for coordinate transfer. Author block 3l; true-fact–irrelevant-fact pairs only. All cells give report-flip rates (%); random conditions average three draws. Top PC and matched subspaces are fitted after removing the selected direction.

Model	Cache	Selected	Matched	Random
OLMo-3-7B	V	11.6%	12.3%	0.0%
OLMo-3-7B	K	25.1%	21.0%	0.0%
Qwen3-8B	V	15.7%	18.5%	0.1%
Qwen3-8B	K	36.2%	49.3%	0.8%

Appendix

Table 20: Held-out difference energy covered by each intervention subspace. Fractions of squared donor–recipient differences, summed over Q+A positions and layers. Matching is on fitting data, so held-out fractions need not be equal.

Document edit	OLMo-3-7B	Qwen3-8B	OLMo-3.1-32B	Qwen3-32B
Drop counterfactual	100%	98.4%	99.4%	100%
Counterfactual $\to$ irrelevant	100%	100%	99.4%	100%
Irrelevant $\to$ true fact	0%	0%	0%	0%
Irrelevant $\to$ counterfactual	0%	0%	0%	0%

Appendix

Table 21: DynamicQA: document edits mostly preserve the source report. Cells give context report rates (%) among eligible items, with Q+A retained. In the first two rows, context is the original report; in the last two, the original report is knowledge .

Recipient $\leftarrow$ Q donor: report	OLMo-3-7B	Qwen3-8B	OLMo-3.1-32B	Qwen3-32B
Counterfactual $\leftarrow$ irrelevant: knowledge	54.5%	100%	95.5%	57.6%
Irrelevant $\leftarrow$ counterfactual: context	78.8%	72.4%	15.2%	72.9%
True fact $\leftarrow$ uninformative: knowledge	94.9%	100%	99.5%	70.8%
Uninformative $\leftarrow$ true fact: context	84.7%	77.1%	25.9%	59.8%
Counterfactual $\leftarrow$ uninformative: knowledge	79.7%	99.2%	94.5%	16.8%
+ reads the false document value	66.7%	90.7%	93.9%	11.9%

Appendix

Table 22: DynamicQA: question-only transplants. Cells give the stated report rate (%) over eligible items; the recipient keeps its document and answer cache. The read-back row measures the joint event: the report becomes knowledge and the model still reads the counterfactual value from its document.

True fact $\leftarrow$ uninformative: report becomes knowledge
Edited span	OLMo-3-7B	Qwen3-8B	OLMo-3.1-32B	Qwen3-32B
Q only	97.6%	100%	99.4%	77.6%
A only	10.6%	8.5%	51.9%	7.2%
Q+A	100%	100%	100%	100%
Uninformative $\leftarrow$ true fact: report becomes context
Q only	82.4%	84.5%	27.1%	72.4%

Appendix

Table 23: DynamicQA: joint Q+A transplants transfer nearly every report with identical text. Cells give donor-following report rates (%) among eligible items. Within each pair, Q, A and Q+A edits use the same subset with identical question and answer tokens. Bold highlights the joint intervention, including its exceptions.

Claim Attribution in Language Models

See all 15 papers

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge
Sep 10, 2026Wenkang Wei, Yuan Fang, Renhe Jiang +2LLM Interpretability Factual Knowledge in Language Models
University of Science and Technology of China Hefei, China · Singapore Management University Singapore, Singapore · The University of Tokyo Tokyo, Japan +1
Position Matters More Than Channel: Talking a Language Model Out of a Correct Answer
Oct 26, 2026Xinyi NiLLM Prompting Knowledge Conflicts in Language Models
Brandeis University
Cache2LoRA: Understanding and Improving Context Internalization
Oct 16, 2026Yi Fang, Gaurav Srivastava, Zhenyu Bi +2LLM Context Compression Knowledge Augmentation for Language Models
Department of Computer Science, Virginia Tech, Blacksburg, VA, USA · Department of Biomedical Data Science, Stanford University, Stanford, CA, USA

Sep 10, 2026J/K move · Enter open · S save

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

Wenkang Wei, Yuan Fang, Renhe Jiang, Hong Cheng+1

University of Science and Technology of China Hefei, China · Singapore Management University Singapore, Singapore · The University of Tokyo Tokyo, Japan+1

LLM Interpretability Factual Knowledge in Language Models Causal Interventions in Language Models Language Model Probing

Open paper AskPDF arXiv

How does a language model's dependence on query-routing information and target knowledge change as it answers a question? We study this question through layerwise interventions on the hidden state at the end of the question. Across Qwen, Llama, and Gemma, we compare country-continent questions with noun, adjective, and code answers while keeping several fitted measurements distinct. A pair-conditioned request direction describes which country is queried in natural single-country questions; a global request direction describes first- versus second-country requests in paired questions; separate selection candidates test control among contents already available in the hidden state. A diagnostic reanalysis of frozen Qwen natural-question states shows that the pair-conditioned direction grows stronger before interventions on it begin to alter later fitted knowledge, with this causal window opening while answer-supporting content is still forming. The paired three-model trajectories are not uniform: Gemma shows a partially overlapping mid-layer routing-content profile, whereas Llama has no sustained routing-effect window under the same gates. In the paired protocol, dependence on the global request direction decreases from fixed earlier to later layer sets while dependence on fitted content persists. A matched Qwen comparison shows that the pair-conditioned direction retains a late effect, so this operational handoff concerns the global fitted direction rather than all request information. These results separate early readability, natural strength, causal steering, and later content dependence.

Abstract

Claim Attribution in Language Models

Explore similar work

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

Position Matters More Than Channel: Talking a Language Model Out of a Correct Answer

Cache2LoRA: Understanding and Improving Context Internalization