Late Attention Layers Alone Can Copy Entity Tokens, but Not Without Attending to Their Context

Muyu He; Yuchen Liu; Ran Tao; Li Zhang

cs.CLSep 28, 2026

Late Attention Layers Alone Can Copy Entity Tokens, but Not Without Attending to Their Context

Authors: Muyu He, Yuchen Liu, Ran Tao, Li Zhang

Organizations: Independent · University of Pennsylvania · Drexel University

Abstract

Large language models (LLMs) reliably perform entity copying, in which a model copies tokens referring to an entity, termed entity tokens, from the prompt into its output to answer a question. Although entity copying is straightforward for most LLMs, existing research does not provide a systematic account of which layers specialize in this fundamental task or how other tokens in the same sequence, termed context tokens, influence the model's ability to copy the entity tokens. To address these questions, we conduct experiments on Qwen3-8B using two novel methods: genie-in-a-bottle, which controls exactly which layers can participate in an entity-copying task, and attention lobotomy, which cuts off specific tokens' attention to entity tokens without affecting the remaining attention distribution. We find that two distinct groups of layers in the second half of the model are both necessary and sufficient for entity copying. Moreover, in addition to the decoding position's attention to entity tokens, context tokens' attention to entity tokens also proves necessary for copying the exact tokens, even though context tokens do not store entity information themselves unless they satisfy particular semantic properties. Our findings establish the critical role of late layers in entity copying under the guidance of context tokens, calling for future work on how models propagate and consume entity information.

Figures & tables

Figure 1: Two methodologies for studying entity copying. (a) Genie-in-a-bottle confines access to the entity token

E

to the layer window from

\ell_{\mathrm{start}}

\ell_{\mathrm{finish}}

by performing three separate forward passes before, within, and after the window. Only the second pass, within the window, enables attention to

E

. (b) Attention lobotomy removes specific attention pathways

x\to y

to study their effect on entity copying. The effective attention weights for selected destination tokens

y

are zeroed after softmax.

Figure 2: Two of the five prompt templates, shown with the same entity and expected answer.

Figure 3: The general_success_rate when only a fixed window of attention layers can attend to entity tokens, for different window spans.

Figure 4: Output distribution after removing attention to entities within a fixed window.

Figure 5: Comparison of general_success_rate with attention to the control entity enabled versus disabled outside the designated window.

Figure 6: (a) Breakdown of output categories for each template after removing context tokens’ attention to entities. (b) For the two templates with the most unsuccessful outputs, the decoding position’s attention to entities decreases sharply after context tokens stop attending to them.

Figure 7: (a) Restoring context tokens’ attention to entity tokens in just two layers improves subsequent positions’ copying success. (b) With attention to entity tokens disabled and attention to context tokens intact, the model generates mostly wrong names or non-name responses.

Figure 8: Number of successful outputs under four context conditions with attention to entity tokens removed.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Template	Prompt
DF	Remember this fact: the person’s name is {entity}. Question: What is the person’s name? Respond with only the name. Answer:
PIL	Among apple, mouse, {entity}, and flute, exactly one item is a person’s name. Respond with only that person’s name. Answer:
F	Here is a fact: {entity} is my friend. Question: What is my friend’s name? Respond with only the name. Answer:
VR	The visitor signed the register with the name {entity}. Question: What name did the visitor write? Respond with only the name. Answer:
NB	The name printed on the badge is {entity}. Question: What name is printed on the badge? Respond with only the name. Answer:

Appendix

Table 1: Complete prompt templates. Line breaks within each prompt are preserved.

One token (50 entities)		Two tokens (50 entities)
Einstein	Picasso	Emily Dickinson	Vera Rubin
Newton	Gandhi	Mary Shelley	John Locke
Darwin	Mandela	Oscar Wilde	Adam Smith
Tesla	Lincoln	Victor Hugo	Benjamin Franklin
Euler	Churchill	Toni Morrison	George Washington
Gauss	Thatcher	Barack Obama	Brad Pitt

Appendix

Table 2: Complete entity lists, grouped by entity-span length.

Explore similar work

To Copy or Not to Copy: Copying Is Easier to Induce Than Recall
Jan 17, 2026Mehrdad Farahani, Franziska Penzkofer, Richard JohanssonFactual Recall Precision Recall
1Chalmers University of Technology · University of Gothenburg
Do Language Models Track Entities Across State Changes?
May 28, 2026Zilu Tang, Qiao Zhao, Gabriel Franco +4Entities State-Tracking
Department of Computer Science, Boston University, Boston, USA · Department of Data Science, Monash University, Indonesia · Faculty of Computer Science, University of Vienna, Austria +1
Learning When to Attend: Conditional Memory Access for Long-Context LLMs
Mar 18, 2026Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato +4Efficient Long-Context Inference Time-To-First-Token
Department of Electrical and Computer Engineering, Purdue University, USA · AWS Agentic AI, USA

Jan 17, 2026, cs.CLJ/K move · Enter open · S save

To Copy or Not to Copy: Copying Is Easier to Induce Than Recall

Mehrdad Farahani, Franziska Penzkofer, Richard Johansson

1Chalmers University of Technology · University of Gothenburg

Factual Recall Precision Recall

Open paper AskPDF arXiv

Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice by extracting an \emph{arbitration vector} from model activations on a curated dataset designed to disentangle (i) irrelevant contexts that elicit parametric recall and (ii) relevant but false contexts that elicit copying. The vector is computed as the residual-stream centroid difference between these regimes across 27 relations, and is injected as an additive intervention at selected layers and token spans to steer behavior in two directions: Copy

\rightarrow

Recall (suppressing context use) and Recall

\rightarrow

Copy (inducing the model to copy any token from the context). Experiments on three architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy reactivation'' process that can be triggered at different locations in the input, while restoring recall is a suppression'' process that is more fragile and strongly tied to object-token interventions.

Abstract

Explore similar work

To Copy or Not to Copy: Copying Is Easier to Induce Than Recall

Do Language Models Track Entities Across State Changes?

Learning When to Attend: Conditional Memory Access for Long-Context LLMs