We ask how much an autoregressive transformer retains of its prompt after generating t tokens. Consider two different prompts followed by the same continuation text, so that any difference in their hidden states comes from the prompt alone, and we find that the distance between their hidden states decays approximately as a power law t−α, where α=1 is the rate at which a fixed prompt's share of uniform attention shrinks and smaller values mean longer memory. Trained models forget fast (α≈0.55--0.7), untrained ones slowly (α≈0.2--0.5). We prove that this offset is O(P(logt)L−1/t) for a prompt of length P and depth L. The bound holds for finite-depth pre-LN softmax transformers with bounded attention logits, under one additional condition: the pre-norm activations stay away from zero. The mechanism is geometric: the prompt's direct influence is diluted as the context grows, and relay --- continuation tokens passing on what earlier tokens read from the prompt --- can only add logarithmic factors. This is what we call the conditional no-hair theorem: given the text generated so far, the prompt's influence on the next token vanishes as t grows, so any property of the prompt that survives, such as its language, must be carried by the text itself. An exactly solvable linear relay model shows that the bound cannot be improved and reproduces the rise of the local exponent towards 1 seen in untrained networks. Cancelling the dilution with a logit bias stops the decay early in training and reverses it after, so dilution is necessary for the forgetting we measure; untrained networks and the solvable model show it is also sufficient.
Figures & tables
Figure 1: Decay of the prompt offset under a shared continuation (Pythia-160m, real-corpus pairs, unbiased signal normalised to the first window; bands: 95% bootstrap over pairs). Dotted: layer 1; solid: last layer. Training makes the deep-layer offset decay much faster; the dashed line is α=1 .
Pythia-160m (12 layers)
L1
L2
L3
L6
L9
L11
init (step 0)
0.82 [.81,.82]
0.65 [.65,.65]
0.57 [.57,.58]
0.47 [.47,.47]
0.42 [.42,.43]
0.40 [.40,.41]
trained (143k)
1.05 [1.04,1.07]
0.92 [.90,.93]
0.63 [.57,.71]
0.59 [.55,.62]
0.64 [.62,.67]
0.66 [.64,.68]
Pythia-410m (24 layers)
L1
L2
L6
L12
L18
L23
init (step 0)
0.82 [.81,.82]
0.65 [.65,.66]
0.47 [.47,.48]
0.40 [.39,.40]
0.36 [.36,.37]
0.35 [.34,.35]
Table 1: Decay exponent α of the prompt offset, hand-written scenes (20 scenes, 16 continuations each), te∈[16,1920] , windows labelled by their centre, plug-in signal, 95% bootstrap intervals over scenes. ∣β∣≤0.07 in all cells.
Pythia-160m, hand-written scenes
layer l
2
3
6
9
11
fitted k
1.07
1.96
3.59
4.65
5.10
relay rounds l−1
1
2
5
8
10
R2
0.985
0.996
0.995
0.993
0.996
Pythia-410m, hand-written scenes
layer l
2
6
12
18
23
Table 2: Effective logarithmic power k and accumulated gain Gl , fitted from the local exponents of the untrained models (windows labelled by their centre). The Gl are effective parameters of the small-gain form; the estimates from the exact solution, which differ because the first gains are not small, are in Section 4.5 .
Figure 2: Local exponents between consecutive octaves. Left: every layer of untrained Pythia-1b; the exponent rises towards 1 in every layer and decreases with depth; dashed: the small-gain form of Corollary 2 , fitted as (1−αloc)−2=u/Gl+c (the offset c absorbs the finite- u corrections, Section 4.5 ). Right: the trained model, where no systematic trend remains in the deep layers.
init (step 0)
trained (step 143000)
layer
detail
para
lang
unrel.
detail
para
lang
unrel.
L1
0.75
0.74
0.69
0.73
0.92
0.98
0.99
0.97
L2
0.59
0.58
0.54
0.57
0.80
0.81
0.80
0.82
L3
0.52
0.51
0.47
0.50
0.67
0.52
0.56
0.67
L6
0.42
0.42
0.39
0.41
0.61
0.54
0.54
0.63
L9
0.39
0.38
0.35
0.37
0.67
0.58
0.61
0.67
Table 3: α on real-corpus prompt pairs, Pythia-160m, 100 pairs per axis, unbiased signal, windows labelled by their centre. 95% intervals over pairs have half-width ≤0.02 at initialisation and ≤0.06 after training; ∣β∣≤0.08 .
model
L1
L2
L3
last
160m
init
0.73
0.57
0.50
L6 0.41
L9 0.37
L11 0.36
trained
0.97
0.81
0.61
L6 0.58
L9 0.63
L11 0.66
1b
init
0.73
0.57
0.50
L4 0.46
L8 0.38
L12 0.35
L15 0.33
trained
0.87
0.79
0.71
L4 0.69
L8 0.61
L12 0.61
L15 0.58
Qwen-0.5B
init
0.63
0.45
0.37
L6 0.28
L12 0.22
L18 0.20
L23 0.18
trained
0.74
0.69
0.69
L6 0.62
L12 0.55
L18 0.54
L23 0.58
Table 4: α on real-corpus prompt pairs (mean over the four axes; per-axis 95% intervals have half-width ≤0.06 ), unbiased signal, windows labelled by their centre. Pythia-160m and Pythia-1b: te∈[16,1024] ; Qwen2.5-0.5B and 1.5B: te∈[16,2048] . At initialisation the two Pythia sizes agree at matched layers.
Figure 3: α against the layer index (mean over the four axes, real-corpus pairs). Untrained models (orange) depend on the layer index alone: the two Pythia sizes coincide; trained models (green) are flat beyond the first layers.
contiguous WikiText
shuffled sentences
model
layer
detail
para
unrel.
detail
para
unrel.
160m, init
L1
0.75
0.74
0.73
0.75
0.74
0.73
L11
0.37
0.37
0.35
0.37
0.37
0.35
160m, trained
L1
0.92
0.98
0.97
0.93
0.99
0.98
L11
0.69
0.59
0.70
0.61
0.54
0.66
1b, trained
L1
0.90
0.86
0.88
0.90
0.86
0.88
Table 5: α with contiguous WikiText continuations and with WikiText sentences in random order (real-corpus pairs, unbiased signal, centre-labelled windows).
Figure 4: Path decomposition (Pythia-160m, real-corpus pairs). Left: share of the late offset carried by relay. Right: decay exponents of the direct (circles, solid) and relay (squares, dashed) paths. At initialisation relay carries almost all of the late offset and barely decays.
step
L1
L2
L3
L6
L9
L11
0
measured
0.71
0.56
0.49
0.40
0.36
0.35
paths (both)
0.70
0.53
0.46
0.38
0.34
0.32
αdirect / αrelay
0.70 / 0.42
0.69 / 0.18
0.69 / 0.13
0.68 / 0.06
0.68 / 0.02
0.68 / 0.01
relay share
0.02
0.73
0.87
0.95
0.97
0.97
512
measured
0.70
0.74
0.79
0.77
0.66
0.67
paths (both)
0.69
0.73
0.79
0.73
0.60
0.61
Table 6: Path decomposition on real-corpus prompt pairs (Pythia-160m, 25 pairs per axis, four axes pooled, 8 WikiText continuations per pair, te∈[16,1024] , centre-labelled windows, unbiased signal for every component). Cumulative attention writes of blocks 0..l−1 reproduce the measured α within 0.06 at every layer and checkpoint. Relay share: relay part of the squared offset in the last window. Bootstrap half-widths of the measured α are ≤0.04 .
step
L1
L2
L3
L6
L9
L11
0
0.73
0.57
0.50
0.41
0.37
0.36
128
0.73
0.57
0.51
0.43
0.41
0.41
256
0.72
0.62
0.62
0.57
0.52
0.52
512
0.72
0.76
0.81
0.77
0.67
0.68
1000
0.72
0.75
0.79
0.67
0.63
0.66
4000
0.96
0.95
0.83
0.60
0.64
0.62
Table 7: α across training, Pythia-160m, real-corpus prompt pairs (100 per axis, mean over the four axes, the same pairs at every checkpoint), te∈[16,1024] , centre-labelled windows, unbiased signal. Per-axis 95% intervals have half-width ≤0.06 .
Figure 5: Training dynamics of Pythia-160m on the same 400 prompt pairs. Left: α per layer. Middle: normalised attention entropy. Right: attention mass on token 0. The shaded band marks steps 128–512, where forgetting sets in and attention beyond layer 1 leaves the uniform regime; the first layer and the sink change only after step 1000. Step 0 is drawn at the left edge.
Figure 6: Left: α per layer before (solid) and after (dotted) per-head compensation of the dilution, at three checkpoints. Right: retention against cost at the end of training; the last-layer α against the perplexity change for a boost on the prompt and for an equally strong placebo boost.
step
L1
L2
L3
L6
L9
L11
0
original
0.71
0.56
0.49
0.40
0.36
0.35
uniform
0.10
0.09
0.08
0.07
0.06
0.06
per-head
0.07
0.06
0.06
0.05
0.05
0.05
512
original
0.70
0.74
0.79
0.77
0.66
0.67
uniform
0.10
0.17
0.23
0.28
0.25
0.26
per-head
0.07
0.08
0.09
0.11
0.12
0.13
Table 8: α under dilution compensation on real-corpus prompt pairs (same setting as Table 6 ). Bootstrap half-widths ≤0.05 . Per-head calibration: residual median head slope −0.07 (step 0) and −0.08 (step 512), −0.33 (10–90%: [−1.16,−0.10] ) at step 143000.
boost on the prompt
placebo boost
s
α L6
α L11
Δ PPL
α L6
α L11
Δ PPL
0
0.56
0.65
—
0.56
0.65
—
0.25
0.45
0.58
+0.7%
0.53
0.64
+0.2%
0.5
0.31
0.48
+2.6%
0.48
0.63
+1.3%
0.75
0.20
0.36
+7.1%
0.41
0.59
+4.2%
1
0.11
0.25
+16.1%
0.34
0.53
+10.3%
Table 9: Trade-off of the uniform boost slogPq+1 on Pythia-160m (step 143000), real-corpus pairs (25 per axis, four axes pooled; centre-labelled windows, unbiased signal; α intervals ≤±0.06 ). Δ PPL: perplexity change on WikiText continuations of 40 real-corpus prompts. The placebo boosts a same-length span at the start of the continuation.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
init (step 0)
trained (step 143000)
input of
min
0.01%
median
min
0.01%
median
block 0 (embeddings)
0.021
0.021
0.023
0.018
0.018
0.028
block 1
0.112
0.112
0.127
0.236
0.241
0.325
block 2
0.162
0.162
0.184
0.279
0.285
0.402
block 3
0.203
0.208
0.229
0.320
0.331
0.477
block 6
0.305
0.307
0.335
0.428
0.436
0.616
Appendix
Table 10: ∥Πh∥/d for Pythia-160m. The theorem uses blocks ≥1 and the final norm: r/d=0.112 (init) and 0.236 (trained).
Transformer attention lets models relate tokens by weighting information from other positions, enabling context-dependent sequence and multimodal processing.
Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from prior tasks, which is often impractical. By contrast, language models can sample from their own training distribution, and we show that these self-generated samples serve as effective replay data, nearly eliminating forgetting. We find that forgetting nonetheless persists when the model has little remaining capacity: models pretrained close to saturation cannot absorb new information without overwriting prior knowledge. When capacity is not the limiting factor, low learning rates reduce forgetting but require substantially more training steps. Replay breaks this tradeoff, enabling fast, high-learning-rate finetuning without forgetting.
Autoregressive pretraining increasingly draws on heterogeneous data, making it important to understand how a model learns from an individual token. The next-token prediction objective naturally identifies a token's contribution with its own loss. However, each token is not only a prediction target but also context for what follows. Using controlled corruption, we decouple these two roles and find a reversal: making a noisy token easier to predict reduces its damage as a target but increases it as context. The same decoupling helps explain text generated by language models: generation selects each token by its fit to the prefix, while its role as context is never tested against an independently determined continuation, because that continuation is generated to fit it. At known corrupted positions, acting through the context can reduce damage that removing the token's own loss does not. Understanding and controlling what a model learns from a token therefore requires decoupling its roles.
Suqin Yuan, Runqi Lin, Kevin Qinghong Lin +4
University of Sydney · University of Oxford · Southeast University
Mid-prefill pruning can reduce the sequence processed by deeper transformer layers, but attention concentration alone does not certify that discarded context is dispensable. We formulate EntroPrefill as a Renyi-guided proposal mechanism coupled to explicit constraints on discarded attention mass. Sink-isolated, regularized head pooling respects grouped-query attention while exposing a quantitative trade-off between specialization and worst-head coverage. We derive a mixture-to-head deletion envelope, a computable upper bound on feasible token removal, and a finite-sample observer guarantee that remains valid when the pruning layer is selected adaptively. We then establish a conditional transformer perturbation bound with explicit sufficient Lipschitz constants and a first-token decision-margin corollary. A counterexample shows why shallow observations alone cannot imply an unconditional future-output guarantee. The systems analysis distinguishes query-head unions, physical page allocation, and KV-transfer payload, and gives an arithmetic break-even condition for pruning. This manuscript is theoretical in scope: it defines the procedure, its assumptions, and its formal limits, but does not report measured acceleration or task-accuracy preservation. Experiments are reserved for subsequent validation of the assumptions, approximation tightness, and end-to-end resource trade-offs.