A Step Towards Forgetting: Optimiser History and the Loss of Answer Mass

Valeria Ruscio; Seth Nabarro; Keiran Thompson

cs.CLOct 2, 2026

A Step Towards Forgetting: Optimiser History and the Loss of Answer Mass

Authors: Valeria Ruscio, Seth Nabarro, Keiran Thompson

Organizations: Intuition Machines Inc

Abstract

During fine-tuning, a language model can assign less probability to previously learned answers even when the current gradient acts to preserve that probability. With momentum, each update also carries gradients computed at earlier model states, and these stored contributions can push the model in the opposite direction. We investigate how this optimiser memory contributes to forgetting by separating old-task loss into confusion among its answers and leakage of probability outside the answer set. Across three language-model families, answer mass consistently declines while discrimination among old answers usually improves: the model becomes less likely to produce answers that it can still distinguish correctly. Decomposing Adam updates reveals opposing contributions to this loss of answer mass. Over training, accumulated history favours leakage, while the current gradient opposes it. Resolving history by age shows that the harmful contributions come mainly from older gradients of the new task, whereas recent gradients tend to protect the old answers. Changes in history's effect are dominated by its orientation relative to the old-task gradient. Interventions that reset momentum while matching the initial update norm establish that stored history affects retention, with state-dependent immediate effects and lower final old-task loss over longer Adam continuations, mainly through recovered answer mass. Finally, integration along finite updates shows that most sampled large loss increases are captured by local projections, while curvature along history amplifies some events. Together, these findings reveal how an optimiser's memory can erode learned behaviour even as its current gradient acts to preserve it.

Figures & tables

Figure 1: One update, two changes in retention. Schematic loss surfaces over the same parameter plane

\theta+uh+vc

, in which the orange and blue curves trace the losses along the history-only and current-only displacements and the purple curve follows their sum. The combined update increases answer-mass leakage while reducing conditional confusion, and curvature bends each trajectory away from its initial slope. Arrows on the floor show the parameter displacements, and vertical arrows compare the loss changes at the endpoints. The surfaces are illustrative, and the reconstruction residual is omitted.

Figure 2: Two forms of retention change after 128 new-task updates. Leakage increases in all 18 trajectories, while conditional confusion decreases in 16 of them. Bars show the two loss components on the full 128-text test population and black markers show their sum, with D/S denoting disjoint and shared answer codebooks, which are paired within seed.

Model	Seed / answers	$\Delta L_{\ell}$	$H_{\ell}$	$C_{\ell}$	$E_{\ell}$	$\Delta L_{\rm conf}$
Pythia-410M	51 / disjoint	+4.540	+4.591	-0.675	+0.624	-0.020
Pythia-410M	51 / shared	+0.084	+0.176	-0.225	+0.134	+1.339
Pythia-410M	52 / shared	+0.136	+0.667	-1.131	+0.599	+0.293
SmolLM2-135M	51 / disjoint	+0.217	+0.277	-0.150	+0.089	-0.650
SmolLM2-135M	51 / shared	+0.045	+0.078	-0.072	+0.039	-0.505
SmolLM2-135M	52 / disjoint	+1.878	+1.859	-0.195	+0.214	-0.387

Table 1: History and current-gradient contributions on the fixed 32-text

A

-test subset. Values are sums over 128 updates in nats.

H_{\ell}=\sum_{t}g_{\ell,t}^{\top}h_{t}

and

C_{\ell}=\sum_{t}g_{\ell,t}^{\top}c_{t}

sum the local leakage projections along the recorded trajectory.

E_{\ell}=\Delta L_{\ell}-H_{\ell}-C_{\ell}

retains finite-step and displacement/numerical residuals.

Figure 3: Age and orientation of optimiser history. The left panel shows cumulative leakage projections by origin and age, where positive values favour leakage, and the right panel shows the absolute-contribution shares for changes in the total-loss history projection, computed with Equation 10 . The left panel therefore describes accumulated leakage contributions, while the right panel describes how the total-loss projection varies across updates.

Model	$n$	Orientation	History motion	Adaptive scaling
Pythia-410M	3	77.19–83.69	65.71–75.56	1.82–3.21
SmolLM2-135M	4	77.35–82.77	68.03–73.41	1.80–2.62
Qwen3-0.6B	6	77.89–83.97	59.10–73.56	1.84–3.23

Table 2: Changes in history’s total-loss projection. Ranges across trajectories, in percent. Orientation uses the three-term decomposition in Equation 10 ; history motion uses Equation 9 ; adaptive scaling is nested within history motion. The columns have separate denominators.

Optimiser	Seed	Step	$A$ : keep	$A$ : reset	$B$ : keep	$B$ : reset
Momentum SGD	23	8	+0.100279	-0.275243	-0.218030	-0.411795
Adam	21	8	+0.079993	-0.035992	+0.040147	-0.911440
Adam	22	8	-0.038177	-0.012278	-0.271431	-0.049603

Table 3: Illustrative fixed-step SmolLM2 history probes. Entries are immediate changes in test loss, and the reset displacement norms match the preserved norms within each block.

Model	Optimiser	Seeds	Forks	$\Delta A_{f}$	$\Delta B_{f}$	Confusion	Leakage
Pythia-410M	Adam	3	9	$-.127170$	$-.001890$	$+.099843$	$-.227013$
SmolLM2-135M	Adam	3	9	$-.266907$	$+.001482$	$-.071779$	$-.195129$
SmolLM2-135M	Momentum	2	6	$-.085724$	$+.030098$	$-.069111$	$-.016613$

Table 4: Differences between first-step block-matched history-reset branches and preserved-history branches, given as mean final test-loss differences after 32 updates, where the matching constrains only the first displacement. The numbers of acquired trajectories and of forks are shown separately.

Study	Component	Accepted	Unresolved	Large	FO	Amp.	Mix
P/S	Total	87	9	19	17	2	1
P/S	Confusion	81	15	5	5	0	1
P/S	Leakage	89	7	15	13	2	0
Q	Total	6	0	2	2	0	0
Q	Confusion	6	0	0	0	0	0
Q	Leakage	6	0	2	2	0	0

Table 5: Uniform test paths on fixed 32-text subsets. P/S: Pythia/SmolLM2, 96 paths per component; Q: Qwen3, six. Acceptance is component-specific. Large increases exceed 0.05 nats; FO and Amp. follow the ordered rule. Mix counts material mixed terms across all accepted paths. Sampling densities differ between studies.

Answers	$R_{A}^{hh}$	$R_{A}^{cc}$	$M_{A}$
Disjoint	$+0.055557$	$+0.000814$	$-0.006253$
Shared	$+0.082203$	$+0.013482$	$-0.054475$

Table 6: Weighted curvature contributions to total old-task loss at update 9 of Pythia seed 53. History curvature is positive and mixed curvature is protective under both codebooks. Values are in nats, and the closure for the full displacement is given in Table 43 .

Appendix figures & tables44 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Old-task loss along diagnostic interpolations of three recorded OLMo displacements, with dashed lines showing the initial linear prediction. The event on the left reverses its initially favourable direction, while the other events largely follow their initial predictions. All displayed paths pass their quadrature checks, and update 21 uses the refined evaluations described in Appendix E .

Checkpoint	Update	$\Delta L_{A}$	$s_{A}(0)$	$I_{A}^{h}$	$I_{A}^{c}$
OLMo-1B	21	+0.287872	-0.130264	+0.309796	-0.021920
Pythia-70M	61	+0.124563	+0.102805	+0.102987	+0.021562
Pythia-410M	65	+0.322116	+0.248560	+0.231985	+0.090131
Qwen2.5 Instruct	107	+0.161973	+0.155524	+0.042224	+0.119749
SmolLM2 base	73	+0.095240	+0.082083	+0.047804	+0.047435
SmolLM2 Instruct	64	+0.106675	+0.101283	+0.072903	+0.033771

Appendix

Table 7: Selected audited old-task paths, all from seed 1. The integrated channel contributions reconstruct the total change, and the small residual and quadrature errors are omitted from this table.

Figure 5: The highlighted OLMo reversal, evaluated at layer 0 on entities held out from the fitting of the rotation. The left panel shows old-task losses at the four fixed channel states before and after the fitted rotation, and the right panel shows endpoint loss changes under the fitted map and the comparison conditions, where the bars for random controls give means and ranges across four realisations. Negative changes indicate improved performance.

Repository	Revision identifier
allenai/OLMo-1B-hf	aee7752d9c08ee4775e9b0091426d8410e8f6a89
EleutherAI/pythia-70m	a39f36b100fe8a5377810d56c3f4789b9c53ac42
EleutherAI/pythia-410m	9879c9b5f8bea9051dcb0e68dff21493d67e9d4f
Qwen/Qwen2.5-0.5B	060db6499f32faf8b98477b0a26969ef7d8b9987
Qwen/Qwen2.5-0.5B-Instruct	7ae557604adf67be50417f59c2c2f167def9a775
HuggingFaceTB/SmolLM2-360M	f8027fd0eaeea54caa13c31d31b9fdc459c38b49

Appendix

Table 8: Model repositories and exact revisions used in the experiments, where the revision identifiers specify the recorded repository versions. The association-learning suite uses SmolLM2-360M, and the natural-text studies use SmolLM2-135M.

Dataset	Revision identifier
SST-2	8d51e7e4887a4caaa95b3fbebbf53c0490b58bbb
AG News	eb185aade064a813bc0b7f42de02595523103ca4

Appendix

Table 9: Dataset revisions for the natural-history text study.

Target	Loss AUROC	+Proj. AUROC	Gain	AP	Brier
Pythia-70M	0.721	0.845	+0.124	0.380 / 0.563	0.084 / 0.073
Pythia-410M	0.615	0.797	+0.182	0.368 / 0.563	0.158 / 0.133
Qwen2.5 base	0.686	0.822	+0.136	0.397 / 0.519	0.149 / 0.121
Qwen2.5 Instruct	0.667	0.804	+0.137	0.352 / 0.520	0.137 / 0.120
SmolLM2 base	0.861	0.957	+0.096	0.491 / 0.575	0.075 / 0.066
SmolLM2 Instruct	0.801	0.913	+0.112	0.351 / 0.388	0.072 / 0.065

Appendix

Table 10: Transfer of the frozen registry predictor, compared with the specified fitted loss baseline. The comparator uses the pre-update feature timestamp, and the latest completed validation-loss baseline appears in Table 21 . All metrics are macro means across three seeds, the AP and Brier columns give loss-only / loss-plus-projection scores, and higher AUROC and AP and lower Brier are better.

Checkpoint	Lead	Features	AUROC	AP	Brier
OLMo-1B	0	Loss	0.843186	0.618288	0.127826
OLMo-1B	0	Loss + projection	0.946781	0.858918	0.083301
OLMo-1B	0	Static geometry	0.963795	0.860140	0.058668
OLMo-1B	0	Geometry + changes	0.959504	0.870068	0.055377
OLMo-1B	1	Loss	0.693560	0.344566	0.177768
OLMo-1B	1	Loss + projection	0.792133	0.525299	0.149386

Appendix

Table 11: Macro prediction metrics over three seeds each. OLMo uses exploratory leave-one-seed-out fits and all other checkpoints use frozen OLMo fits, with lead 0 serving as a current-update diagnostic and lead 1 as an advance warning.

Checkpoint	Lead	Features	AUROC	AP	Brier
Qwen2.5 base	0	Loss	0.847762	0.557215	0.118395
Qwen2.5 base	0	Loss + projection	0.986020	0.924421	0.070667
Qwen2.5 base	0	Static geometry	0.985373	0.925991	0.046151
Qwen2.5 base	0	Geometry + changes	0.980436	0.887451	0.049490
Qwen2.5 base	1	Loss	0.685923	0.397295	0.149246
Qwen2.5 base	1	Loss + projection	0.821588	0.518821	0.120706

Appendix

Table 12: Macro prediction metrics over three seeds each. OLMo uses exploratory leave-one-seed-out fits and all other checkpoints use frozen OLMo fits, with lead 0 serving as a current-update diagnostic and lead 1 as an advance warning. (continued)

Checkpoint	Lead	Features	AUROC	AP	Brier
SmolLM2 Instruct	0	Loss	0.961628	0.492285	0.054131
SmolLM2 Instruct	0	Loss + projection	0.998380	0.980903	0.041752
SmolLM2 Instruct	0	Static geometry	0.995311	0.950626	0.037380
SmolLM2 Instruct	0	Geometry + changes	0.999145	0.989933	0.021764
SmolLM2 Instruct	1	Loss	0.800813	0.351235	0.071948
SmolLM2 Instruct	1	Loss + projection	0.912504	0.388456	0.064845

Appendix

Table 13: Macro prediction metrics over three seeds each. OLMo uses exploratory leave-one-seed-out fits and all other checkpoints use frozen OLMo fits, with lead 0 serving as a current-update diagnostic and lead 1 as an advance warning. (continued)

Checkpoint	Seed	N	Events	AUROC	AP	Brier
OLMo-1B	1	126	29	0.6715 / 0.7540	0.3827 / 0.5432	0.1711 / 0.1489
OLMo-1B	2	126	14	0.6767 / 0.7621	0.1710 / 0.2542	0.1613 / 0.1368
OLMo-1B	3	126	35	0.7325 / 0.8603	0.4800 / 0.7784	0.2009 / 0.1625
Pythia-70M	1	126	7	0.4958 / 0.6579	0.0638 / 0.1474	0.0829 / 0.0743
Pythia-70M	2	126	10	0.8241 / 0.9181	0.5467 / 0.7661	0.0835 / 0.0732
Pythia-70M	3	126	13	0.8421 / 0.9578	0.5310 / 0.7755	0.0857 / 0.0727

Appendix

Table 14: Per-seed advance-warning comparison. Paired entries give loss-only / loss-plus-projection scores, and N is the number of evaluated target updates.

Checkpoint	Seed	Anchor $L_{A}$	Final $L_{A}$	Final $L_{B}$	Large	Reversals
OLMo-1B	1	0.225999	0.844272	0.072626	29	2
OLMo-1B	2	0.137821	0.447399	0.151627	14	0
OLMo-1B	3	0.157457	0.627281	0.128464	35	0
Pythia-70M	1	0.092410	0.602690	0.220334	7	0
Pythia-70M	2	0.215931	0.515659	0.336593	12	0
Pythia-70M	3	0.213590	0.418440	0.186810	13	0

Appendix

Table 15: Per-seed task losses and event counts after 128 task-

B

updates.

Checkpoint	Seed/update	$\Delta L_{A}$	$g_{A}^{\top}\delta$	$R_{A}$
OLMo-1B	1/21	+0.287872	-0.130264	+0.418136
OLMo-1B	1/76	+0.064027	-0.038522	+0.102548
Pythia-410M	3/18	+0.051748	-0.371233	+0.422981
Qwen2.5 Instruct	1/96	+0.057541	-0.056290	+0.113831

Appendix

Table 16: All four large sign reversals on the old task in the measured trajectories. Only OLMo 1/21 has a corresponding audited dense path in this event set.

Checkpoint	Seed/update	Task	Points	Quad.	Slope	Closure
OLMo-1B	1/021	A	157	Pass	Pass	+3.82e-06
OLMo-1B	1/021	B	217	Pass	Pass	+7.48e-06
OLMo-1B	1/048	A	33	Pass	Pass	+1.70e-07
OLMo-1B	1/048	B	33	Pass	Pass	-9.20e-07
OLMo-1B	2/011	A	1025	Fail	Pass	+8.39e-07
OLMo-1B	2/011	B	761	Fail	Pass	+2.60e-05

Appendix

Table 17: Path checks, using the refined OLMo records for updates 1/21 and 2/11. The checks concern task-population means, and acceptance requires endpoint closure, component convergence, and slope agreement.

Checkpoint	Seed/update	Task	Points	Quad.	Slope	Closure
SmolLM2 base	1/028	A	33	Pass	Pass	+4.60e-07
SmolLM2 base	1/028	B	33	Pass	Pass	+2.09e-07
SmolLM2 base	1/073	A	33	Pass	Pass	-2.44e-07
SmolLM2 base	1/073	B	33	Pass	Pass	-5.61e-08
SmolLM2 Instruct	1/043	A	33	Pass	Pass	-2.13e-07
SmolLM2 Instruct	1/043	B	33	Pass	Pass	+3.94e-07

Appendix

Table 18: Path checks, using the refined OLMo records for updates 1/21 and 2/11. The checks concern task-population means, and acceptance requires endpoint closure, component convergence, and slope agreement. (continued)

Seed/update	$\alpha$	Task	$\delta^{\top}H\delta$	Sym.	FD	Last width
1/021	0.582031250	A	-127.072809	Pass	Pass	0.00250000
1/021	0.582031250	B	+20.523470	Pass	Pass	0.00500000
1/021	0.583984375	A	-127.870972	Pass	Pass	0.00250000
1/021	0.583984375	B	+27.130345	Pass	Pass	0.00250000
2/011	0.343383789	A	+0.053302	Pass	Pass	0.00125000
2/011	0.343383789	B	-46.336686	Pass	Fail	0.00015625

Appendix

Table 19: Refined OLMo curvature audits. A measurement passes when the finite differences agree and are stable across adjacent widths and mixed symmetry holds, and the width column gives the last attempted finite-difference width for each measurement.

Seed/update	Layer	$\Delta L_{A}$	$\Delta L_{B}$	A acc. (%)	Mean shift	$\Delta J_{A}$
1/020	0	+0.044355	-0.018944	100.0 / 100.0	+0.043797	+0.129258
1/020	1	+0.002779	+0.019591	100.0 / 100.0	+0.050376	+0.000436
1/020	9	+0.078667	+0.170582	100.0 / 93.8	-0.009604	+0.025977
1/021	0	-0.266076	-0.061470	50.0 / 100.0	-0.047052	+0.066994
1/021	1	+0.019199	+0.041272	50.0 / 50.0	-0.199902	+0.017846
1/021	9	-0.014137	+0.005740	50.0 / 50.0	-0.041969	-0.013997

Appendix

Table 20: Fitted-rotation endpoint effects at

\alpha=1

on 16 held-out entities. Loss changes compare patched with unpatched endpoints, accuracy entries are unpatched / patched percentages, the mean shift is the corresponding old-task control, and

\Delta J_{A}

is a separate interaction contrast.

Score	Seed 11	Seed 12	Seed 13
Previous actual projection	.8183	.8465	.9272
Previous task-relative cosine	.8081	.8556	.9274
Latest validation-loss increase	.8509	.8522	.9405
Frozen loss-history predictor	.7029	.7117	.8305
Previous population-gradient interference	.7551	.6218	.6777
Positive events	56	58	56

Appendix

Table 21: Pythia text-task forecast baselines on the same 119 eligible decisions per seed. The target is an increase of more than 0.05 nats in

A

-test loss at the next update, and all entries are AUROC. The fitted loss-history baseline differs from the latest available change in validation loss.

Condition	Seed 12	Seed 13
Natural	4.518 / .999	1.959 / 1.050
Projection-triggered	4.433 / .724	2.322 / 1.360
Loss-history-triggered	2.047 / .728	2.028 / 1.529
Constant quarter learning rate	2.575 / .990	1.929 / 1.090
Reset first moment, nominal rate	4.277 / .785	2.434 / .929
Reset first moment, quarter rate	2.190 / .891	1.877 / 1.137

Appendix

Table 22: Pythia text-policy comparison, where each cell gives the final

A/B

NLL, so that the two task losses together describe the trade-off between retention and acquisition.

Seed/update	$\Delta L_{A}$	Initial slope	Remainder	History	Current
12/8	+0.074220	-0.012870	+0.087090	+0.052256	+0.021964
12/9	+0.454895	—	—	+0.402912	+0.051983
13/6	-0.067790	—	—	-0.087281	+0.019484

Appendix

Table 23: Selected Pythia text paths. Entries are changes in old-task test loss, or contributions to them, in nats, and all paths pass their recorded integration and sampled Hessian checks.

Seed/update	History	First $\Delta L_{A}$	First $\Delta L_{B}$	Final $L_{A}$	Final $L_{B}$
12/9	Preserved	+0.454895	-0.086883	3.095976	2.553748
12/9	Block-matched reset	+0.194391	-0.248681	2.075273	2.046184
13/6	Preserved	-0.067790	-0.674917	1.404899	1.509092
13/6	Block-matched reset	+0.128456	+0.139535	1.130651	3.194577

Appendix

Table 24: Selected Pythia text branches. Reset rows match the initial displacement norm within blocks, and immediate changes and losses after eight branch updates are in nats.

Evaluation	Recorded endpoint effect
Seed 11, tested conditions	Largest absolute task-loss effect $\approx 5.14\times 10^{-4}$ nats
Seeds 12/13, twelve old-task sites	Range $[-0.000156,+0.002701]$ nats; three improvements

Appendix

Table 25: Pythia rotation replication. Endpoint effects compare patched and unpatched losses on the intervention evaluation populations.

Model	Optimiser	Seed	LR	Gate	$A_{0}$	$A_{f}$	$B_{f}$
Pythia-410M	Adam	21	5e-06	yes	1.054	1.729	0.525
Pythia-410M	Adam $\beta_{1}=0$	21	5e-06	yes	1.173	2.065	0.369
Pythia-410M	Momentum SGD	21	1e-02	yes	1.280	2.630	0.479
Pythia-410M	SGD	21	1e-02	yes	0.701	1.377	0.475
Pythia-410M	Adam	22	5e-06	yes	0.928	3.772	0.564
Pythia-410M	Adam $\beta_{1}=0$	22	5e-06	yes	1.192	2.316	0.634

Appendix

Table 26: All main optimiser confirmation trajectories, including the acquisition failure. Losses are full-vocabulary NLL in nats.

Figure 6: Same-update projection diagnostics and preceding-update forecasts in the optimiser suite. Points are acquisition-passed seeds and horizontal bars are seed means, and the change between diagnostic and forecasting performance is especially clear for Pythia. The line at .5 marks chance AUROC for an uninformative ranking, and scores below chance retain their predeclared orientation.

Seed	B labels	$A$ loss	Confusion	Leakage	Restricted	Full
21	disjoint	3.746	0.531	3.216	75.8	0.0
21	shared	1.470	1.120	0.350	57.8	53.9
22	disjoint	1.681	0.407	1.274	85.2	37.5
22	shared	1.367	0.722	0.645	70.3	43.8
23	disjoint	1.158	0.609	0.549	75.8	64.1
23	shared	1.801	0.571	1.230	71.1	23.4

Appendix

Table 27: SmolLM2 paired answer-codebook controls, in which the acquired

A

state is identical within each seed. Accuracy columns are percentages.

Model	Seed / answers	Observed	History	Current	Remainder
Pythia-410M	51 / disjoint	+4.519647	+4.677210	-0.894080	+0.736518
Pythia-410M	51 / shared	+1.423522	+2.014214	-1.046515	+0.455823
Pythia-410M	52 / shared	+0.428061	+1.001943	-1.301416	+0.727534
SmolLM2-135M	51 / disjoint	-0.432795	-0.259566	-0.288048	+0.114819
SmolLM2-135M	51 / shared	-0.460483	-0.184538	-0.528463	+0.252518
SmolLM2-135M	52 / disjoint	+1.490655	+1.574926	-0.353639	+0.269367

Appendix

Table 28: Decomposition of the total loss on the fixed 32-text test population. Values are in nats, and a negative observed change means a net improvement relative to the acquired anchor on this subset.

Model	Seed / answers	Orient.	Channel motion	Denom.	$\cos(h_{t},h_{t-1})$	$\cos(g_{t},g_{t-1})$
Pythia-410M	51 / disjoint	77.19	65.71	3.21	0.9046	0.9984
Pythia-410M	51 / shared	83.69	75.56	1.82	0.9035	0.9983
Pythia-410M	52 / shared	78.57	73.35	2.62	0.8992	0.9980
SmolLM2-135M	51 / disjoint	82.77	68.88	1.80	0.9050	0.9989
SmolLM2-135M	51 / shared	82.02	68.03	2.12	0.8966	0.9978
SmolLM2-135M	52 / disjoint	77.35	70.41	2.62	0.8998	0.9993

Appendix

Table 29: Cross-time geometry of history for the total old-task loss. The first three columns give the defined absolute-contribution shares in percent, taken from separate or nested decompositions, and the last two columns give median cosines between consecutive updates. The shares follow the attribution conventions of Appendix I .

Model	Optimiser	Seed	Loss	Combined	Previous proj.	Latest
Pythia-410M	Adam	41	.725313	.837812	.863750	.871563
Pythia-410M	Adam	42	.862448	.883289	.911330	.913225
Pythia-410M	Adam	43	.760782	.828167	.883423	.883423
SmolLM2-135M	Adam	41	.887046	.945764	.931421	.929628
SmolLM2-135M	Momentum	41	.575728	.706349	.720238	.745370
SmolLM2-135M	Adam	42	.847882	.892169	.900193	.903723

Appendix

Table 30: Next-update AUROC for an increase of more than

.05

nats in

A

-test loss, in the acquisition-passed trajectories of the matched-start study (seeds 41–43). “Loss” is the frozen multivariate loss-history model, and “Latest” is the raw latest increase in validation loss.

Qwen3 seed	Pre- $B$ accuracy (%)	Post- $B$ accuracy (%)
1	100	46.875
2	100	62.5
3	100	50
Event population	Loss increases	Sign reversals
All checkpoints, large events	421	4
OLMo, strict-positive events	200	17

Appendix

Table 31: Behavioural retention and reversal counts. Large events use the common threshold

\Delta L_{A}>0.05

, and strict-positive events use

\Delta L_{A}>0

Quantity	Value
Old-task loss before / after	$0.089042\;/\;0.376914$
Initial history projection $g_{A}^{\top}h$	$-0.089216$
Initial current projection $g_{A}^{\top}c$	$-0.041048$
Initial full slope	$-0.130264$
Observed old-task loss change	$+0.287872$
Finite-step remainder $R_{A}$	$+0.418136$

Appendix

Table 32: The highlighted OLMo reversal at seed 1, update 21. The components of the remainder sum to

R_{A}

up to rounding, and the curvature entries are local directional contractions at the stated location on the path.

Quantity	Update 59	Update 3
Old-task loss change	$+0.075012$	$+0.000294$
Old-task integrated history	$+0.061498$	$-0.000307$
Old-task integrated current	$+0.013514$	$+0.000601$
History share of old-task increase	$\approx 82\%$	—
New-task integrated history	$+0.033516$	—
New-task integrated current	$-0.006812$	—

Appendix

Table 33: Contrasting Qwen3 events for seed 1. The orientation term is a component of the finite-step remainder of the old task.

Quantity	Value
Pre-update / unpatched endpoint loss	$0.085549\;/\;0.376039$
Rotated endpoint loss	$0.109962$
Fraction of event loss increase recovered	$91.6\%$
Two-answer accuracy: pre / post / rotated	$100\%\;/\;50\%\;/\;100\%$
Entities with improved loss	$13/16$
Median entity loss change	$-0.273151$

Appendix

Table 34: The highlighted activation rescue at OLMo seed 1, update 21, layer 0, evaluated on 16 entities held out from fitting. Control effects are patched minus unpatched old-task loss at the full-update endpoint, and the geometric discrepancies compare the rotated and unrotated activation clouds.

Model	Optimiser	$n$	Same-update	Previous proj.	Latest loss
Pythia-410M	SGD	3	0.979	0.386	0.388
Pythia-410M	Momentum SGD	3	0.971	0.860	0.879
Pythia-410M	Adam $\beta_{1}=0$	3	0.966	0.433	0.417
Pythia-410M	Adam	3	0.987	0.919	0.921
SmolLM2-135M	SGD	3	0.867	0.436	0.446
SmolLM2-135M	Momentum SGD	2	0.900	0.790	0.855

Appendix

Table 35: Local diagnostics versus advance forecasting in the text optimiser study. Entries are equal-weight means of seed-level AUROC for an

A

-test increase above 0.05 nats.

Checkpoint	Final $L_{A}$ minus anchor	Final $L_{B}$	Large increases	Reversals
OLMo-1B	+0.466	0.118	78	2
Pythia-70M	+0.338	0.248	32	0
Pythia-410M	+0.787	0.136	76	1
Qwen2.5 base	+0.824	0.088	75	0
Qwen2.5 Instruct	+0.726	0.093	64	1
SmolLM2 base	+0.809	0.012	24	0

Appendix

Table 36: Retention and local events. Loss columns average three seeds and event counts sum their 384 updates, and reversals require both

\Delta L_{A}>0.05

and a negative initial slope.

Model	Seed / answers	Anchor (both)	Final full	Final restricted
Pythia-410M	51 / disjoint	87.50	0.00	63.28
Pythia-410M	51 / shared	87.50	57.03	59.38
Pythia-410M	52 / disjoint	84.38	0.00	78.12
Pythia-410M	52 / shared	84.38	39.84	46.09
Pythia-410M	53 / disjoint	86.72	0.00	81.25
Pythia-410M	53 / shared	86.72	32.03	80.47

Appendix

Table 37: Full-test accuracy before and after new-task training for Pythia and SmolLM2, in percent. Both accuracy measures share the same acquired anchor in these runs.

Model	Seed / answers	Task $A$	B ages 1–4	B ages 5–16	B ages 17+
Pythia-410M	51 / disjoint	+0.017229	-0.649152	+3.381657	+1.840978
Pythia-410M	51 / shared	+0.002387	-0.374941	+0.350112	+0.198164
Pythia-410M	52 / shared	+0.021419	-1.845042	+2.274017	+0.217016
SmolLM2-135M	51 / disjoint	-0.000818	-0.251938	+0.347463	+0.182692
SmolLM2-135M	51 / shared	-0.005015	-0.106745	+0.152827	+0.037062
SmolLM2-135M	52 / disjoint	+0.003544	+0.030012	+1.183401	+0.641751

Appendix

Table 38: Cumulative leakage projections by gradient age on the fixed 32-text test subset, in nats, summed over 128 updates. The contributions keep their original magnitudes, bin widths, and decay weights under the current preconditioner, and the recorded roundoff in the sums is below

1.4\times 10^{-7}

for Pythia and SmolLM2 and below

10^{-6}

for Qwen3.

Model	Seed / answers	Total	Confusion	Leakage
Pythia-410M	51 / disjoint	+4.703567	-0.099648	+4.803215
Pythia-410M	51 / shared	+1.269549	+1.164286	+0.105263
Pythia-410M	52 / disjoint	+4.027747	-0.830915	+4.858662
Pythia-410M	52 / shared	+0.297949	+0.051880	+0.246069
Pythia-410M	53 / disjoint	+6.158765	-0.285333	+6.444098
Pythia-410M	53 / shared	+1.418819	-0.185479	+1.604298

Appendix

Table 39: Endpoint changes on the full 128-text test set after 128 updates, where D/S denote disjoint and shared answers. All trajectories pass acquisition.

Seed	Answers	Orientation	Channel motion	Denominator change
61	disjoint	78.18	64.19	3.17
61	shared	83.34	73.23	2.63
62	disjoint	77.89	59.10	3.23
62	shared	77.93	69.92	2.22
63	disjoint	83.97	73.56	2.85
63	shared	78.25	69.78	1.84

Appendix

Table 40: Absolute-contribution shares, in percent, for changes in the total-loss history projection of Qwen. Orientation, channel motion, and denominator change use the separate or nested normalisations of Appendix I , so the columns are not additive.

Population	Selection	Component	N	Acc.	U	Small	FO	Amp.	Mix
Validation	All	Total	126	116	10	85	29	2	1
Validation	All	Confusion	126	113	13	108	5	0	1
Validation	All	Leakage	126	118	8	93	23	2	0
Validation	Uniform	Total	96	90	6	74	14	2	1
Validation	Uniform	Confusion	96	88	8	88	0	0	1
Validation	Uniform	Leakage	96	91	5	75	14	2	0

Appendix

Table 41: Path counts for each component. N: completed; Acc.: accepted by the audit; U: unresolved; Small: ordinary or nonlarge; FO: first-order dominated; Amp.: nonlinear amplification; Mix: material mixed term. The three event columns partition the accepted paths, since no accepted path is classified as a reversal or as another large increase, while Mix is an independent flag. Uniform rows are subsets of the corresponding rows for all selected paths.

Answers	Observed	Initial projection	Remainder	Mixed term
Disjoint	+0.099559	+0.049355	+0.050205	-0.006253
Shared	+0.139375	+0.098201	+0.041175	-0.054475

Appendix

Table 42: Paired uniform paths for Pythia seed 53/update 9 on the 32-text test subset, all passing every component audit. Positive remainders coexist with protective mixed terms, and values are rounded to six decimals.

Answers	$hh$	$cc$	$hc+ch$	$dd$	Closure
Disjoint	+0.055557	+0.000814	-0.006253	+0.050176	-0.000028
Shared	+0.082203	+0.013482	-0.054475	+0.041202	+0.000028

Appendix

Table 43: Weighted curvature integrals for the fully audited paired amplification examples, where

hh

and

cc

denote the diagonal directional terms,

hc+ch

their mixed contribution, and

dd

the integral over the full displacement, with closure defined as

dd-R

. Values are in nats, and the small residuals in the direction sums are retained in the export.

Seed	Answers	Step	Select.	$\Delta$	$P$	$R$	$M$
61	D	73	U	-0.036533	-0.062171	+0.025638	+0.005214
61	D	103	L	+0.066291	+0.062903	+0.003387	+0.001178
61	S	73	U	+0.070418	+0.063665	+0.006753	-0.005559
61	S	94	L	+0.129903	+0.130472	-0.000569	-0.000319
62	D	51	L	+0.176322	+0.164273	+0.012049	+0.001512
62	D	121	U	-0.074695	-0.073804	-0.000890	+0.000330

Appendix

Table 44: Qwen3 held-out paths for total loss. D/S: disjoint/shared answers; U/L: uniform or validation-selected large-

A

case.

\Delta

is the observed change,

P

the initial projection,

R=\Delta-P

, and

M

the mixed contribution, all in nats and rounded to six decimals, and all component audits pass.

Population	Selection	Component	Small	FO
Validation	Uniform	Total	4	2
Validation	Uniform	Confusion	6	0
Validation	Uniform	Leakage	4	2
Validation	Large- $A$	Total	0	6
Validation	Large- $A$	Confusion	4	2
Validation	Large- $A$	Leakage	1	5

Appendix

Table 45: Qwen3 component classifications by population and selection, where every row contains six completed and accepted paths. Small: ordinary or nonlarge; FO: first-order dominated. No reversal, amplification, other large increase, or material mixed term occurs, and the large-

A

cases are selected on validation loss.

Seed	Answers	Step	Select.	$\Delta$	$P$	$R$	$M$
61	D	73	U	-0.030631	-0.028159	-0.002472	-0.000566
61	D	103	L	-0.003343	-0.003699	+0.000356	+0.000086
61	S	73	U	+0.015562	+0.015279	+0.000283	-0.000209
61	S	94	L	-0.026391	-0.027637	+0.001247	+0.000602
62	D	51	L	-0.027878	-0.028722	+0.000843	+0.000314
62	D	121	U	+0.012740	+0.011925	+0.000816	-0.001797

Appendix

Table 46: Qwen3 held-out paths for confusion loss. D/S: disjoint/shared answers; U/L: uniform or validation-selected large-

A

case.

\Delta

is the observed change,

P

the initial projection,

R=\Delta-P

, and

M

the mixed contribution, all in nats and rounded to six decimals, and all component audits pass.

Seed	Answers	Step	Select.	$\Delta$	$P$	$R$	$M$
61	D	73	U	-0.005902	-0.034012	+0.028110	+0.005780
61	D	103	L	+0.069634	+0.066602	+0.003032	+0.001093
61	S	73	U	+0.054856	+0.048386	+0.006470	-0.005350
61	S	94	L	+0.156293	+0.158109	-0.001816	-0.000921
62	D	51	L	+0.204200	+0.192994	+0.011205	+0.001198
62	D	121	U	-0.087435	-0.085729	-0.001706	+0.002127

Appendix

Table 47: Qwen3 held-out paths for leakage loss. D/S: disjoint/shared answers; U/L: uniform or validation-selected large-

A

case.

\Delta

is the observed change,

P

the initial projection,

R=\Delta-P

, and

M

the mixed contribution, all in nats and rounded to six decimals, and all component audits pass.

LLM Fine-Tuning

LLM fine-tuning adapts pretrained large language models to tasks, domains, languages, and behaviors through additional training.

See all 755 papers

May 25, 2026

Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay

Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from prior tasks, which is often impractical. By contrast, language models can sample from their own training distribution, and we show that these self-generated samples serve as effective replay data, nearly eliminating forgetting. We find that forgetting nonetheless persists when the model has little remaining capacity: models pretrained close to saturation cannot absorb new information without overwriting prior knowledge. When capacity is not the limiting factor, low learning rates reduce forgetting but require substantially more training steps. Replay breaks this tradeoff, enabling fast, high-learning-rate finetuning without forgetting.

Martin Marek, Dongkyu Cho, Shikai Qiu +3

New York University

Continual Learning for LLMs LLM Fine-Tuning Replay-Based Continual Learning Catastrophic Forgetting Fine-Tuning Language Models

Ask PDF

Oct 6, 2026

When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting

Knowledge that a language model appears to forget during finetuning often remains stored and can be recovered, a phenomenon called spurious forgetting. Finetuning on new facts can even produce forgetting that undoes itself: recall of the old facts collapses, recovers as training continues on new facts alone, and only then erodes for good. We seek to understand when such forgetting is not catastrophic. A minimal associative memory reproduces these dynamics with three ingredients: keys with shared structure, concentrated new values, and normalization in the network. Finetuning moves all old representations along a common direction, hiding the old facts while preserving their relative geometry; normalization withdraws this shift once the new facts are learned, whereas fact-specific changes accumulate and cause the erosion. Moreover, subtracting the common shift eliminates the collapse in a Transformer trained on synthetic data, and removing a single direction from each weight update restores old facts in a pretrained language model. Forgetting thus combines a shared, reversible loss of access with a slow erosion of individual facts, and only the second is catastrophic. Which one dominates depends on whether the new data move old memories together or apart.

Vedant Palit, Florent Draye, Nicolas Zucchet +2

MPI for Intelligent Systems, Tübingen · Jinesis Lab, University of Toronto & Vector Institute · EuroSafeAI +3

Memorization in Language Models LLM Fine-Tuning Catastrophic Forgetting

Ask PDF

Sep 28, 2026

Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models

Memorization has been proposed as a mechanism to explain how language models fit the tail of their training distributions, but its training dynamics are not understood well. In this work, we take a fine-grained look at memorization by decomposing the loss trajectory of memorized sequences over training and model parameters. Across the Pythia family, we study memorization of duplicated training sequences (recitation) and rare ones (recollection). We find that memorization in both cases is characterized by sequence-level gradient alignment, though recitation suffers from misalignment with other training influences which causes forgetting, explaining the necessity for higher duplication of these examples. We further show that the lower model layers are the most involved in memorization and forgetting. Predicting memorization, our decomposition improves over a cross-entropy baseline, especially in larger models and early in training. Intervening on a small set of highly influential parameters we are able to ablate memorization in the final model. Together, these findings advance our understanding of how memorization develops during training and offer insights for predicting and intervening on it.

Florian Eichin, Philipp Mondorf, Andrei Mircea +3

MaiNLP, Center for Information and Language Processing, LMU Munich & Munich Center for Machine Learning (MCML), Germany · Mila – Quebec AI Institute & University of Montreal, Canada · Saarland University, Germany

Memorization in Language Models Data Repetition in Language Model Training Gradient Alignment

Ask PDF

Abstract

LLM Fine-Tuning

Explore similar work

Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay

When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting

Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models