Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss

Yi Ren; Wenlong Deng; Guanzhe Hong; Clare Lyle; Yarin Gal

cs.LGSep 27, 2026

Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss

Authors: Yi Ren, Wenlong Deng, Guanzhe Hong, Clare Lyle, Yarin Gal

Organizations: OATML · University of Oxford · UBC · Google DeepMind

Abstract

Modern language models are likely to be updated throughout their lifetime rather than trained once and frozen. Each update therefore participates in a recurring cycle: decide which experience to learn from, understand what that update changes, and remain capable of learning from what comes next. We show that these challenges are governed by the same evolving update--behavior interaction. We derive a token- and layer-wise decomposition of how learning from one token changes another prediction. By separating the softmax force, shared readout geometry, and residual connections, it exposes two interaction channels and yields a forward-computable approximation. Following this interaction through time reveals a unified picture of continual adaptation. Positive interaction identifies useful experience; negative interaction produces either concentrated collision or accumulated erosion; over longer horizons, updates reshape the shared geometry mediating future learning signals, reducing their transmission. These predictions lead to effective data selection, mechanism-specific controls for interference, and a readout-based diagnostic of future learnability whose degradation predicts the benefit of restoring the readout. Across models and training regimes, the same local interaction thus explains both what an update changes now and how learning today changes what can be learned tomorrow. This view connects data attribution, forgetting, and plasticity loss as distinct regimes of the same evolving learning dynamics.

Figures & tables

Figure 1: A unified view of continual adaptation through the evolving update–behavior interaction underlying selection, interference, and future learnability.

Figure 2: Geometry and fidelity of the two-channel approximation. (a) Direct token-force geometry in CH1 versus readout-mediated geometry in CH2 . (b) Empirical readout geometry $\bm{\mathsf{w}}\bm{\mathsf{w}}^{\top}$ across task-related tokens, showing substantial off-diagonal coupling, measured on Qwen2.5-1.5B. (c) The forward-computable approximation tracks the measured one-step change $\Delta_{t}(o,u)$ .

Figure 3: Residual architecture and identity-path approximation. All layers share the output-side signal

\bm{\mathsf{w}}^{\top}g

; we retain the direct residual-stream path

J_{\ell}^{(1)}

and omit higher-order paths.

Method	Sentence Transform.		Controlled Math		Cross-lingual Retrieval		Efficiency
(Qwen2.5-1.5B-Inst.)	AUC $\uparrow$	Recall $\uparrow$	AUC $\uparrow$	Recall $\uparrow$	MMLU $\uparrow$	GSM8K $\uparrow$	Bwd.	Complexity
Random	0.500	0.100	0.500	0.100	0.025	0.020	$\times$	$\mathcal{O}(n)$
Embd	0.546	0.148	0.555	0.146	0.125	0.085	$\times$	$\mathcal{O}(nd)$
DataInf	0.981	0.826	0.985	0.878	0.419	0.070	✓	$\mathcal{O}(nd_{\mathrm{in}}L)$
HyperINF	0.993	0.934	0.986	0.942	0.694	0.370	✓	$\mathcal{O}(nd^{3}L)$
LESS	0.785	0.370	0.835	0.592	0.469	0.140	✓	$\mathcal{O}(np_{\mathrm{proj}})$

Table 1: Forward-computable interactions identify useful experience. On controlled tasks with strong direct alignment, CH1 nearly saturates attribution. When alignment is weakened cross-lingually, the readout-mediated CH2 provides substantial complementary signal, with CH1+2 markedly outperforming gradient-based baselines. Both scores use only forward-pass quantities at a fixed checkpoint.

Figure 4: Collision and erosion as two mechanisms of forgetting. (a) Collision arises from a few strong negative force interactions, often with high-energy updates. (b) Erosion arises from many weak conflicts that accumulate and push behavior toward alternative continuations.

Figure 5: Collision and erosion respond to different controls. (a) Extreme-energy updates mark confident conflicts. (b–c) Energy masking mitigates collision but has little effect on erosion. (d) For erosion, energy control improves retention mainly by sacrificing downstream learning; format separation shifts this trade-off. (e) Erosion follows the fine-tuning format, so separating fine-tuning from deployment redirects interference away from the deployed behavior. More in Appendix E .

Figure 6: Readout transmission governs future learnability. (a–b) The shared readout gates task forces into the hidden space and can form a self-reinforcing bottleneck. (c) The corresponding score $R_{\mathcal{D}}$ declines during long-horizon training for OOD tasks.

Figure 7: Readout degeneration predicts future learnability. (a) Later checkpoints adapt more slowly on OOD tasks but not ID PubMed. (b) Readout restoration partially recovers downstream learning. (c) More degeneration predicts larger reset gains across models, tasks, and checkpoints.

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Empirical validation and geometry of the two-channel approximation. (a–c) Token-level validation over sampled GSM8K updating and MMLU observing token pairs, comparing the measured one-step log-probability change, the exact first-order gradient interaction, and our forward-computable approximation at an early in-distribution checkpoint obtained after a brief adaptation on the updating distribution. Correlations are reported in each panel.

Panel of Figure 9 — Figure 9: Cold start: magnitude is preserved more reliably. (a) The approximation tracks the absolute gradient interaction well in both value and rank. (b) In the signed setting, the characteristic X-shaped pattern reveals frequent sign flips despite preserved magnitude structure. The same behavior is observed across multiple models and dataset pairs; one representative setting is shown here. ( GSM8K as $u$ -side and MMLU as $o$ -side).

Figure 10: Signed approximation quality recovers within the first 1% of updates. Auxiliary SFT experiment on GSM8K for 2 epochs; insets show scatter plots of the exact first-order interaction against Equation 5 at the marked checkpoints. At initialization the scatter also exhibits the characteristic X-shape, indicating frequent sign flips despite preserved magnitude structure. The signed geometry aligns rapidly and is largely recovered before the training loss shows systematic improvement. ( GSM8K as $u$ and Dolly as $o$ ). Similar trends across different models and update–observation pairs.

Figure 11: Signed metrics improve early and remain stable thereafter. Pearson correlation, signed Spearman correlation, and sign agreement between Equation 5 and both the exact first-order interaction (“Exact”) and the measured log-probability change (“Actual”), evaluated at checkpoints along the auxiliary SFT run. Top: log-scaled x-axis; bottom: linear x-axis. Most of the improvement occurs within the first 100 updates, below 1% of the run (dashed line), after which all three metrics stay flat through 14k updates. The two curves nearly coincide, indicating that the residual error comes from the structural approximations rather than the first-order expansion.

Figure 12: A brief warm start restores signed fidelity across depth. Approximation quality for CH2 as lower layers are progressively included, starting from the output. (a) At initialization, signed metrics degrade sharply once middle layers enter the sum, reaching chance-level sign agreement and near-zero signed correlation when all 24 layers are aggregated, while absolute Spearman correlation remains around 0.7. (b) After 100 auxiliary SFT updates, the same aggregation retains signed Spearman above 0.6 and sign agreement near 0.7 at full depth. Dashed line marks chance-level sign agreement (0.5). Magnitude structure is thus preserved regardless of depth, whereas signed fidelity at depth requires a warm start.

Figure 13: Sign agreement as a function of interaction magnitude before and after a brief warm start.

Panel of Figure 14 — Figure 14: Narrow-cone geometry of hidden representations across models. We compute token-level pairwise hidden-state alignment between 10 randomly sampled GSM8K examples and 10 MMLU examples, leading to roughly 5k $u$ - $o$ pairs. For this analysis, we randomly sample approximately 5k token pairs from the full set of token pairs induced by the selected example pairs. We evaluate Qwen models across multiple scales, together with representative dense Llama , Mistral and MoE OLMoE architectures. (a) Distributions of pairwise cosine similarity $\langle\bm{\mathsf{h}}_{i},\bm{\mathsf{h}}_{j}\rangle/\|\bm{\mathsf{h}}_{i}\|_{2}\|\bm{\mathsf{h}}_{j}\|$ at representative layers, with the vertical line marking zero. (b) Fraction of pairwise cosine similarities that are non-negative at each layer; blank cells denote unavailable layers. Positive alignment dominates across model families, scales, and architectures, supporting a pervasive narrow-cone structure in the residual representations.

Method	Sentence transformations		Math problems (w/o reasoning)		Math problems (w/ reasoning)		Efficiency
Method	AUC $\uparrow$	Recall $\uparrow$	AUC $\uparrow$	Recall $\uparrow$	AUC $\uparrow$	Recall $\uparrow$	Backward	Complexity
Random	$0.50$	$0.10$	$0.50$	$0.10$	$0.50$	$0.10$	$\times$	$\mathcal{O}(n)$
Embd	$0.546\pm 0.306$	$0.148\pm 0.205$	$0.555\pm 0.298$	$0.146\pm 0.295$	$0.560\pm 0.310$	$0.198\pm 0.311$	$\times$	$\mathcal{O}(nd)$
DataInf	$0.981\pm 0.019$	$0.826\pm 0.121$	$0.985\pm 0.032$	$0.878\pm 0.154$	$0.987\pm 0.030$	$0.892\pm 0.155$	$\checkmark$	$\mathcal{O}(nd_{in}dL)$
HyperINF	${0.993\pm 0.013}$	${0.934\pm 0.063}$	${0.986\pm 0.024}$	${0.942\pm 0.080}$	${0.988\pm 0.023}$	${0.950\pm 0.060}$	$\checkmark$	$\mathcal{O}(nd^{3}L)$
LESS	$0.785\pm 0.096$	$0.370\pm 0.139$	$0.835\pm 0.235$	$0.592\pm 0.291$	$0.829\pm 0.172$	$0.524\pm 0.350$	$\checkmark$	$\mathcal{O}(nd_{\text{proj}})$

Appendix

Table 2: Influential-data identification and efficiency on Qwen2.5-1.5B across three controlled benchmarks (5 seeds). Backward indicates whether per-example backpropagation is required, and Complexity reports the asymptotic attribution cost. Here,

n

is the number of candidate-target pairs,

d

the hidden dimension,

d_{\mathrm{in}}

the layer-input dimension,

L

the number of layers, and

d_{\mathrm{proj}}

the projected-gradient dimension used by LESS. Complexity denotes the pairwise scoring cost after caching per-example forward quantities; one-time feature construction costs are excluded.

Method	Qwen2.5 1.5B-Instruction		Qwen3-4B-Instruction		Llama3.2-3B-Instruction
Method	MMLU Acc.	GSM8K Acc.	MMLU Acc.	GSM8K Acc.	MMLU Acc.	GSM8K Acc.
Random	0.025	0.020	0.025	0.020	0.025	0.020
Embd	0.125	0.085	0.131	0.150	0.475	0.370
HyperINF	0.694	0.370	0.356	0.450	0.606	0.645
DataInf	0.419	0.070	0.138	0.115	0.313	0.175
LESS	0.469	0.140	0.188	0.140	0.319	0.355

Appendix

Table 3: Cross-lingual data attribution accuracy on MMLU and GSM8K across different models.

Figure 15: Top-K prediction token overlap between English and others.

Method	Qwen2.5-1.5B-Instruction		Qwen3-1.7B		Llama3.2-1B-Instruction		Llama3.2-3B-Instruction
Method	Top-1 Retrieval Acc.	Answer Acc.	Top-1 Retrieval Acc.	Answer Acc.	Top-1 Retrieval Acc.	Answer Acc.	Top-1 Retrieval Acc.	Answer Acc.
Base	–	0.000	–	0.000	–	0.000	–	0.000
Random	0.016	0.389	0.000	0.373	0.010	0.137	0.028	0.500
Embd	0.339	0.532	0.157	0.353	0.248	0.206	0.861	0.889
CH1	0.484	0.597	0.431	0.569	0.765	0.363	0.861	0.889
CH1+2	0.468	0.597	0.392	0.549	0.902	0.382	1.000	0.944

Appendix

Table 4: Multilingual agent-memory retrieval results on GSM8K .

	Qwen2.5-1.5B				Llama-3.2-3B
Method	1% (69)	5% (348)	10% (697)	GPU h	1% (69)	5% (348)	10% (697)	GPU h
Random	0.646 $\pm$ 0.004	0.610 $\pm$ 0.007	0.608 $\pm$ 0.005	–	0.2290 $\pm$ 0.010	0.295 $\pm$ 0.012	0.326 $\pm$ 0.016	–
LESS	0.639 $\pm$ 0.007	0.587 $\pm$ 0.004	0.597 $\pm$ 0.007	2.65	0.248 $\pm$ 0.005	0.301 $\pm$ 0.005	0.330 $\pm$ 0.007	4.16
CH1	0.652 $\pm$ 0.007	0.628 $\pm$ 0.002	0.631 $\pm$ 0.004	0.09	0.202 $\pm$ 0.011	0.307 $\pm$ 0.007	0.348 $\pm$ 0.007	0.12
CH1+2	0.654 $\pm$ 0.013	0.636 $\pm$ 0.007	0.623 $\pm$ 0.007	0.87	0.202 $\pm$ 0.002	0.313 $\pm$ 0.006	0.341 $\pm$ 0.002	3.12

Appendix

Table 5: Closed-loop results. Test accuracy after full-parameter fine-tuning on the top 1%, 5%, or 10% selected examples. We reserve 500 examples from the GSM8K training split as a fixed probing set for score computation, leaving 6,973 examples as the candidate pool for selection and downstream fine-tuning. Accuracy is reported on the

[0,1]

scale as mean

\pm

standard deviation over three training seeds. The corresponding base-model accuracies are 0.6194 for Qwen2.5-1.5B and 0.1069 for Llama-3.2-3B . GPU h denotes aggregate A100 GPU-hours for score computation, including all method-specific setup such as warm-up and gradient extraction; downstream fine-tuning is excluded.

Objective	Update force	Supervision / sampling	Risk of extreme energy
SFT	$\bm{\mathsf{e}}_{y_{u}}-\pi_{\theta}$	Off-policy, one-hot	Possible
RL	$\bm{\mathsf{e}}_{y_{u}}-\pi_{\theta}$	On-policy	Uncommon
GKD	$\pi_{\mathrm{teacher}}-\pi_{\theta}$	Soft teacher target	Suppressed
OPD	$A_{u}(\bm{\mathsf{e}}_{y_{u}}-\pi_{\theta})$	On-policy	Suppressed by sampling; scaled by $A_{u}$
LS	$(1-\epsilon)\bm{\mathsf{e}}_{y_{u}}+\epsilon\bm{\mathsf{q}}_{\mathrm{unif}}-\pi_{\theta}$	Smoothed target	Bounded by $\epsilon$

Appendix

Table 6: Comparison of update-force characteristics under different finetuning objectives.

Figure 16: Cross-model validation of the entropy–energy geometry in Figure 5 (a). Token entropy as a function of the target-token probability $\pi(y\mid s)$ (top) and gradient energy $\|g\|_{2}^{2}$ (bottom) across six models and four supervision settings. The characteristic geometry observed in the main text persists across model families and scales, further supporting gradient energy as a more intrinsic characterization of token-level uncertainty than probability alone.

Model	Method	GSM8K	MMLU	IFEval	Dolly-CLS	Dolly-QA
Qwen2.5-1.5B-Instruct	base	68.46	60.12	42.14	38.77	40.68
	SFT	70.20	58.88	26.99	28.78	34.12
	EAFT	72.18	58.86	27.36	27.46	33.68
	Mask $E_{u}{>}1.8$	71.42	58.89	26.80	27.71	33.92
	Mask $E_{u}{>}1.5$	68.31	59.03	28.65	29.68	36.98
	Mask $E_{u}{>}1.0$	63.46	58.98	26.25	29.17	34.33

Appendix

Table 7: Downstream learning and general-capability retention under different OpenMathInstruct-2 finetuning methods. We compare standard SFT, EAFT, and energy-controlled variants. GSM8K reports generative accuracy, MMLU likelihood-based multiple-choice accuracy, IFEval prompt-level strict accuracy, and Dolly classification/closed-QA Rouge-L F1. Energy-controlled rows differ only in the masking threshold

E_{u}

metric / learning rate	$10^{-5}$	$2{\times}10^{-5}$	$5{\times}10^{-5}$
MMLU	$-0.02$	$+0.53$	$+1.10$
IFEval	$+0.37$	$+0.74$	$+0.93$
Dolly-CLS	$-1.32$	$-0.74$	$+0.38$
Dolly-QA	$-0.44$	$+0.11$	$+1.75$

Appendix

Table 8: EAFT

-

SFT difference (percentage points) on Qwen2.5-1.5B-Instruct trained on OpenMathInstruct-2 , across three learning rates. Positive values favour EAFT . The gap increases monotonically with learning rate on all four retention metrics, and EAFT is ahead on every metric only at the largest learning rate: at

10^{-5}

it is behind on MMLU , Dolly-CLS , and Dolly-QA . The benefit of suppressing high-energy updates therefore grows with update magnitude rather than being uniform, consistent with the collision mechanism of Section 4 . Since changing the learning rate scales all update components rather than energy alone, we treat this as supporting rather than isolating evidence.

Model	Method	Non-IF	Boxed	P(A…D)	Acc.	Worked
Qwen2.5-1.5B	base	20.5%	1.8%	0.954	54.5%	4.0%
	SFT	99.9%	67.6%	0.201	47.9%	75.6%
	EAFT	100.0%	58.1%	0.202	48.3%	78.1%
	Mask $E_{u}{>}1.8$	99.9%	73.7%	0.200	47.0%	79.1%
	Mask $E_{u}{>}1.5$	99.7%	60.5%	0.240	47.6%	69.2%
	Mask $E_{u}{>}1.0$	99.0%	42.3%	0.247	46.7%	76.6%

Appendix

Table 9: Generative MMLU evaluation after OpenMathInstruct-2 fine-tuning. We compare base, SFT, and EAFT across three models, together with a learning-rate sweep on Qwen2.5-1.5B . LR ×2 and LR ×5 denote two and five times the default learning rate, respectively.

Model	SFT form	Non-IF Q/A $\downarrow$	Non-IF P/R	Boxed Q/A $\downarrow$	Boxed P/R	$P(A\ldots D)$ Q/A $\uparrow$	$P(A\ldots D)$ P/R	GSM8K $\uparrow$
Qwen3-4B	base	11.4%	7.7%	0.0%	0.0%	0.980	0.984	85.21
	SFT (standard)	63.7%	64.0%	53.3%	60.3%	0.399	0.388	89.23
	SFT (P/R)	31.5%	94.9%	26.2%	92.7%	0.588	0.201	89.84
Llama3.2-3B	base	59.9%	55.4%	0.0%	0.0%	0.962	0.954	62.55
	SFT (standard)	88.1%	86.2%	21.3%	26.2%	0.371	0.351	76.80
	SFT (P/R)	42.5%	99.9%	7.6%	34.4%	0.743	0.011	76.88

Appendix

Table 10: Response-format separation redirects erosion toward the fine-tuning format. We evaluate all 14,042 MMLU questions using both the Question:/Answer: (Q/A) and Problem:/Result: (P/R) prompts. Compared with standard SFT, using an alternative P/R response format during OpenMathInstruct-2 fine-tuning produces strongly format-dependent erosion: behavior in the deployed Q/A format is better preserved, while interference is redirected toward P/R. Downstream GSM8K performance remains similar, indicating that format separation redirects interference rather than simply suppressing learning.

Model	Instruction Following Acc.				MMLU Acc.				GSM8K Acc.
Model	Before FT		After FT		Before FT		After FT		Before FT		After FT
Eval. format	Q/A	P/R	Q/A	P/R	Q/A	P/R	Q/A	P/R	Q/A	P/R	Q/A	P/R
Qwen2.5-1.5B-QA	100.00	99.85	100.00	93.13	59.31	58.41	58.89	55.73	8.42	9.25	39.27	41.62
Llama-3.2-3B-QA	99.98	99.94	99.53	46.51	55.28	53.95	53.81	42.44	5.69	5.53	33.43	35.78
Qwen3-4B-QA	100.00	99.97	99.99	13.30	71.04	70.74	70.65	23.81	22.37	22.29	67.63	67.10
Qwen2.5-1.5B-PR	100.00	99.99	90.14	99.01	59.76	59.76	56.97	59.04	0.00	12.66	56.71	58.45

Appendix

Table 11: Controlled test of context-dependent erosion under matched and mismatched response formats. Starting from base models, we first establish the MMLU behavior using either the Question:/Answer: (Q/A) or Problem:/Result: (P/R) format, and then fine-tune on GSM8K using the alternative format. We report instruction-following accuracy on MMLU together with MMLU and GSM8K task accuracy before and after the second-stage fine-tuning.

Established	Model	After est.	Default	Protected	$\Delta$
			(FT $=$ deploy)	(FT $\neq$ deploy)
Q/A	Qwen2.5-1.5B	100.00	97.35	100.00	$+2.65$
	Llama-3.2-3B	99.98	44.77	99.53	$+54.76$
	Qwen3-4B	100.00	68.64	99.99	$+31.35$
P/R	Qwen2.5-1.5B	99.99	85.98	99.01	$+13.03$
	Llama-3.2-3B	99.99	82.35	99.34	$+16.99$

Appendix

Table 12: Format separation removes context-dependent erosion. Each model first establishes MMLU behavior in one response format, then is fine-tuned on GSM8K in either the same format ( default ) or the alternative one ( protected ). We report instruction-following accuracy on MMLU in the established format, i.e. the format the model is deployed in.

Model	Base		GSM8K Q/A FT		GSM8K P/R FT
Model	Q/A	P/R	Q/A	P/R	Q/A	P/R
Qwen2.5-1.5B-Ins	75.27%	96.14%	73.20%	72.52%	52.63%	31.06%
Llama-3.2-3B-Ins	92.34%	97.02%	47.59%	44.85%	44.10%	39.87%
Qwen3-4B-Ins	85.89%	76.93%	0.00%	0.05%	32.12%	0.00%

Appendix

Table 13: Instruction Following Accuracy comparison on MMLU under different prompt formats and GSM8K fine-tuning settings. Q/A denotes the MMLU Question/Answer: format, while P/R denotes the MMLU Problem/Result: format.

Panel of Figure 17 — Figure 17: Effects of template separation on CH1 and CH2 . The Q/A-to-P/R shift reduces CH1 mainly through weaker prediction-error alignment and reduces CH2 mainly through lower layer-wise hidden-state similarity.

Figure 18: Sequential SFT empirically exhibits this predicted block-wise reshaping: the geometry associated with the current task is strengthened while that of the other tasks deteriorates. The model is trained sequentially on GSM8K (epochs 0-20), MBPP (20-40), and Dolly-QA (40-60).

Table 14: Additional experimental details for Section 5 .

Panel of Figure 19 — Figure 19: Relative evolution of task-specific readout geometry during sequential SFT across models. Each metric is reported relative to its value before training. The model is trained on GSM8K , MBPP , and Dolly-QA during epochs 0-20, 20-40, and 40-60, respectively. Despite model-specific differences, the overall evolution is consistent with task-dependent reshaping of the readout geometry.

Figure 20: Training dynamics under sequential SFT, same settings as Figure 6 -(c). A Qwen2.5-0.5B-Instruct model is trained on GSM8K , MBPP , and Dolly-QA for epochs 0-20, 20-40, and 40-60, respectively. While plasticity loss is weak and difficult to distinguish from the training-loss curves alone, the average token entropy shows a pronounced deviation from training the latter tasks directly from the base model. This suggests that distribution-level learning dynamics can change substantially before the loss-level effect becomes clearly visible.

Figure 21: Readout-transmission dynamics are largely insensitive to weight decay. We track

R_{\mathcal{D}}

during 100M-token long-horizon training on PubMed2025 with weight decay

0

0.1

, using GSM8K , MBPP , Dolly-QA , and held-out PubMed as probes. The two trajectories are nearly identical across all probing tasks, suggesting that the observed degeneration cannot be explained simply by uniform contraction induced by weight decay.

Figure 22: Evolution of

R_{\mathcal{D}}

during long-horizon training for different probing tasks. The

R_{\mathcal{D}}

generally decreases for the OOD tasks, while the in-distribution PubMed task does not show the same systematic decay. This suggests that long training progressively reshapes the shared readout geometry toward the current task distribution at the expense of gradient transmission for other tasks.

Panel of Figure 23 — Figure 23: The training and validation loss during SFT on different models.

Figure 24: Readout degeneration across models, learning rates, and a broader training mixture. We track

R_{\mathcal{D}}

during 300M-token long-horizon training on a hybrid corpus consisting of 40% general web text, 20% scientific/PubMed text, 15% mathematics, 15% code, and 10% QA/instruction-like data, across four model families and multiple learning rates. The long-horizon training data are trained in causal-LM format, whereas the probing datasets use downstream task formats, such as mathematical QA, code generation, and instruction-style responses; thus, semantic domain overlap does not imply exact overlap in the token-level learning signals being probed. Task-conditioned transmission generally decreases during training, but the magnitude and timescale of the effect depend strongly on the model and optimization regime, with larger learning rates typically producing faster and stronger changes. These results show that readout degeneration persists under a broader and more heterogeneous training distribution, although its strength is highly setting-dependent.

Panel of Figure 25 — Figure 25: Plasticity retention under different reset strategies (reset-last-4 means reset last four transformer blocks together with the readout), viewed under three complementary aggregations. Retained plasticity is measured by downstream loss AUC relative to the corresponding base model (integrated over downstream optimizer steps), such that 1 denotes the pre-training plasticity level; lower values indicate larger plasticity loss. Bars extend downward from this base reference. We compare standard long-training, resetting the readout layer, and resetting the last four layers, with error bars denoting one standard deviation over the aggregated dimension. Top and bottom rows report training and probe-set retention, respectively. (a) Results aggregated over models, shown separately for GSM8K , MBPP , and Dolly-QA across different checkpoints. (b) Results aggregated over downstream tasks, shown separately for each model across different checkpoints. (c) Results aggregated over checkpoints, shown for each model-task pair. Resetting the readout generally recovers part of the lost plasticity, while resetting additional upper layers can provide further recovery; the magnitude of this effect is model- and task-dependent.

Explore similar work

Jul 8, 2026cs.LG

When Does Continual Learning Require Learning

As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn? Today, the field largely frames this as a problem of context management and mitigating forgetting. We argue this framing is incomplete: continual learning is fundamentally about increasing model competence as the world changes. We disentangle this change along two axes -- space, where the model encounters new domains, and time, where the underlying data drifts under a fixed task. This framing lets us study continual learning under realistic conditions: new domains arrive over time, facts drift past their training cutoff, and agentic interactions accumulate state across episodes. To evaluate methods under this setting, we recast widely used LLM benchmarks as sequential problems and introduce a single mechanism-agnostic protocol that compares prompt-based methods (GEPA, ACE), supervised learning (SFT, SDFT), reinforcement learning (GRPO, SDPO), and context compression (Cartridges, In-place TTT). Prompt-based methods fit each new stage quickly but degrade on future tasks. Distillation-based methods accumulate knowledge stably but struggle to update outdated facts. Context compression improves efficiency without substantially improving the ability to learn new tasks. Online reinforcement learning adapts most effectively to knowledge updates but remains sensitive to noisy reward signals. Overall, our results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors, determining when adaptation must be learned inside model weights and when it can be achieved through external scaffolding. We hope that understanding where each method succeeds and fails will guide the design of stronger continual learning systems.

Anne Harrington, Nayan Saxena, Michael Murphy +7

1UC Berkeley · 2Independent · 3Capital Fund Management

Continual Learning In-Context Learning Multiple Prompt Learning Frameworks Competence

Ask PDF

Aug 19, 2026cs.LG

Harness Continual Learning: Continual Adaptation Beyond Model Parameters

Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on textual reasoning, multimodal perception, and open-world interaction demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and show that the stability--plasticity trade-off can be explicitly adjusted.

Borui Kang, Jinrui Gu, Junhan Lv +3

State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia

Replay-Based Continual Learning Continual Learning Catastrophic Forgetting Plasticity Open-World

Ask PDF

May 14, 2026cs.LG

MoRe: Modular Representations for Principled Continual Representation Learning on Sequential Data

Continual learning requires models to adapt to new data while preserving previously acquired knowledge. At its core, this challenge can be viewed as principled one-step adaptation: incorporating new information with minimal interference to existing representations. Most existing approaches address this challenge by modifying model parameters or architectures in a supervised, task-specific manner. However, the underlying issue is representational: tasks require distinct yet structured representations that can be selectively updated without disrupting representations, while structure should reflect intrinsic organization in the data rather than task boundaries. In sequential data, time-delayed dependencies provide a natural signal for uncovering this organization, revealing how fundamental representations give rise to more specific ones. Inspired by the modular organization of the human brain, we propose MoRe, a framework that identifies modularity in the representation itself rather than allocating it at the architectural level. MoRe decomposes knowledge into a hierarchy of fundamental and specific modules with identifiability guarantees, enabling principled module reuse, alignment, and expansion during adaptation while preserving old modules by construction. Experiments on synthetic benchmarks and real-world LLM activations demonstrate interpretable hierarchical structure, improved plasticity-stability trade-offs, suggesting MoRe as a principled foundation for continual adaptation

Jiaqi Sun, Boyang Sun, Rasmy M. H. +2

1Carnegie Mellon University · 2Mohamed bin Zayed University of Artificial Intelligence

Replay-Based Continual Learning Continual Learning Parameter-Efficient Adaptation Representation Learning Plasticity Modular Framework+2

Ask PDF