Adapting pretrained models to new data can cause catastrophic forgetting of previously learned behavior. When only a few past samples remain, they give continual learning methods sparse and narrow evidence about what to preserve. We show that language models can expand this evidence through self-probing, in which the frozen model generates new inputs from the retained samples and records its own predictions on them. Unlike prior work that replays such data as training examples, our method, CPLUS uses self-probe and past-sample gradients to scale down parameter updates that conflict with prior behavior. Experiments with five language models on four benchmarks show three results. First, the same probes preserve more prior behavior as gradient signals than as replay data. Second, CPLUS learns the new data while consistently reducing forgetting more than existing baselines, especially when past data are scarce, and this protection extends to benchmarks not used for training. Third, we observe that CPLUS also becomes more effective as models grow: within the Qwen3 model family, it recovers an increasing share of the forgetting caused by standard fine-tuning.
Figures & tables
Figure 1: Limited past-data access leaves preservation signals both sparse and narrow. (a) Preservation weakens when past samples are scarce. Qwen3-0.6B is adapted to new CommonsenseQA data. Replay, Collaborative Parameter Learning (CPL), and CPLUS (Ours) use A retained examples (anchors); supervised fine-tuning (SFT) uses only the new data. Retention is the percentage of initially correct examples that remain correct after adaptation (Correct → Correct). It generally decreases as A shrinks, indicating increased forgetting. CPLUS achieves higher Retention across anchor budgets. (b) Preserving the anchored target task does not necessarily preserve performance elsewhere. After the same CommonsenseQA adaptation, we evaluate five benchmarks that are not used for adaptation or as anchors. Colored points report each task’s accuracy change from the base model, and diamonds report the five-task average. Points and error bars are means and standard errors over five seeds; all methods attain 100% Acquisition (Incorrect → Correct) on CommonsenseQA. CPLUS remains closest to the base model across the profile.
Figure 2: Overview of CPLUS. 1. Self-probes. The frozen base model uses the limited anchors PA as in-context demonstrations to generate a self-probe bank SA . 2. Gradient blend. Gradients from the anchors and self-probes are blended into g . 3. Smooth Update Masking. The blended preservation gradient gates the optimizer’s proposed new-data update δt parameter -wise, and the binary mask is smoothed by an exponential moving average. Finally, the smoothed mask is applied to δt . Thus, self-probes supply preservation evidence rather than replay targets.
Figure 3: CPLUS reduces forgetting even with few past examples. Curves show CPLUS (Ours) , CPL , Replay , Self-Probe Replay , and SFT across anchor budgets. Retention is the percentage of initially correct examples that remain correct after adaptation (Correct → Correct). Error bars show five-seed standard errors. Panels share a vertical scale within each model row. Off-scale MedQA SFT values are annotated in the 4B and Gemma rows.
Figure 4: CPLUS preserves broader performance on other tasks while learning new data. Panels show changes from the base model after ARC-C, MMLU, or MedQA adaptation. Diamonds are unweighted averages over benchmarks external to the adaptation task; in the ARC-C panel, its own point is excluded from that average. Points and error bars show means and standard errors over five seeds. Each panel has its own labeled vertical range.
Figure 5
Figure 6: CPLUS in limited-anchor sequential multi-task adaptation. Training follows CSQA → ARC-C → MMLU → MedQA, carrying forward each checkpoint; MMLU thus inherits CSQA and ARC-C training. The CSQA checkpoint is shared. Curves show accuracy on all encountered tasks (mean and standard error, three seeds). Replay, CPL, and CPLUS use A=8 anchors per past task.
Figure 7: CPLUS offers a favorable compute–Retention tradeoff across the evaluated model scales. On CommonsenseQA, the axes report Retention gain over SFT and training FLOPs relative to SFT (log scale). Preservation methods use eight anchors; SFT uses none. One-time preparation is excluded. App. E.12 gives the accounting and baseline definitions. Y-axis scales differ.
Figure 8: Early forgetting can be followed by recovery as training continues. Retention over epochs (left) and the Acquisition–Retention path (right) for a 25-epoch Qwen3-0.6B/CommonsenseQA diagnostic with eight anchors. The lines show epoch-level scores without uncertainty bands. This diagnostic is independent of the five-run comparison in Figure 3 . Filled markers indicate the first epoch reaching 100% Acquisition.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Question
Location
E1
Does CPLUS reduce forgetting when anchors are limited?
Figures 3 and E.1
E2
Does preservation extend beyond the behavior represented by the anchors?
Figures 4 and E.10
E3
Are self-probes more effective as preservation gradients than as replay targets?
Figure 5
E4
Which CPLUS components and configurations matter?
Figures 5 and E.2
E5
Does CPLUS remain effective in the tested task sequence?
Figures 6 and E.9
E6
How do Acquisition, Retention, and computational cost evolve during adaptation?
Figures 7 and E.3
Appendix
Table 2: Experiment index.
Figure 9: Retention under limited anchor access across model scale. Retention denotes Correct → Correct. We compare access-matched CPL and CPLUS with four anchors on MMLU and MedQA. Labels report the absolute Retention improvement of CPLUS over CPL; points and error bars show means and standard errors. Every setting attains 100% Acquisition (Incorrect → Correct).
Figure 10: Model-scale perspective. Colored lines show individual benchmarks and the black line is their macro average. Retention denotes Correct → Correct. Left: Retention after unconstrained fine-tuning, measuring each model’s native vulnerability to forgetting. Center: CPLUS Retention averaged over A∈{4,8} . Right: the fraction of SFT forgetting recovered by CPLUS, computed as 100(RCPLUS−RSFT)/(100−RSFT) , with CPLUS Retention averaged over the same two values of A . Every CPLUS run in the fixed grid attains 100% Acquisition (Incorrect → Correct). Because each model induces its own mastered and new-data sets, the right panel measures the share of each model’s available forgetting headroom that CPLUS recovers.
Figure 11: Development-only hyperparameter searches. Retention denotes Correct → Correct. Left: Retention for two- and four-example in-context generation across sampling temperatures, evaluated with Qwen3-0.6B on CommonsenseQA at A=4 and N=512 . The selected configuration uses two examples and T=0.8 . Right: Retention across A∈{4,8,16,32} for self-probe-only preservation, 75:25 blending, and equal-weight blending under a separate development protocol. Equal-weight blending is strongest at all four budgets. All runs achieve 100% Acquisition (Incorrect → Correct).
Reduction
EMA
Retention (%)
Acquisition (%)
Mean
None
81.11±2.95
100
Mean
0.9
82.32±0.64
100
CVaR
0.9
77.40±0.28
100
Appendix
Table 3: Reference reduction and mask smoothing. Development results with A=64 , averaged over three runs.
Figure 12: Epoch-level Acquisition–Retention dynamics under limited anchor access. We evaluate SFT, Replay, CPL, and CPLUS after every epoch over 25 epochs on Qwen3-0.6B and CommonsenseQA with A=8 anchors and lr=1e-6 . Left and center show Retention and Acquisition over training; right traces the corresponding Acquisition–Retention path. Both Retention panels use the same vertical scale, labeled on the left. Per-epoch uncertainty is not shown. Filled markers identify the first epoch reaching 100% Acquisition.
Anchor composition
CommonsenseQA (%)
Base-correct
Base-wrong
Retention
Correction
Overall
8
0
83.49
13.42
40.15
6
2
81.34
21.24
44.16
4
4
80.62
23.16
45.07
2
6
80.38
20.80
43.52
0
8
80.86
23.45
45.35
Appendix
Table 4: Effect of anchor composition when adapting Qwen3-0.6B to MedQA with A=8 ground-truth-labeled CommonsenseQA anchors. All rows obtain 100% MedQA Acquisition and 100% accuracy on the eight supplied anchors. Results use one seed.
Method
Acquisition (%)
Retention (%)
Overall accuracy (%)
SFT
98.97
21.77
69.53
Replay
100.00
12.68
66.70
CPL
42.63
9.57
30.02
CPLUS
98.97
39.00
76.09
Appendix
Table 5: Fixed aggressive-update stress test on Qwen3-0.6B and CommonsenseQA with A=8 . Every method uses AdamW at lr=1e-5 for exactly 10 epochs, with no early stopping, epoch selection, or Acquisition filtering. Results are from seed 0.
Method
Matched property
Retention (%)
Acquisition (%)
Norm-matched SFT
Applied update norm
45.45
100
Randomized CPLUS mask
Per-tensor mask coefficients
47.13
100
CPLUS
Probe-informed coordinate placement
71.05
100
Appendix
Table 6: Matched update controls. Every row achieves 100% Acquisition. The randomized mask preserves CPLUS’s exact within-tensor coefficient multiset, while norm-matched SFT matches the norm of the update applied by CPLUS.
Figure 13: Instantaneous mask geometry under limited anchor access. CPL and CPLUS are audited at seven pre-update states in separate, matched Qwen3-0.6B CommonsenseQA runs with A=8 . The horizontal axis counts training epochs already completed. Left: fraction of coordinates admitted by the diagnostic gradient-sign mask. Right: fraction of each method’s candidate AdamW update squared norm admitted by that mask. The CPLUS diagnostic uses its blended probe–anchor gradient before EMA smoothing. The audit is read-only; results use seed 0.
Figure 14: CPLUS need not recover the full-data mask. Each method is compared with its own full-data reference. CPL is closer at every budget despite lower Retention (Correct → Correct), so mask similarity alone cannot explain CPLUS’s advantage. Coordinate-level masks appear in Figure 15 .
Figure 15: Coordinate-level view of the mask audit. We replay the same 678 proposed optimizer updates under controlled probe-side inputs. Left: each row shows the update-weighted mean CPLUS mask at one anchor budget; “Full” uses all 418 past examples. Columns are 512 pre-registered coordinates from a middle-layer MLP down-projection, ordered identically in every row. Purple indicates a suppressed update coefficient (0) and yellow a retained coefficient (1). Right: each method is compared with its own full-data reference mask—CPL with full-data CPL and CPLUS with full-data CPLUS. Agreement is update-energy weighted, and 1 denotes identical masks.
Method
After ARC-C
After MMLU
After MedQA
SFT
66.81
72.20
61.21
Replay
72.04
77.29
69.73
CPL
75.18
70.77
69.65
CPLUS
83.94
80.86
78.87
Appendix
Table 7: Average Task Accuracy during sequential multi-task learning. Each entry is the unweighted mean of absolute accuracy over all tasks encountered through the indicated stage, averaged across three seeds. Task-level standard errors are reported in Table 8 for the final stage and visualized throughout Figure 6 .
Method
CSQA
ARC-C
MMLU
MedQA
Average
SFT
36.04±0.51
53.78±0.49
74.82±0.44
80.19±0.43
61.21
Replay
55.29±0.49
63.86±0.45
79.59±0.41
80.19±0.42
69.73
CPL
59.85±0.47
62.60±0.45
74.38±0.42
81.77±0.40
69.65
CPLUS
69.34±0.46
78.27±0.44
83.43±0.44
84.45±0.42
78.87
Appendix
Table 8: Accuracy after the final sequential-learning stage. Following adaptation to MedQA, CPLUS achieves the highest accuracy on every task encountered in the sequence. The final Average Task Accuracy is the unweighted mean of the four absolute task accuracies. Values are means ± standard errors across three seeds; the Average column is computed from the task means.
Figure 16: Base-normalized capability scores after adaptation. Each axis reports the post-adaptation score as a percentage of the corresponding base-model score. The dashed polygon marks the base-model level (100%); values above 100 indicate improvement. Raw score changes are shown in Figure 1 .
Figure 17: Method-matched mask agreement across transformer depth. Each cell reports update-weighted agreement for one layer and anchor budget. (a) CPL with A anchors compared with CPL using all 418 past examples. (b) CPLUS with A anchors compared with CPLUS using all 418 past examples under the same controlled probe inputs. The shared color scale ranges from zero to one.
Figure 18: Cross-reference comparison with full-data CPL. CPLUS masks are compared directly with the mask constructed by CPL from all 418 past examples. (a) Agreement for the pre-registered Layer-14 MLP down-projection. (b) Agreement across all transformer layers. This deliberately mismatched reference measures how differently CPLUS suppresses updates; it is not an oracle target for CPLUS.
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.
Dongkyu Cho, Rumi Chunara, Ali Behrouz
New York University · Google Research · Cornell University
Models trained on a new task typically degrade on prior tasks, a phenomenon known as forgetting. Traditionally, mitigating forgetting has required replaying stored exemplars from prior tasks, which is often impractical. By contrast, language models can sample from their own training distribution, and we show that these self-generated samples serve as effective replay data, nearly eliminating forgetting. We find that forgetting nonetheless persists when the model has little remaining capacity: models pretrained close to saturation cannot absorb new information without overwriting prior knowledge. When capacity is not the limiting factor, low learning rates reduce forgetting but require substantially more training steps. Replay breaks this tradeoff, enabling fast, high-learning-rate finetuning without forgetting.
Large language models (LLMs) often suffer from catastrophic forgetting in continual learning: after learning new tasks sequentially, they perform worse on earlier tasks. Existing methods mitigate catastrophic forgetting by data replay, parameter freezing, or regularization. However, these methods lack understanding of LLM mechanisms and cannot distinguish which parameters store important knowledge from previous tasks and which parameters can be updated for new tasks. To address this, we propose the attribution-guided continual fine-tuning framework that leverages Layer-wise Relevance Propagation (LRP) to estimate parameter importance based on the internal computational process of LLMs. During continual learning, parameters critical to previous tasks are constrained to receive smaller updates, while less relevant parameters remain available for learning new tasks. Extensive experiments show that, compared with baseline methods, our approach reduces catastrophic forgetting while preserving adaptability to new tasks, highlighting the value of mechanistic attribution for continual fine-tuning of LLMs.
Yazheng Liu, Yuxuan Wan, Rui Xu +3
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Beijing University of Posts and Telecommunications, Beijing, China