cs.CLOct 8, 2026
SaveSelf-Preservation in Language Models
Organizations: New York University · Google Research · Cornell University
Abstract
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.
Figures & tables
Figure 1: Generated replay across supplied targets can protect held-out capabilities while retaining new-task gains. Top: prior-task accuracy changes. Bottom: incoming-task gains for Qwen3-0.6B, 1.7B, and 4B after one epoch on 1,024 MedQA examples at 1e-5 . SFT uses incoming data only. Replay uses real prior-task examples and labels. Task-Bank Self-Replay uses model-generated questions and answers. Both replay methods cover MMLU, GSM8K, HellaSwag, and BoolQ. The plot shows seven prior tasks. Stars mark held-out tasks: ARC-C, WinoGrande, PIQA, and SVAMP. Points are five-seed means. Displayed error bars are SE. The zero ring denotes pre-update accuracy, and all radial panels span to pp.
Figure 2: Nearly unchanged accuracy can conceal substantial loss of previously correct answers. Leftward bars show correct-to-incorrect transitions. Rightward bars show incorrect-to-correct transitions. Diamonds show their net effect on accuracy. The four-task mean loses 3.56% and gains 3.61% of examples, for a net pp change. Qwen3-4B learns MedQA using 256 input rows, one epoch, 1e-6 , and two seeds. Each task has 512 paired evaluation items. Bars use all items as the denominator. Retention (correct correct) conditions on initially correct items.
Figure 3: Broad replay protects held-out tasks better than single-target replay while retaining similar new-task gains. SFT uses incoming data only. Single-Task Replay rehearses one prior target. Replay uses real examples and labels from MMLU, GSM8K, HellaSwag, and BoolQ. Task-Bank Self-Replay generates replay questions and answers for that bank. Rows show incoming gain, changes on three displayed replay tasks, and changes on four held-out tasks. Qwen3-4B learns C-STANCE for one epoch at 1e-5 . Circles are five-seed means with SE relative to fixed pre-update scores. Diamonds and stars summarize the mean and minimum of the displayed task means. Zero denotes pre-update performance.
Figure 4: Combining partial banks improves mean held-out Retention, although the best bank varies by task. Bank A contains MMLU and GSM8K. Bank B contains HellaSwag and BoolQ. A+B combines all four. Replay questions and answers are model-generated. Top: Retention (correct correct) on ARC-C, WinoGrande, PIQA, and SVAMP, all held out from replay construction and target selection. The dashed outer ring is the frozen model’s 100% Retention reference, not 100% accuracy. The radial range is 60–100%. Bottom: MedQA accuracy gain. Each model uses the Figure 1 setting: 1,024 incoming examples, one epoch, and 1e-5 . Points and bars are means SE across five seeds.
Figure 5: Incoming-aware selection improves held-out Retention at 4B and approaches the full-bank reference with two targets. Random, bank-only, and incoming-aware selection each choose two entries from MMLU, GSM8K, HellaSwag, and BoolQ before replay generation. Bank-only selection receives the inventory. Incoming-aware selection also receives incoming-task information. The full bank uses all four entries. No corresponding mean advantage over random selection appears at 0.6B or 1.7B. Top: held-out Retention (correct correct), with a 60–100% radial range and a dashed 100% pre-update reference. Bottom: MedQA accuracy gain. Training follows Figure 1 : 1,024 incoming examples, one epoch, and 1e-5 . Points and bars are five-seed means SE. All four evaluation tasks are excluded from selection and replay.
Figure 6: Replay can reduce the worst held-out loss while also reducing new-task gains under a fixed example allocation. Left: incoming accuracy gain. Right: mean seed-wise worst held-out accuracy change. Higher is better. Qwen3-4B learns MedQA at 1e-6 for one epoch and 16 updates. SFT uses 256 incoming rows. Replay arms use 197 incoming and 59 replay rows. Task-Bank Self-Replay generates answers to supplied questions. Replay uses stored examples and labels. Random replay uses one bank task. The post-hoc selector uses observed SFT damage as an analysis reference. Open circles are seeds. Diamonds average two seeds. Input rows and updates are matched, but tokens and generation cost are not. Table 8 gives paired forgetting and complete scores.
Figure 7: Different incoming tasks cause different patterns of forgetting under the same training protocol. Qwen3.5-4B is updated with SFT on MedQA or C-STANCE at 1e-6 , holding the model and optimization protocol fixed. Paired markers show each capability’s score change averaged across two seeds. Connecting segments compare update conditions, not training checkpoints. The two panels group three displayed bank tasks and four held-out tasks on the same horizontal scale. These groups identify evaluation tasks. Neither supplies replay in these SFT runs. The zero line denotes the pre-update model. The respective new-task gains are for MedQA and for C-STANCE.
Figure 8: A learning rate that supports useful new-task learning can still cause uneven losses across prior capabilities. The complete main sweep contains 36 SFT runs: three model rows, two update-task columns, three learning rates, and two seeds. Each run uses 256 selected input rows for one epoch. Cells show seed-mean percentage-point score change from the pre-update model. The top row of each panel is new-task gain. Horizontal rules separate it from three displayed bank tasks and four held-out tasks. Color uses one common scale saturated at pp. Printed values preserve larger magnitudes. At 1e-5 , several conditions lose new-task accuracy as well as prior-task accuracy, indicating failed adaptation rather than a necessary retention–acquisition trade-off. Bank tasks are replay-eligible evaluation tasks. No replay is used in these SFT runs. These are endpoint comparisons across learning rates, not training trajectories. The full capability profiles appear in Appendix D .
| Learning rate | Incoming test accuracy | Held-out mean accuracy | Worst held-out accuracy | Held-out Retention (%) |
|---|---|---|---|---|
| 1e-7 | ||||
| 3e-7 | ||||
| 1e-6 | ||||
| 3e-6 | ||||
| 1e-5 |
Table 1: A lower learning rate can improve both new-task learning and retention of prior correct answers. Qwen3-1.7B learns C-STANCE from 1,024 examples for one epoch (64 updates), separate from the 256-row study in Figure 8 . Values are means SE across seeds 0 and 1 at the FP32 final checkpoint. Accuracy changes compare matched pre/post predictions and use percentage points. ARC-C, WinoGrande, PIQA, and SVAMP each have 512 evaluation items. Incoming test accuracy uses a separate 512-item panel. Worst change is the minimum across held-out tasks within each seed, then averaged. Retention (correct correct) is averaged equally across held-out tasks.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Role | Task | Evaluation format | Replay data |
| Update | MedQA | multiple choice | update data only |
| Update | C-STANCE | three-way choice | update data only |
| Replay tasks | MMLU | multiple choice | yes |
| Replay tasks | HellaSwag | multiple choice | yes |
| Replay tasks | GSM8K | numeric generation | yes |
| Replay tasks | BoolQ | binary choice | yes |
Table 2: Evaluation beyond replay targets tests the breadth of preservation. Replay data is available only for the replay tasks. The two update tasks supply training and validation data, while the held-out tasks are evaluated only after the update rule is frozen.
| Model | Revision | TP | TPU |
|---|---|---|---|
| Qwen3-0.6B | c1899de289a0 | 4 | v6e-4 |
| Qwen3-1.7B | 70d244cc86cc | 4 | v6e-4 |
| Qwen3-4B | 1cfa9a720891 | 8 | v6e-8 |
| Qwen3.5-4B | 851bf6e806ef | 8 | v6e-8 |
Table 3: Fixed model revisions help make preservation comparisons interpretable. Tensor parallelism (TP) and TPU shape describe the main sweep and additional 0.6B sweep. The Qwen3-4B protection replication used TP on v6e-4 with the same pinned weights.
| Task | Pre-update | SFT | Single-Task Replay | Replay | Task-Bank Self-Replay |
|---|---|---|---|---|---|
| New task | |||||
| C-STANCE | |||||
| Displayed replay tasks | |||||
| MMLU | |||||
| HellaSwag | |||||
| BoolQ | |||||
Table 4: Broad replay can limit prior-capability losses while maintaining new-task gains. Single-Task Replay uses one prior target. Replay and Task-Bank Self-Replay use the four-task inventory: Replay uses stored examples and labels, and Task-Bank Self-Replay uses model-generated questions and answers. Qwen3-4B, learning rate 1e-5 , one epoch, five seeds. Scores are absolute accuracy percentages. Pre-update scores come from the initial checkpoint. Other entries are endpoint means SE.
| A. Confidence-based competence estimates | |||||
|---|---|---|---|---|---|
| Task-level | Item-level | ||||
| Model | Spearman | Gap | AUROC | ECE | Brier |
| Qwen3-1.7B | 0.679 | 0.328 | 0.681 | 0.331 | 0.332 |
| Qwen3-4B | 0.607 | 0.235 | 0.756 | 0.238 | 0.239 |
| Qwen3.5-4B | 0.857 | 0.059 | 0.787 | 0.069 | 0.139 |
Table 5: Improved calibration does not guarantee improved task-level competence ranking. Panel A contrasts task-level estimates with item-level confidence metrics across seven multiple-choice capabilities. Panel B reports mean absolute error (MAE) when task accuracy is estimated from labeled behavioral probes and tested on the disjoint remainder, over 100 resamples. Larger probe budgets reduce this error for every model. Arrows indicate the preferred direction. Calibration gap, ECE, and MAE use the 0–1 accuracy scale, while Spearman, AUROC, and Brier score are dimensionless. Bold identifies the best model within each column, not statistical significance.
| Replay interface | Supplied to the model | Model contributes |
|---|---|---|
| Replay | Replay questions and labels | No generated replay answer |
| Supplied-question answers | Demonstrations and unlabeled questions | Answers to supplied questions |
| Question-and-answer generation | Supplied task entries | Replay questions and answers |
| Exploratory expanded bank | Incomplete task inventory | Additional task entries, then questions and answers |
Table 6: Replay designs differ in the preservation information they require. These are alternative interfaces. The endpoint and supplied-bank composition/selection studies generate both questions and answers. The fixed-example-allocation study instead generates answers to supplied questions. Model-proposed additions are examined in the separate exploratory bank-expansion pilot.
| Task | Base (%) | Post (%) | Change (pp) | Retention (%) |
|---|---|---|---|---|
| MedQA test | 43.95 | 39.45 | 47.11 [40.44, 53.34] | |
| ARC-C | 74.80 | 67.77 | 78.59 [74.41, 82.51] | |
| WinoGrande | 54.30 | 51.37 | 49.28 [43.17, 55.76] | |
| PIQA | 64.65 | 61.72 | 45.62 [40.79, 51.06] | |
| SVAMP | 78.32 | 19.53 | 22.69 [18.95, 26.93] |
Table 7: The expanded-bank pilot loses new-task accuracy and held-out behavior at the tested operating point. The pilot uses Qwen3-1.7B on MedQA with seed 1 and 512 matched examples per endpoint. Retention (correct correct) conditions on initially correct answers. Brackets are 95% paired-example bootstrap intervals (1,000 resamples), which describe evaluation uncertainty within this run. Accuracy changes are post minus base. Mean held-out change is pp. This single-arm result does not measure expansion’s effect relative to a fixed bank.
| Replay tasks | Held-out evaluation | |||||
| Method | New-task | Replay | Worst | MF | Worst | MF |
| gain (pp) | rows | (pp) | (%) | (pp) | (%) | |
| SFT | +1.17 | 0 | -3.04 | 3.68 | -1.56 | 5.74 |
| Random replay | +0.98 | 59 | -1.37 | 3.02 | -1.76 | 5.00 |
| Task-Bank Self-Replay | +0.49 | 59 | -0.39 | 2.03 | -0.68 | 3.52 |
| Replay | +0.98 | 59 | -1.27 | 2.26 | -0.88 | 3.82 |
Table 8: Generated replay reduces measured forgetting alongside smaller new-task gains at a fixed example allocation. Qwen3-4B is adapted to MedQA at 1e-6 for one epoch with two seeds per arm. All arms allocate 256 input rows. Replay occupies 59 rows before tokenization, replacing new-task inputs. Worst is the mean of each seed’s minimum capability change within the indicated task group. Retention (correct correct) equals on the same eligible tasks. MF is mean forgetting among initially correct examples over tasks with enough initially correct examples under the rule in Appendix A.1 . It need not cover every task included in Worst. Higher changes and lower MF are better. The post-hoc selector is analysis-only. Bold identifies the best reported retention summary, not statistical significance.
Figure 9: Positive average changes can conceal losses on individual prior tasks even when the incoming task improves. Qwen models after C-STANCE SFT at selected operating points with new-task gains of , , and , respectively. Each circle is a task’s change averaged across two seeds. Diamonds and stars show the mean and minimum across eight task means. Filled markers identify bank tasks and open markers held-out tasks. These labels identify the evaluation split, not replay in these SFT runs. Blue denotes 0.6B parameters and orange 4B. Horizontal ranges differ by model, with zero marking the pre-update score. Compare magnitudes by tick values, not visual distances. The learning rates are 3e-6 , 1e-6 , and 1e-6 . The 0.6B panel uses the separate 256-row C-STANCE calibration, not the 1,024-row follow-up. This selected comparison does not isolate the effect of model scale from model generation and learning rate. The complete grid follows.
Figure 10: An update can damage prior capabilities without improving performance on the new task. Rows identify models and columns identify update tasks. Color identifies learning rate. Rightward is greater new-task gain, and upward is a more positive prior-task accuracy change. Each circle is one capability’s change averaged across two seeds: filled circles denote bank tasks and open circles denote held-out tasks. Diamonds and stars summarize the mean and minimum of these eight capability means. Circles have small horizontal offsets for visibility. Summary markers retain the exact new-task gain coordinate. Zero denotes no change from the pre-update model. Vertical scales match within model rows but differ across rows. All main sweep learning-rate conditions are retained.
Figure 11: Learning rate changes both the severity and the distribution of forgetting, with different patterns across models and incoming tasks. The top three rows show the complete paired main sweep, with model by row and update task by column. The bottom panel separately shows the additional Qwen3-0.6B C-STANCE sweep. Circles show capability means, with filled markers for bank tasks and open markers for held-out tasks. Diamonds and stars are the mean and minimum of the eight capability means. Labels below each learning rate report new-task gain in percentage points and seed count . Vertical scales match within model rows but differ across rows. The additional Qwen3-0.6B calibration uses 256 rows and covers C-STANCE only. It is separate from the 1,024-row follow-up. Every measured condition is retained, including high-rate failures as stress tests.
Figure 12: Broad replay reduces held-out accuracy losses across the tested sizes, but an average accuracy gain can still conceal an individual task loss. Qwen3 models are adapted to 1,024 MedQA examples for one epoch at learning rate 1e-5 . Average and worst-task accuracy loss summarize ARC-C, WinoGrande, PIQA, and SVAMP. Each point summarizes five-seed task means. Negative values indicate improvement. Lines connect measured sizes.
| Initial final change (pp) | Stage 1 2 (pp) | ||||
|---|---|---|---|---|---|
| Method | MedQA | C-STANCE | Worst replay | Worst held-out | MedQA |
| Sequential SFT | +5.68 | +13.00 | -12.55 | -8.79 | +0.39 |
| Replay | +9.39 | +13.60 | -2.35 | +2.54 | +0.59 |
| Task-Bank Self-Replay | +3.91 | +17.20 | -5.49 | -12.70 | +0.20 |
| Task-Bank Self-Replay + MedQA | +3.52 | +16.60 | -9.80 | -3.71 | -0.20 |
Table 9: Adding a replay target can improve held-out accuracy while worsening accuracy on replay tasks. Adding MedQA to Task-Bank Self-Replay improves worst held-out change but worsens worst replay-task change relative to the fixed bank. Qwen3-0.6B learns MedQA then C-STANCE with one seed. The first four score columns compare the final model with the initial model. The last compares MedQA accuracy immediately before and after the C-STANCE update on the same 511 evaluation examples. All changes are in percentage points. The expanded arm remains above the initial MedQA score but slightly below its own first-stage score, illustrating why final gains do not establish retention of every intermediate gain. Stage-relative changes measure net accuracy, not preservation of individual correct answers. No uncertainty or significance is inferred from this one-seed pilot.
More in this topic
Continual Learning for LLMs
Continual learning for large language models studies how models absorb new knowledge and skills while retaining previously learned capabilities.
See all 233 papers