cs.CLMar 13, 2026
SaveBeyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages
Organizations: Zhejiang Lab, Hangzhou 311100, China
Abstract
Continual learning (CL) has emerged as a pivotal paradigm to enable large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting, a critical limitation of the static pre-training paradigm inherent to modern LLMs. This paper presents a comprehensive overview of CL methodologies tailored for LLMs, structured around three core training stages: continual pre-training, continual fine-tuning, and continual alignment. Beyond the canonical taxonomy of rehearsal-, regularization-, and architecture-based methods, we further subdivide each category by its distinct forgetting mitigation mechanisms and conduct a rigorous comparative analysis of the adaptability and critical improvements of traditional CL methods for LLMs. In doing so, we explicitly highlight core distinctions between LLM CL and traditional machine learning, particularly with respect to scale, parameter efficiency, and emergent capabilities. Our analysis covers essential evaluation metrics, including forgetting rates and knowledge transfer efficiency, along with emerging benchmarks for assessing CL performance. While current methods show promising results, fundamental challenges persist in achieving seamless knowledge integration, and we identify key open problems and future directions for the field.
Figures & tables
Fig. 1: A schematic overview of continual learning (CL) methodologies for large language models (LLMs). The outermost ring shows the three training stages (continual pre-training, continual fine-tuning, and continual alignment); inner nodes show method families within each stage (e.g., rehearsal, regularization, architecture/PEFT, RL-free, and RL-based); cited works appear as leaf nodes. Color coding indicates the primary knowledge-dynamic mode: collision mitigation (red family), isolation (green family), or fusion (purple family). Arrows indicate the sequential dependency between stages.
| Paradigm | Storage | Training | Inference | Main Advantage |
| Rehearsal | High (buffers or stored samples) | Medium (joint replay updates) | No extra latency | Most direct mitigation of forgetting; strong fusion potential [ 70 , 22 ] |
| Regularization | Low (masks or importance statistics) | Medium (constraint-based updates) | No extra latency | Stable parameter budget; works without stored data [ 45 , 46 ] |
| Architecture-based | Medium (growing task-specific modules) | Low (module-focused updates) | Small extra latency | Strong isolation; near-zero forgetting potential; best suited for fine-tuning scale [ 78 , 40 ] |
| Hybrid (MoE/PEFT) | Medium (experts, adapters, or routing) | Low (sparse PEFT-style updates) | Adaptive (router-dependent) | Scalable LLM adaptation; high flexibility via sparse activation [ 51 , 105 ] |
TABLE I: Stage-aware qualitative trade-offs among major continual learning paradigms.
Fig. 2: Qualitative resource-cost vs. retention-benefit map for representative CL methods across the three training stages (CPT, SFT, Align). Methods in the upper-left quadrant are deployment-friendly; axes are normalized for conceptual comparison. Architecture-based and hybrid methods dominate the efficient frontier at the fine-tuning scale, while rehearsal methods remain competitive at the pre-training scale.
Fig. 3: Knowledge dynamics and intervention mechanisms in continual learning for LLMs. Left: new updates may collide with, isolate from, or fuse with prior representations. Right: representative method families regulate these dynamics by revisiting historical support, restricting drift, allocating modules, or reweighting preference memories.
| Paper | Rehearsal | Data augmentation | Process optimization | Architecture-based | Base model |
| Fujii et al. [22] | ✓ | ✗ | ✗ | ✗ | LLaMA2 |
| Cui et al. [13] | ✓ | ✗ | ✗ | ✗ | LLaMA-2 7B/13B |
| Que et al. [64] | ✓ | ✗ | ✗ | ✗ | Qwen-1.5 |
| Zheng et al. [104] | ✓ | ✗ | ✗ | ✗ | LLaMA2 |
| Gu et al. [23] | ✓ | ✗ | ✗ | ✗ | LLaMA architecture |
| Xi et al. [91] | ✓ | ✗ | ✗ | ✗ | LLaMA-3 8B/70B |
TABLE II: Representative continual pre-training methods and their main design choices
| Paper | Rehearsal | Regularization-based | Architecture-based | Base model |
| Scialom et al. [70] | ✓ | ✗ | ✗ | T0 |
| Mok et al. [59] | ✓ | ✓ | ✗ | BART |
| Zhao et al. [102] | ✓ | ✗ | ✓ | T5, LLaMA-2 |
| Wang et al. [78] | ✗ | ✗ | ✓ | T5-large |
| Razdaibiedina et al. [67] | ✗ | ✗ | ✓ | T5, BERT |
| Wang et al. [84] | ✗ | ✗ | ✓ | BERT |
TABLE III: Representative continual fine-tuning methods and their main design choices
Fig. 4: Average performance (AP, higher is better) vs. average forgetting rate (F.Ra, lower is better) for seven continual fine-tuning methods under a unified T5-Large protocol (averaged over Long Sequence and SuperNI benchmarks, two task orders each). Points in the upper-left are preferable. Color indicates method family: gray = sequential baseline, blue = regularization, green = architecture/PEFT.
| Method | Long Sequence Benchmark | SuperNatural Instructions | Average | |||||||
| Order 1 | Order 2 | Order 3 | Order 4 | AP↑ | F.Ra↓ | |||||
| AP↑ | F.Ra↓ | AP↑ | F.Ra↓ | AP↑ | F.Ra↓ | AP↑ | F.Ra↓ | |||
| SeqLoRA | 34.48 | 44.92 | 55.79 | 19.84 | 7.30 | 47.60 | 7.03 | 47.97 | 26.15 | 40.08 |
| IncLoRA | 37.79 | 40.91 | 58.00 | 15.60 | 12.33 | 41.93 | 16.65 | 36.56 | 31.19 | 33.75 |
| LFPT5 | 66.62 | 14.57 | 67.40 | 13.20 | 39.03 | 10.87 | 29.70 | 20.72 | 50.69 | 14.84 |
| EWC | 43.24 | 23.66 | 46.25 | 32.90 | 15.32 | 26.78 | 18.19 | 30.28 | 30.75 | 28.41 |
TABLE IV: Overall results of T5-Large on two continual fine-tuning benchmarks under four task orders. LSB = Long Sequence Benchmark; SNI = SuperNatural Instructions. AP↑ = average performance (higher is better); F.Ra↓ = forgetting rate (lower is better). Best value in each column is bold .
| Method | Type | Base Model | Mechanism | Key Idea |
| COPR [ 96 ] | RL-free | DPO-based | C | Regularizes current preference learning with optimal-policy targets from prior stages |
| LEMoE [ 77 ] | RL-free | MoE-based | I | Expert routing with consistency constraints to isolate preference-specific modules |
| BaFT [ 57 ] | RL-free | LLM | I | Input-conditioned module composition with expert freezing for selective alignment editing |
| CPPO [ 97 ] | RL (PPO) | PPO-based | C | Extends PPO with retention-aware weighting to balance policy improvement and stability |
| LifeAlign [ 54 ] | RL (PPO) | LLM | F | Memory-augmented focalized preference optimization; consolidates preference traces over time |
| LANCET [ 98 ] | RL-free | LLM | C | Traces undesirable behaviors to influential training data and applies targeted corrections |
TABLE V: Representative continual alignment papers. “Type” indicates whether the method avoids (RL-free) or uses (RL) reinforcement learning. “Mechanism” maps each method to the knowledge-dynamics framework: C = collision mitigation, I = isolation, F = fusion/memory integration.
| Benchmark | Stage | # Tasks | Domain | Forgetting metric | Notes | Open-ended |
| MMLU + GSM8K | CPT | 2+ | General/Math | AP, F.Ra | Widely used as CPT retention gauges | No |
| TRACE [ 79 ] | CFT | 8 | Multi-domain | AP, F.Ra | Covers code, math, multilingual, domain tasks | Yes |
| CITB / InstrDialog++ [ 100 ] | CFT | 16 | Instructions | AP, F.Ra, BWT | Rich natural-language instruction stream | Yes |
| FUAR [ 34 ] | CFT | 3 | Knowledge | FUAR metric | Evaluates retention, update, and acquisition | No |
| Prog. Prompts (15-task) [ 67 ] | CFT | 15 | Text classif. | AP, F.Ra, BWT, FWT | Long sequence; widely adopted for comparison | No |
| StreamBench [ 87 ] | CFT/Agent | Stream | Multi-task | Improvement rate | Focuses on in-context online improvement | Yes |
TABLE VI: Comparison of representative continual learning benchmarks. “Stage” indicates the primary training stage targeted; “# Tasks” is the number of tasks or task streams; “Forgetting metric” lists the evaluation metrics provided; “Open-ended” indicates whether free-form generation (rather than classification) is the primary evaluation format.
| Collision mitigation | Isolation | Fusion | |
| Cont. pre-training | Data augmentation [ 25 , 92 ] ; process optimization [ 36 ] (C) | Adapter plugins [ 40 ] ; DGA/DAS [ 44 ] ; LLaMA-MoE [ 105 ] (I) | Rehearsal data mixing [ 22 , 13 , 64 ] (F) |
| Cont. fine-tuning | Regularization inside LoRA adapters [ 78 ] (C) | O-LoRA [ 78 ] ; Progressive Prompts [ 67 ] ; L2R [ 4 ] (I) | SAPT shared attention [ 102 ] ; InsCL replay [ 80 ] ; SSR [ 33 ] (F) |
| Cont. alignment | COPR [ 96 ] ; CPPO [ 97 ] ; LANCET [ 98 ] (C) | LEMoE [ 77 ] ; BaFT [ 57 ] (I) | LifeAlign memory consolidation [ 54 ] (F); — |
TABLE VII: Three-stage knowledge-dynamics mapping. Each cell lists representative method(s) and their primary role. C = collision mitigation; I = isolation; F = fusion. “—” indicates that the combination is not currently well-explored.
More in this topic
Continual Learning for LLMs
Continual learning for large language models studies how models absorb new knowledge and skills while retaining previously learned capabilities.
See all 233 papers