MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning

Yiyang Lu; Yu He; Jianlong Chen; Hongyuan Zha

cs.LGMar 10, 2026

MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning

Authors: Yiyang Lu, Yu He, Jianlong Chen, Hongyuan Zha

Organizations: The Chinese University of Hong Kong, Shenzhen · Nanjing University

Abstract

Continual fine-tuning of large language models (LLMs) is becoming increasingly crucial as these models are deployed in dynamic environments where tasks and data distributions evolve over time. While strong adaptability enables rapid acquisition of new knowledge, it also exposes LLMs to catastrophic forgetting, where previously learned skills degrade during sequential training. Existing replay-based strategies, such as fixed interleaved replay, accuracy-supervised, and loss-driven scheduling, remain limited: some depend on heuristic rules and provide only partial mitigation of forgetting, while others improve performance but incur substantial computational overhead. Motivated by retention dynamics under sequential fine-tuning, we propose Memory-Inspired Sampler and Scheduler Replay (MSSR), an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting while maintaining fast adaptation. Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines, with particularly strong gains on reasoning-intensive and multiple-choice benchmarks.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of replay triggering strategies in continual fine-tuning. (a) Fixed replay performs replay at a constant interval, ignoring optimization dynamics. (b) Loss-based replay triggers replay when the loss exceeds a threshold, but noisy high-frequency fluctuations can cause frequent spurious triggers. (c) Accuracy-based replay reacts to evaluation drops, yet often suffers from lag since replay starts after accuracy has already degraded. (d) MSSR (ours) is time-aware and memory-inspired, scheduling replay based on time-dependent retention to stabilize long-term performance.

Figure 2: Overall architecture of the MSSR framework. The framework consists of two core components that jointly govern replay behavior. (1) a sample-level replay sampler (left), which tracks per-sample memory strength by modeling loss-driven and time-dependent decay, and converts memory states into probabilistic replay weights. (2) an adaptive replay scheduler (right), which which regulates replay timing via expanding intervals and replay volume via a time-decaying ratio. At each replay event, sampled data are merged with current-task samples and used for LoRA-based fine-tuning, forming a closed-loop, memory-aware continual learning process.

Model	Mistral-7B-v0.3				Llama-3.1-8B				Qwen2.5-7B
Dataset	MMLU	GSM8K	MATH	AVG	MMLU	GSM8K	MATH	AVG	MMLU	GSM8K	MATH	AVG
Vanilla	60.2	57.7	27.5	48.5	64.5	63.8	29.3	52.5	57.6	70.1	28.6	52.1
R ${}_{\text{fixed}}$	60.5	58.8	28.3	49.2	65.1	64.4	29.5	53	58.5	70.5	29.3	52.8
R ${}_{\text{loss}}$	60.7	58.4	28.8	49.3	65.3	64.7	29.6	53.2	58.4	72.3	30.4	53.7
R ${}_{\text{accu}}$	61.0	59.3	29.4	49.9	64.9	64.5	29.9	53.1	59.5	71.7	29.9	53.7
MSSR ${}_{\text{sch}}$	60.8	59.5	28.6	49.6	64.8	65.3	29.5	53.2	59.1	71.9	31.1	54.3

Table 1: Basic 3-task continual learning results. Best results are shown in bold and second-best are underlined within each backbone.

Model	Method	AGNews	SQuAD	SciQ	BoolQ	ARC	GSM8K	MATH 1	MATH 2	MATH 3	MATH 4	MATH 5
Gemma2-9B	None	0.714	73.98	0.952	0.889	0.336	0.520	0.778	0.688	0.649	0.425	0.242
	Fixed	0.761	74.23	0.944	0.876	0.388	0.638	0.873	0.743	0.626	0.387	0.228
	Loss	0.758	73.42	0.961	0.890	0.425	0.643	0.847	0.726	0.632	0.394	0.257
	Accu	0.754	74.67	0.959	0.901	0.441	0.636	0.864	0.731	0.650	0.392	0.276
	MSSR ${}_{\text{sch}}$	0.772	75.83	0.952	0.906	0.456	0.654	0.834	0.725	0.649	0.409	0.302
	MSSR ${}_{\text{spl}}$	0.769	76.10	0.957	0.903	0.462	0.646	0.852	0.741	0.635	0.399	0.263

Table 2: Extended 11-task continual learning results. Best results are shown in bold and second-best are underlined within each backbone.

Buffer Size	Average Retention (%)
512	62.1
1024	71.4
2048	78.9
4096	80.3

Table 3: Effect of buffer size on retention performance under the Ebbinghaus scheduler.

Interval Sequence	Acc. (%)	Forget ↓
Fixed ( $\Delta t_{r}{=}3$ )	84.6	0.0
Geometric $\{1,3,7,14,30\}$	87.2	2.6
Ebbinghaus $\{1,2,4,7,15\}$	89.1	4.5

Table 4: Replay interval comparison. Ebbinghaus-inspired spacing achieves the highest accuracy and lowest forgetting. (Uniform, Heuristic, and Cognitive correspond to the three row types.)

Strategy / $\lambda_{0}$	fixed	loss	eval	0.05	0.10	0.20	0.30
GSM8K Avg Acc	0.706	0.698	0.727	0.716	0.725	0.748	0.731
BoolQ Avg Acc	0.876	0.854	0.865	0.874	0.892	0.889	0.883

Table 5: Sensitivity to initial replay ratio

\lambda_{0}

(Qwen2.5-7B, 11-task sequence).

Buffer Size (samples)	512	1024	2048	4096
Average Retention (%)	68.7	73.5	75.9	76.8

Table 6: Effect of buffer size on average retention (%).

Method	Wall-clock Time	Peak Memory	Throughput (samples/s)	Per-step Latency
Fixed Replay	1.00	1.00	1.00	1.00
MSSR ${}_{\text{spl}}$	1.03	1.04	0.99	1.02
MSSR ${}_{\text{sch}}$	1.04	1.05	0.98	1.02
MSSR ${}_{\text{full}}$	1.05	1.06	0.98	1.05

Table 7: Training and memory overhead of MSSR (normalized to fixed replay).

Interval Pattern	Interval Sequence	Avg Acc (%)	Forget Drop (%)
Fixed	3,3,3,…	76.1	4.8
Geometric	1,3,7,15,…	77.8	3.1
MSSR	1,2,4,7,15,…	78.5	2.4

Table 8: Effect of interval scheduling on average accuracy (%) and forgetting drop (%).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Default	Candidate Values	Role	Where used
$\alpha$ (shared)	$0.01$	$\{0,\,0.005,\,0.01,\,0.02,\,0.05\}$	baseline decay	Eq. ( 4 ), Eq. ( 7 )
$\gamma_{d}$	$0.20$	$\{0.05,\,0.10,\,0.20,\,0.30,\,0.50\}$	loss sensitivity	Eq. ( 4 ), Eq. ( 7 )
$\phi(\cdot)$	sigmoid	—	monotone mapping	Eq. ( 4 ), Eq. ( 7 ) (via $\phi(\widehat{\ell})$ )
$k$ (sigmoid)	$10$	$[5,20]$	slope / hardness band width	Eq. ( 20 )
$c$ (sigmoid)	$0.5$	$[0.4,0.6]$	center of difficulty band	Eq. ( 20 )
EMA $\beta_{\mathrm{ema}}$	$0.95$	$\{0.90,\,0.95,\,0.97,\,0.99\}$	loss denoising	Eq. ( 13 )

Appendix

Table 9: Hyperparameters for the sample-level memory dynamics (Appendix A ).

Parameter	Default	Candidate Values	Role	Where used
$\Delta t_{r}^{(1)}$	$100$	$\{50,\,100,\,200,\,500\}$	initial replay interval	Eq. ( 26 )
$\theta$	$0.5$	$\{0.3,\,0.5,\,0.7\}$	replay trigger threshold	Eq. ( 22 )
$\eta_{p}$	$0.5$	$\{0.1,\,0.3,\,0.5,\,0.7,\,1.0\}$	early expansion gain	Eq. ( 25 ), Eq. ( 26 )
$\rho_{p}$	$0.05$	$\{0.01,\,0.03,\,0.05,\,0.10\}$	decay rate of expansion gain	Eq. ( 25 ), Eq. ( 26 )
$\lambda_{0}$	$0.3$	$\{0.1,\,0.3,\,0.5\}$	initial replay ratio	Eq. ( 29 )
$\lambda_{\min}$	$0.05$	$\{0,\,0.01,\,0.05,\,0.10\}$	minimal replay ratio	Eq. ( 29 )

Appendix

Table 10: Hyperparameters for dataset-level replay scheduling (Appendix B ).

Item	Setting
Backbone model	Gemma2-9B / Qwen2.5-7B / LLaMA-3.1-8B
Fine-tuning framework	LLaMAFactory ( Zheng et al., 2024 )
LoRA rank $r$	8
LoRA scaling $\alpha_{\mathrm{lora}}$	16
LoRA target modules	q_proj , v_proj (all Transformer layers)
Optimizer	AdamW

Appendix

Table 11: LoRA fine-tuning and optimization hyperparameters (shared across replay strategies).

Item	Setting
Replay buffer $B_{\mathrm{replay}}$ size	1024 samples (maintained within each stage)
Buffer refresh interval	every 50 training steps
Replay batch composition	mix current-task samples with replayed samples at ratio $\lambda_{t}$
Replay sampling distribution	$p_{i}^{(t_{k})}\propto m_{i,t_{k}}^{-\zeta}$ (Eq. ( 31 ))
Lazy state update	compute $m_{i,t}$ on demand using the latest epoch-/cycle-level hazard estimate (App. A.4 )
Staleness safety cap	force refresh if $\Delta t_{i}$ exceeds a fixed cap (default: 10 epochs)

Appendix

Table 12: Practical replay implementation settings.

Continual Learning for LLMs

Continual learning for large language models studies how models absorb new knowledge and skills while retaining previously learned capabilities.

See all 233 papers

Attribution-Guided Continual Learning for Large Language Models
May 6, 2026Yazheng Liu, Yuxuan Wan, Rui Xu +3Continual Learning for LLMs LLM Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Beijing University of Posts and Telecommunications, Beijing, China
Forget About It: Mitigating Forgetting Without Replay or Auxiliary Objectives
Jan 9, 2025Anat Kleiman, Gintare Karolina Dziugaite, Jonathan Frankle +2Catastrophic Forgetting Continual Learning for LLMs
Harvard University, Kempner Institute · Google DeepMind · Databricks
On-Policy Replay for Continual Supervised Fine-Tuning
May 28, 2026Yan Chen, Taojie Zhu, Meng Zhang +4Continual Learning for LLMs Replay-Based Continual Learning
Tsinghua University · Alibaba Group

May 6, 2026J/K move · Enter open · S save

Attribution-Guided Continual Learning for Large Language Models

Yazheng Liu, Yuxuan Wan, Rui Xu, Xi Zhang+2

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Beijing University of Posts and Telecommunications, Beijing, China

Continual Learning for LLMs LLM Fine-Tuning Catastrophic Forgetting

Open paper AskPDF arXiv

Large language models (LLMs) often suffer from catastrophic forgetting in continual learning: after learning new tasks sequentially, they perform worse on earlier tasks. Existing methods mitigate catastrophic forgetting by data replay, parameter freezing, or regularization. However, these methods lack understanding of LLM mechanisms and cannot distinguish which parameters store important knowledge from previous tasks and which parameters can be updated for new tasks. To address this, we propose the attribution-guided continual fine-tuning framework that leverages Layer-wise Relevance Propagation (LRP) to estimate parameter importance based on the internal computational process of LLMs. During continual learning, parameters critical to previous tasks are constrained to receive smaller updates, while less relevant parameters remain available for learning new tasks. Extensive experiments show that, compared with baseline methods, our approach reduces catastrophic forgetting while preserving adaptability to new tasks, highlighting the value of mechanistic attribution for continual fine-tuning of LLMs.

Abstract

Continual Learning for LLMs

Explore similar work

Attribution-Guided Continual Learning for Large Language Models

Forget About It: Mitigating Forgetting Without Replay or Auxiliary Objectives

On-Policy Replay for Continual Supervised Fine-Tuning