Representation Drift in Continual Learning: LoRA versus N-gram Memory

Hao Wu

cs.LGOct 10, 2026

Representation Drift in Continual Learning: LoRA versus N-gram Memory

Authors: Hao Wu

Organizations: Illinois Institute of Technology

Abstract

A language model that keeps learning online must not slowly erode its own base capabilities. We ask which part of a modern LLM should absorb continual updates, by measuring how each choice changes the model's internal representations. On a 48-layer mixture-of-experts model with a native pretrained N-gram memory, we train a seven-task chain under three update mechanisms with identical data and budgets: updating only the N-gram memory table, continually training one shared LoRA adapter, and training a fresh LoRA per task that is merged into the backbone. We capture fixed-context representations at every layer and task boundary. Updating only the N-gram memory changes the base model's representations of general math and code inputs by 26%/15% (relative RMS), against 64%/35% for merged and 97%/39% for shared LoRA. This drift does not grow with the number of tasks: it reaches 25% after the first task and stays there for all seven, while both LoRA variants keep accumulating. Its generic behaviour is essentially unchanged (no code problem lost, math NLL -0.01), and every change is confined to inspectable, reversible table rows. The cost appears elsewhere: N-gram memory acquires new tasks more weakly and its stored tasks interfere with one another, losing as much old-task NLL as shared LoRA. We show that this forgetting is governed by drift on the learned tasks' own inputs, measured from the checkpoint at which each was learned, not by generic drift (Spearman rho=0.85; rho=0.71 on Llama-3-8B), and we derive the exact relations behind it: with a frozen head, every pairwise logit margin changes by the projection of drift onto a head-difference direction. These results support a design principle for long-running online learning: write new knowledge only into the memory table and keep the backbone fixed; the remaining problem, interference between stored tasks, then lies inside the memory.

Figures & tables

Method	Domain	Rel. RMS	Mean cos	Median	P95	Norm ratio	Radial	Direct.	Dir. share	CKA	$\Delta$ NLL
N-gram	math	26.37%	0.9658	13.48%	58.37%	1.0059	0.0020	0.0676	97.17%	0.627	$-0.0119$
	code	15.13%	0.9889	12.49%	23.58%	1.0019	0.0003	0.0226	98.66%	0.986	$-0.0022$
Shared	math	97.33%	0.4502	97.75%	105.27%	0.8274	0.0358	0.9116	96.22%	0.366	$-0.1001$
	code	38.90%	0.9263	34.76%	54.89%	1.0003	0.0014	0.1499	99.09%	0.896	$+0.0802$
Merged	math	64.36%	0.7894	52.64%	113.52%	0.9275	0.0143	0.3998	96.54%	0.127	$-0.0865$
	code	34.98%	0.9387	32.58%	51.61%	0.9639	0.0026	0.1198	97.86%	0.913	$+0.0246$

Table 1: Big-

K

drift

F_{0}\to F_{7}

(head input, last prompt token). Radial and directional energies are the two terms of Equation 7 , pooled and normalised by base energy; their sum is the squared relative RMS.

\Delta

NLL is the teacher-forced reference-answer loss change relative to

F_{0}

Figure 1: Big- $K$ drift by layer for the seven rounds (one column per method, one row per generic domain; one curve per round). Layer curves are four residual streams concatenated (10,240 dimensions).

		Math			Code
Method	Round	Drift	Cos	NLL	Drift	Cos	NLL
N-gram	1	25.21%	0.968	0.881	13.40%	0.991	0.200
	3	24.81%	0.970	0.878	14.66%	0.990	0.199
	5	26.14%	0.966	0.874	14.22%	0.990	0.199
	7	26.37%	0.966	0.876	15.13%	0.989	0.199
Shared	1	45.51%	0.893	0.750	24.91%	0.969	0.210

Table 2: Big-

K

trajectory at the head input: drift from

F_{0}

after each round, and reference NLL (base: math 0.888, code 0.201).

Method	Domain	Final drift	$\sum_{j\leq 7}E_{j}$	$R_{7}$
N-gram	math	26.37%	0.4781	0.145
	code	15.13%	0.1207	0.190
Shared	math	97.33%	1.6958	0.559
	code	38.90%	0.3865	0.392
Merged	math	64.36%	1.0220	0.405
	code	34.98%	0.3019	0.405

Table 3: Cross-round accumulation of big-

K

drift ( Equation 9 ).

\sum_{j}E_{j}

is the sum of per-round squared changes, normalised by base energy;

R_{7}<1

means the cross-round inner products sum to a negative number.

Figure 2: Left: big-

K

drift relative to

F_{0}

over seven rounds. Right: cosine between round

j

’s change and the change accumulated before it,

(D_{j}-D_{j-1}-E_{j})/(2\sqrt{D_{j-1}E_{j}})

Method	Domain	L1	L12	L24	L36	L48	Max layer
N-gram	math	0.00%	4.25%	8.62%	34.52%	36.39%	L48 (36.39%)
	code	0.00%	2.50%	5.06%	14.63%	14.57%	L43 (15.53%)
Shared	math	12.76%	39.50%	56.58%	139.45%	99.73%	L36 (139.45%)
	code	6.68%	18.45%	25.35%	41.55%	38.78%	L34 (41.82%)
Merged	math	9.09%	19.02%	34.40%	84.60%	81.09%	L36 (84.60%)
	code	7.13%	14.76%	19.78%	35.31%	34.80%	L42 (35.84%)

Table 4: Big-

K

drift

F_{0}\to F_{7}

at sampled layer outputs (four residual streams concatenated).

Domain	Hook	Method	L1	L48	Median
math	input	N-gram	0.05	22.21	10.59
		Shared	14.19	88.73	72.91
		Merged	8.34	58.05	44.41
	output	N-gram	0.09	74.50	19.72
		Shared	17.93	123.51	112.13
		Merged	11.48	151.05	69.87

Table 5: Big-

K

drift

F_{0}\to F_{7}

at MLP/MoE blocks (%). “Median” is the median over the 48 per-layer pooled drifts. Input and output are each normalised by their own base energy, so the two columns must not be subtracted.

Figure 3: Per-layer big-

K

drift

F_{0}\to F_{7}

at MLP/MoE block input and aggregated output.

Method	Domain	q25	q50	q75	q100
N-gram	math	12.66%	13.66%	20.40%	15.00%
	code	7.95%	10.65%	10.68%	17.95%
Shared	math	50.49%	53.08%	54.36%	62.74%
	code	21.98%	26.21%	27.95%	53.03%
Merged	math	36.69%	40.50%	39.90%	46.41%
	code	20.47%	23.21%	23.28%	43.42%

Table 6: Big-

K

drift

F_{0}\to F_{7}

at the head input by reference-answer position.

Task	Method	Rel. RMS	Mean cos	Median	P95	Norm ratio	Dir. share	$\Delta$ NLL
SME	N-gram	40.43%	0.9205	38.00%	56.79%	1.0099	98.18%	$+0.236$
	Shared	50.65%	0.8836	47.32%	67.03%	1.0703	96.83%	$+0.356$
	Merged	37.19%	0.9313	36.03%	49.12%	0.9972	99.01%	$\mathbf{+0.210}$
NAV	N-gram	61.98%	0.8172	61.78%	77.22%	1.0465	99.07%	$+0.753$
	Shared	60.60%	0.8223	57.86%	78.65%	1.0262	99.11%	$+1.018$
	Merged	47.43%	0.8881	46.34%	59.10%	1.0022	99.37%	$\mathbf{+0.450}$

Table 7: Small-

k_{i}

drift

F_{i}\to F_{7}

on each learned task’s 64 probes (head input, last prompt token), with the change in teacher-forced reference NLL over the same interval. AMH is learned last and has no subsequent training.

Figure 4: Per-layer drift (four residual streams concatenated, last prompt token). (a) Big

K

on math probes,

F_{0}\to F_{7}

. (b) Small

k_{i}

, mean over the six learned tasks with min–max band,

F_{i}\to F_{7}

. The dotted line marks the N-gram read at layer 2.

Figure 5: Small- $k_{i}$ drift $F_{i}\to F_{t}$ for every learned task $i$ and later round $t$ (head input, last prompt token).

Method	Math NLL	$\Delta$	Code NLL	$\Delta$	Old-task mean $\Delta$ NLL	Big- $K$ drift
N-gram	0.8759	$-0.0119$	0.1985	$-0.0022$	$+0.3052$	20.75%
Shared	0.7877	$-0.1001$	0.2809	$+0.0802$	$+0.2995$	68.12%
Merged	0.8036	$-0.0865$	0.2244	$+0.0246$	$\mathbf{+0.1573}$	49.67%

Table 8: Final (

F_{7}

) reference NLL: generic probes relative to

F_{0}

, learned tasks relative to their acquisition checkpoint

F_{i}

(mean over the six tasks with later training), next to the mean big-

K

drift.

Figure 6: Big-

K

drift does not track forgetting; small-

k_{i}

drift does. (a,c) Mean old-task

\Delta

NLL against big-

K

drift: the memory arm has the smallest generic drift but not the least forgetting. (b,d) Per-task

\Delta

NLL against small-

k_{i}

drift

F_{i}\to F_{7}

for all 18 task–method pairs. (a,b) Qwen3.8 with native N-gram memory; (c,d) Llama-3-8B with an added zero-initialised lookup table.

Domain ( $n$ )	N-gram	Shared	Merged
Math (128)	$+0.036$	$-4.350$	$-3.103$
Code (64)	$+0.011$	$-0.038$	$+0.345$
Seven tasks (448)	$+2.172$	$+4.067$	$+4.412$

Table 9: Median paired change of first-reference-token margin

F_{0}\to F_{7}

(logits).

Figure 7: Median first-reference-token margin change $F_{0}\to F_{7}$ by domain and method.

Task	N-gram	Shared	Merged
SME	13 $\to$ 13 (0.0)	32 $\to$ 22 ( $-15.6$ )	31 $\to$ 27 ( $-6.3$ )
NAV	0 6 $\to$ 0 4 ( $-3.1$ )	23 $\to$ 0 8 ( $-23.4$ )	23 $\to$ 11 ( $-18.8$ )
HYE	61 $\to$ 55 ( $-9.4$ )	63 $\to$ 55 ( $-12.5$ )	64 $\to$ 57 ( $-10.9$ )
AFB	42 $\to$ 35 ( $-10.9$ )	52 $\to$ 49 ( $-4.7$ )	51 $\to$ 49 ( $-3.1$ )
SQI	50 $\to$ 45 ( $-7.8$ )	53 $\to$ 46 ( $-10.9$ )	51 $\to$ 48 ( $-4.7$ )
MKD	54 $\to$ 46 ( $-12.5$ )	59 $\to$ 57 ( $-3.1$ )	56 $\to$ 57 ( $+1.6$ )

Table 10: Greedy exact match on the 64 probes of each task, from acquisition (

F_{i}

) to

F_{7}

(percentage-point change).

	Math acc.	Lost / gained	Unparsed	Code acc.	Lost / gained
Base	71/128 (55.5%)	–	56	51/64 (79.7%)	–
N-gram	68/128 (53.1%)	10 / 7	59	55/64 (85.9%)	0 / 4
Shared	32/128 (25.0%)	47 / 8	96	52/64 (81.3%)	7 / 8
Merged	94/128 (73.4%)	1 / 24	33	55/64 (85.9%)	6 / 10

Table 11: Generic capabilities at

F_{7}

: greedy accuracy on 128 MATH-500 problems and 64 HumanEval problems (pass@1, sandboxed WASI execution), with paired per-problem transitions relative to the base, and the number of math outputs whose final answer could not be parsed (counted as wrong).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Drift	Mean cos	$\Delta$ NLL	Drift	Mean cos	$\Delta$ NLL
	Math			Code
Lookup	4.86%	0.9988	$+0.0024$	7.29%	0.9973	$-0.0002$
Shared	69.10%	0.7661	$+0.0712$	31.87%	0.9496	$+0.0590$
Merged †	39.04%	0.9245	$+0.0266$	21.51%	0.9771	$+0.0217$

Appendix

Table 12: Llama-3-8B big-

K

drift

F_{0}\to F_{7}

and reference NLL change.

	Small- $k_{i}$ drift			$\Delta$ NLL
Task	Lookup	Shared	Merged †	Lookup	Shared	Merged †
SME	54.58%	75.91%	73.86%	$+1.051$	$+1.496$	$+0.838$
NAV	86.25%	96.38%	75.07%	$+1.730$	$+2.166$	$+1.181$
HYE	61.74%	66.16%	62.45%	$+0.170$	$+0.186$	$+0.087$
AFB	59.38%	77.42%	52.60%	$+0.374$	$+1.023$	$+0.221$
SQI	57.52%	65.08%	57.99%	$+0.422$	$+0.518$	$+0.417$

Appendix

Table 13: Llama-3-8B small-

k_{i}

drift

F_{i}\to F_{7}

and task reference

\Delta

NLL.

		Math			Code
Method	Round	Drift	Cos	NLL	Drift	Cos	NLL
N-gram	1	25.21%	0.9682	0.8811	13.40%	0.9914	0.1995
	2	22.16%	0.9750	0.8767	14.67%	0.9896	0.1990
	3	24.81%	0.9695	0.8780	14.66%	0.9897	0.1987
	4	20.95%	0.9778	0.8774	14.19%	0.9903	0.2003
	5	26.14%	0.9664	0.8741	14.22%	0.9901	0.1994

Appendix

Table 14: Qwen3.8 big-

K

drift from

F_{0}

, mean cosine and reference NLL at every task boundary (head input, last prompt token). Base NLL: math 0.8878, code 0.2007.

Continual Learning for LLMs

Continual learning for large language models studies how models absorb new knowledge and skills while retaining previously learned capabilities.

See all 231 papers

May 7, 2026

CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

Large language models (LLMs) can learn new tasks through fine-tuning, but learning tasks sequentially often leads to catastrophic forgetting of earlier ones. This paper proposes CRAFT, a continual learning framework that keeps model weights frozen and instead learns low-rank interventions on hidden representations. Similar tasks share an intervention to help knowledge transfer, with similarity judged by how each task alters the model's output distribution rather than by a learned gating network. Within a shared intervention, a Kullback-Leibler (KL) penalty keeps each new task close to the intervention's existing behavior, which limits forgetting of earlier tasks, while a small, task-private intervention in the deepest layers preserves dedicated capacity that later tasks cannot overwrite. The learned interventions also make it possible to identify the task at test time, so task identity labels are not needed. Across multiple benchmarks and model scales, CRAFT improves overall performance and reduces forgetting compared with strong LoRA-based methods, while remaining robust to task ordering.

Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz +2

Iowa State University · Maro Systems, USA · University of California, Berkeley +1

Continual Learning for LLMs Low-Rank Adaptation Incremental Learning Catastrophic Forgetting

Ask PDF

Jul 26, 2026

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting. To mitigate this, LoRA-based continual learning methods allocate a separate low-rank adapter per task, yet existing approaches either require task identity at inference or sum all adapters indiscriminately, letting irrelevant branches distort the output. Recent gating-based solutions route inputs to the correct adapter but introduce trainable parameters that themselves need protection against forgetting. In this work, we observe that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence. A Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test time. This eliminates the need for a learned gating module. On the adapter side, constraining each task's parameters to the principal subspace of the pretrained weights via SVD yields a compact latent-space parameterization. Within this subspace, orthogonal regularization directly controls inter-task interference. The resulting system, Latent-LoRA, is replay-free, requires no trainable routing component, and uses substantially fewer parameters per task. Experiments across five model scales and two established continual learning benchmarks show state-of-the-art performance with near-zero forgetting.

Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli +2

College of Health Solutions, Arizona State University, Phoenix, Arizona, USA · Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA +1

Continual Learning for LLMs Continual LoRA Incremental Learning Parameter-Efficient Fine-Tuning Adapter-Based Continual Learning

Ask PDF

Sep 30, 2026

ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

Continual parameter-efficient fine-tuning for large language models (LLMs) must balance retention of previously acquired knowledge, adaptation to new tasks, and strict parameter budgets. We present ChainLoRA, a replay-free continual merging framework built on chain-updated task-vector geometry. From a parameter-merging perspective, we formulate a geometric view of forgetting through a measurable interaction between task updates, separating directional overlap from coefficient coupling. Building on this view, ChainLoRA combines chain-updated training with post-stream adaptive SVD merging. During training, initialization and a one-sided orthogonality proxy use only the last carrier, keeping their historical-state footprint and regularization overhead constant as the task stream grows. At merging time, Adaptive SVD extracts a shared carrier and aligns it to the latest task through Procrustes adaptation. Our theoretical analysis shows that Procrustes adaptation facilitates geometric approximate separation of shared and task-specific components. The one-sided proxy further bounds inter-task interference. An effective-rank penalty additionally promotes efficient utilization of the task subspace during continual learning. Experiments show that ChainLoRA achieves state-of-the-art performance among the evaluated replay-free methods on the Large and SuperNI benchmarks, while remaining competitive on Standard CL and attaining almost the closest average scores to the evaluated replay-based method across all three benchmarks.

Hang Yin, Haozhe Wang, Yuhua Luo +3

Shanghai Jiao Tong University · Shanghai Innovation Institute · Beijing Zhongguancun Academy +3

Continual Learning for LLMs Low-Rank Adaptation Task Vector Merging Language Model Merging Model Merging Parameter-Efficient Fine-Tuning

Ask PDF

Abstract

Continual Learning for LLMs

Explore similar work

CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs