CaLLM: Continual Adaptation of LLMs via Non-Parametric Adapter Routing

Bilge Celik; Anna Vettoruzzo; Joaquin Vanschoren

cs.LGOct 27, 2026

CaLLM: Continual Adaptation of LLMs via Non-Parametric Adapter Routing

Authors: Bilge Celik, Anna Vettoruzzo, Joaquin Vanschoren

Organizations: Eindhoven University of Technology

Abstract

Continual adaptation of large language models (LLMs) to evolving task distributions while mitigating catastrophic forgetting remains an open problem. Existing strategies often struggle to scale or lack a robust mechanism for zero-shot task-to-expert assignment. We introduce CaLLM, a framework that resolves these tensions by combining metric-based meta-learning with modular parameter isolation via lightweight adapters. First, CaLLM projects each input batch into a task-agnostic embedding space to find its nearest prototype. It then routes the batch to the specific adapter specialized for that data distribution. This formulation provides (i)~controllable cross-task interference by design, (ii)~constant computational complexity regardless of task count, (iii)~seamless integration of novel distributions through prototype expansion, and (iv)~regime agnosticism as the local routing decision enables a unified approach to both online and offline settings. Our experiments demonstrate that this routing mechanism discovers semantically coherent task groupings while facilitating constructive interference between similar tasks, and keeping overhead at a minimum. Across diverse and fine-grained continual learning benchmarks, CaLLM consistently outperforms traditional isolation and replay methods, establishing prototype-based modular routing as a principled and scalable approach to lifelong adaptation of LLMs.

Figures & tables

Figure 1: CaLLM system overview (training + evaluation). Each new data batch is embedded with a frozen base model (using mean-pooling). The non-parametric router manages a dynamic pool of specialized adapters, each mapped to a prototype. Based on semantic similarity between the new embedding and existing prototypes, the router triggers the reuse, creation, or repurposing of an adapter. The selected adapter is then trained or finetuned and the pool is updated. The diagram further details how this routing mechanism integrates with the distinct training and evaluation loops.

Figure 2: Router Embedding Space (2D Projection) for TRACE-8. t-SNE projections of the router’s embeddings, colored by task identity.

Task	Metric	Inf-Only	Seq-FT	EWC	ER	oLoRA	TreeLoRA 3 3 3 2 2 2 TreeLoRA is reported on TRACE-8 only, as its public implementation is TRACE-oriented and its cumulative checkpoint-wise inference protocol would make evaluation over the 19-task CITB stream computationally prohibitive.	CaLLM-B
C-STANCE	acc (0–1)	0.62	0.03	0.28	0.04	0.09	0.41	0.43
FOMC	acc (0–1)	0.64	0.30	0.31	0.14	0.06	0.51	0.68
MeetingBank	ROUGE-L (0–1)	0.21	0.06	0.10	0.10	0.16	0.11	0.23
Py150	EDIM (0–100)	14.91	11.96	10.55	12.98	9.81	5.52	17.24
ScienceQA	acc (0–1)	0.35	0.13	0.61	0.19	0.00	0.72	0.81
NumGLUE-cm	acc (0–1)	0.05	0.14	0.12	0.06	0.04	0.67	0.15

Table 1: Per-Task Final Performance on TRACE-8 (Batched). Final scores after training on all tasks sequentially.

Panel of Figure 3 — Figure 3: Task-Incremental Learning Performance (mean $\pm$ 1 std). Average normalized score as tasks are added sequentially, for CaLLM-B and all baselines. (a) TRACE-8. (b) CITB-19.

Figure 4: CaLLM Online Learning Dynamics (TRACE Stream, $P{=}8$ ). (a) OMA. (b) Task $\times$ expert assignment heatmap; aggregate adapter timeline on top and per-task routing below. See Appendix D.3 for the full 500-chunk view.

Figure 5: Router behavior under pool capacity constraints for TRACE-8 in Batched scenario.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Metric	Llama-3.1-8B-Instruct	gemma-3-12b-it	Qwen3-8B
C-STANCE	acc (0–1)	0.20	0.62	0.20
FOMC	acc (0–1)	0.14	0.64	0.04
MeetingBank	ROUGE-L (0–1)	0.05	0.21	0.04
Py150	EDIM (0–100)	6.42	14.91	0.00
ScienceQA	acc (0–1)	0.09	0.35	0.07
NumGLUE-cm	acc (0–1)	0.01	0.05	0.00

Appendix

Table 2: Inference-only (Zero-Shot) Performance on TRACE-8. gemma-3-12b-it achieves the highest score on all 8 tasks. All models show low performance on math, motivating the need for continual adaptation.

Setting	LR	$r$	$\alpha$	Epochs	Batch	GAS	LR Sched.
CaLLM-B, TRACE	$1\!\times\!10^{-4}$	16	32	3	16	4	cosine
CaLLM-B, CITB	$1\!\times\!10^{-3}$	8	16	5	25	4	cosine
CaLLM-O, TRACE	$3\!\times\!10^{-3}$	4	8	1	10	2	linear
CaLLM-O, CITB	$1\!\times\!10^{-3}$	8	8	1	20	1	cosine

Appendix

Table 3: CaLLM hyperparameters per scenario. All runs use the HuggingFace SFTTrainer with 8-bit AdamW, gradient clipping at norm 1.0, and bfloat16 mixed precision. LR = learning rate,

r

\alpha

= LoRA rank / scaling factor, Batch = batch size, GAS = gradient accumulation steps, LR Sched. = learning rate schedule.

Method	Config.	LR	Epochs	C-STANCE	FOMC	MeetingBank	Py150	ScienceQA	NumGLUE-cm	NumGLUE-ds	20Minuten
ER	Original	$1\times 10^{-4}$	3	0.04	0.14	0.10	12.98	0.19	0.06	0.13	39.63
ER	Tuned	$5\times 10^{-4}$	5	0.11	0.19	0.12	13.04	0.22	0.09	0.15	39.58
EWC	Original	$1\times 10^{-4}$	3	0.28	0.31	0.10	10.55	0.61	0.12	0.15	40.06
EWC	Tuned	$5\times 10^{-4}$	3	0.17	0.35	0.09	10.61	0.55	0.21	0.11	39.01

Appendix

Table 4: Baseline hyperparameter ablation on TRACE-8 (Batched). Original rows use the configuration from the main comparison; tuned rows are selected from the same learning-rate/epoch grid used for CaLLM while keeping LoRA rank, scaling factor, and dropout fixed.

Category	Task Type	Count
Dialogue State Tracking	Slot Filling, State Update	4
Dialogue Generation	Response Gen, Clarification	11
Intent Identification	Classification, Intent Detection	4
Total		19

Appendix

Table 5: The 19 dialogue tasks in the CITB InstrDialog stream.

Method	Pool $P$	Train TFLOPs	Samples/s	Peak GPU (GB)	Wall (h)	# Adapters
CaLLM-B	2	7,339	3.81	12.15	60.0	2
CaLLM-B	4	7,339	3.81	12.15	59.3	4
CaLLM-B	6	7,339	3.67	12.15	59.7	6
CaLLM-B	8	7,339	3.87	12.15	59.5	8
Seq-FT	1	7,339	3.53	12.15	68.2	1
EWC †	1	—	—	12.48	76.9	1

Appendix

Table 6: Compute and Memory Overhead on TRACE-8 (Batched). CaLLM consumes the same training FLOPs as single-adapter Seq-FT regardless of pool size, with constant GPU memory. ER doubles wall-clock time due to replay, and oLoRA requires

2.1\times

more GPU memory for orthogonal projections. † EWC, ER, and oLoRA use custom training loops that do not report total_flos ; EWC’s FLOPs are comparable to CaLLM (same data/rank), ER’s are higher due to replay augmentation, and oLoRA’s are lower due to smaller rank (

r{=}4

Panel of Figure 6 — Figure 6: Online Pool Constraints (500-chunk stream). OMA and BWT for CaLLM-O and the SeqFT baseline. Top row (TRACE-8): (a) OMA and (b) BWT for pool sizes $P\in\{4,6,8\}$ . Bottom row (CITB-19): (c) OMA and (d) BWT for pool sizes $P\in\{3,10,19\}$ . Knowledge accumulation (OMA) is successful across all configurations; performance is non-monotonic with respect to pool size on TRACE, while CITB shows comparable cumulative scores but diverging forgetting profiles.

Task	C-STANCE	FOMC	MeetingBank	Py150	ScienceQA	NumGLUE-cm	NumGLUE-ds	20Minuten
C-STANCE	0.42	0.42	0.42	0.42	0.41	0.42	0.42	0.42
FOMC	–	0.56	0.55	0.56	0.56	0.56	0.56	0.56
MeetingBank	–	–	0.22	0.23	0.23	0.23	0.23	0.23
Py150	–	–	–	16.94	16.95	16.85	17.15	17.12
ScienceQA	–	–	–	–	0.56	0.57	0.56	0.56
NumGLUE-cm	–	–	–	–	–	0.19	0.19	0.19

Appendix

Table 7: Per-Task Performance Table on TRACE-8 for CaLLM-B . Results are reported for a single random seed. Metrics used are accuracy (C-STANCE, FOMC, ScienceQA, NumGLUE-cm/ds), ROUGE-L (MeetingBank), EDIM (Py150), and SARI (20Minuten).

Task	task1590	task1713	task565	task1730	task573	task294	task1500	task1603	task576	task639	task1714	task1501	task1729	task574	task848	task611	task766	task1384	task1600
task1590	0.41	0.50	0.55	0.44	0.53	0.45	0.52	0.44	0.46	0.46	0.40	0.43	0.47	0.38	0.56	0.55	0.44	0.46	0.53
task1713	–	0.60	0.52	0.56	0.53	0.59	0.59	0.51	0.56	0.53	0.55	0.64	0.52	0.53	0.52	0.49	0.56	0.49	0.54
task565	–	–	0.37	0.41	0.38	0.39	0.33	0.31	0.37	0.33	0.29	0.29	0.21	0.36	0.37	0.29	0.45	0.29	0.33
task1730	–	–	–	0.24	0.15	0.21	0.18	0.14	0.18	0.18	0.20	0.21	0.18	0.21	0.19	0.13	0.20	0.14	0.18
task573	–	–	–	–	0.45	0.43	0.47	0.49	0.48	0.44	0.55	0.56	0.45	0.44	0.34	0.50	0.56	0.60	0.51
task294	–	–	–	–	–	0.11	0.11	0.12	0.09	0.10	0.12	0.13	0.15	0.13	0.13	0.11	0.16	0.09	0.14

Appendix

Table 8: Per-Task Performance Table on CITB-19 for CaLLM-B . Results are reported for a single random seed. The metric used is ROUGE-L for all tasks.

Panel of Figure 7 — Figure 7: Task-Incremental BWT Comparison Across Baselines (mean $\pm$ std). CaLLM-B maintains a BWT score close to zero throughout the stream on both benchmarks.

Task	Inf-Only	Seq-FT	EWC	ER	oLoRA	CaLLM-B
Diplomacy (T1590)	0.84	0.14	0.05	0.20	0.10	0.31
ConvAI3-1713	0.36	0.05	0.07	0.14	0.18	0.28
CIRCA (T565)	0.08	0.10	0.04	0.09	0.28	0.40
P.Chat Choose (T1730)	0.02	0.24	0.02	0.09	0.39	0.45
AirDial. Class. (T573)	0.92	0.05	0.07	0.08	0.27	0.36
Story Motiv. (T294)	0.34	0.09	0.07	0.09	0.49	0.25

Appendix

Table 9: Per-Task Final Performance on CITB-19 (Batched). Scores measured after all 19 tasks have been trained sequentially; all tasks use ROUGE-L (0–1) averaged over 3 seeds.

Figure 8: Prompt-template sensitivity of the Inference-Only baseline. Per-task final performance under the Alpaca template (blue, used in the main paper) and Gemma 3’s native chat template (red). (a) TRACE-8 (Batched): normalized scores (EDIM, SARI divided by 100); Alpaca dominates on every task (mean $0.31$ vs. $0.21$ ). (b) CITB-19 (Batched): ROUGE-L; the two templates are tied in aggregate (mean $0.34$ vs. $0.35$ ) but redistribute mass across generation- and classification-style tasks. Tasks are shown in original training/streaming order.

Panel of Figure 9 — Figure 9: Router Embedding Space (2D Projection). t-SNE projections of the router’s embeddings, coloured by task identity. (a) TRACE-8: 8 tasks spanning diverse domains. (b) CITB-19: 19 dialogue sub-tasks.

Panel of Figure 10 — Figure 10: Router Embedding Space with Prototypes (TRACE-8, t-SNE). Yellow stars are prototype positions; grey lines show nearest-prototype assignment. The transition from $P{=}8$ to $P{=}2$ maps directly onto the isolation $\to$ constructive clustering $\to$ destructive overload $\to$ collapse progression described in § 4.5.1 .

Figure 11: Full Expert-Assignment Heatmap (TRACE, 500 chunks, P=8). Each row is a task; each column a chunk; colour encodes the assigned expert ID. Dark blue (“null”) cells indicate chunks where that task was absent from the stream. The per-task colour patterns remain stable across the full 500-chunk stream, confirming persistent task–expert binding.

Figure 12: Router Assignment Matrices on TRACE (Online). Pool sizes $P\in\{4,6,8\}$ . Each cell shows the fraction of chunks from that task routed to each expert.

Continual Learning for LLMs

Continual learning for large language models studies how models absorb new knowledge and skills while retaining previously learned capabilities.

See all 231 papers

CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning
May 7, 2026Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz +2Continual Learning for LLMs Low-Rank Adaptation
Iowa State University · Maro Systems, USA · University of California, Berkeley +1
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
Jul 26, 2026Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli +2Continual Learning for LLMs Continual LoRA
College of Health Solutions, Arizona State University, Phoenix, Arizona, USA · Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA +1
Unifying Detection and Adaptation in Task-Free Continual Learning
Aug 27, 2026Dezheng Han, Anbang Zhang, Zhihao Zhu +1Task-Free Continual Learning Continual Learning
School of Control Science and Engineering, Shandong University, Jinan, China · Department of ECE, The Hong Kong University of Science and Technology, Hong Kong · National Medicine-Engineering Interdisciplinary Industry-Education Integration Innovation Platform, Shandong University, Jinan, China +1

May 7, 2026J/K move · Enter open · S save

CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz, Tanya Roosta+1

Iowa State University · Maro Systems, USA · University of California, Berkeley+1

Continual Learning for LLMs Low-Rank Adaptation Incremental Learning Catastrophic Forgetting

Open paper AskPDF arXiv

Large language models (LLMs) can learn new tasks through fine-tuning, but learning tasks sequentially often leads to catastrophic forgetting of earlier ones. This paper proposes CRAFT, a continual learning framework that keeps model weights frozen and instead learns low-rank interventions on hidden representations. Similar tasks share an intervention to help knowledge transfer, with similarity judged by how each task alters the model's output distribution rather than by a learned gating network. Within a shared intervention, a Kullback-Leibler (KL) penalty keeps each new task close to the intervention's existing behavior, which limits forgetting of earlier tasks, while a small, task-private intervention in the deepest layers preserves dedicated capacity that later tasks cannot overwrite. The learned interventions also make it possible to identify the task at test time, so task identity labels are not needed. Across multiple benchmarks and model scales, CRAFT improves overall performance and reduces forgetting compared with strong LoRA-based methods, while remaining robust to task ordering.

Abstract

Continual Learning for LLMs

Explore similar work

CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

Unifying Detection and Adaptation in Task-Free Continual Learning