When Skills Remain But Access Fails: Factorized Skill Memory for Continual Skill Internalization

Haomiao Qiu; Yining Tang; Hongyi Wan; Changfeng Sun; Ziyue Qiao; Miao Zhang

cs.AIOct 11, 2026

When Skills Remain But Access Fails: Factorized Skill Memory for Continual Skill Internalization

Authors: Haomiao Qiu, Yining Tang, Hongyi Wan, Changfeng Sun, Ziyue Qiao, Miao Zhang

Organizations: Harbin Institute of Technology, Shenzhen · Great Bay University · Zhejiang University

Abstract

Skill internalization aims to convert externally specified procedural skills into persistent model capabilities, allowing language-model agents to execute them without retrieving or reinjecting skill descriptions at inference time. Yet when new skills are learned continually, whether previously internalized skills remain usable is still poorly understood. We study forgetting in continual skill internalization and uncover a clear hierarchical structure. Early in subsequent training, a model may lose the ability to invoke an old skill from the raw request, while supplying only the correct canonical skill name substantially restores the learned behavior; with further optimization, this conditional execution also degrades. We call these two stages skill-addressing forgetting and skill-execution forgetting. Guided by this finding, we introduce Factorized Skill Memory (FSM), which separates continual skill memory into a plastic Address Memory and a stable Execution Memory. At each stage, FSM trains a new LoRA executor using only current-stage data, freezes it after acquisition, and appends it to Execution Memory; in parallel, Address Memory incrementally learns to map current requests to executors without replaying historical requests or execution trajectories. At inference time, Address Memory selects one executor from the raw request, and only that frozen executor performs a single language-model generation. On a five-stage benchmark built from 100 TaskBench skills, FSM achieves 65.95% mean exact match and 93.38% addressing accuracy across two task orders, improving over the strongest replay-free parameter-isolation baseline by 14.3 percentage points while remaining within 1.9 points of Replay, which retains historical training examples.

Figures & tables

Figure 1: Continual skill internalization and hierarchical forgetting. (a) Sequential learning can erase old-task closed-book performance within a few optimization steps. (b) A matched request-level example exposes two stages of forgetting. During addressing forgetting, the correct Skill name recovers execution that the raw request no longer reaches. Continued optimization can later damage execution under the correct skill address itself.

Figure 2: Hierarchical forgetting during continual skill internalization. Top: during the T2

\rightarrow

T3 transition, closed-book Task2 EM can collapse while Task2 remains recoverable when the correct skill address is supplied; with continued optimization, execution under the correct skill address can also erode. The frozen-executor diagnostic preserves the old task while learning Task3. Bottom: five-stage evaluation matrices for four continual-learning conditions. Off-diagonal cells report closed-book / + Correct Skill Address EM. Across both rows, pale green denotes retained performance, pale yellow denotes addressing forgetting, and pale purple denotes execution forgetting. Diagonal cells show task-acquisition EM. The FSM column is our method.

Figure 3: Factorized Skill Memory (FSM). Left: memory write. At stage $t$ , current requests update Address Memory while a fresh executor $E_{t}=\mathcal{B}\oplus\Delta_{t}$ is trained from the frozen base and a stage-specific LoRA. The completed executor is frozen and appended to Execution Memory as a new skill-family shard. Right: memory read. Address maps the raw request to $p(j\mid x)$ , selects $j^{\star}$ , and reads one frozen execution shard. Only $E_{j^{\star}}$ performs language-model generation.

Method	Tool Choice	Arg. F1	EM						BWT
Method	Tool Choice	Arg. F1	T1	T2	T3	T4	T5	Avg.	BWT
Base + Skill	49.57	52.24	19.91	22.79	50.30	29.77	47.70	34.09	–
Base + Oracle Address+Fields	90.71	81.01	23.84	70.10	59.94	69.23	56.89	56.00	–
Order 1: $T1\rightarrow T2\rightarrow T3\rightarrow T4\rightarrow T5$
Replay (reference)	85.90	83.75	43.75	77.21	70.48	80.27	74.91	69.32	-1.76
Progressive-LoRA	65.54	65.89	31.13	66.67	32.83	76.25	53.00	51.98	-8.38

Table 2: Main comparison across both task orders. Metrics are on a 0–100 scale; Order 2 values are placed by task identity. Tool Choice is ordered tool-list exact match, Arg. F1 compares argument-value tuples, Avg. is task-macro EM, and BWT is final minus post-acquisition EM over the first four arrival stages. Indented w/ correct skill address rows report percentage-point changes from the closed-book row above. A dash denotes an unreported metric.

Method	Tool Choice	Arg. F1	EM						BWT
Method	Tool Choice	Arg. F1	T1	T2	T3	T4	T5	Avg.	BWT
Replay (reference)	86.58	84.05	43.87	80.15	71.08	79.26	74.20	69.71	-1.39
Progressive-LoRA	56.78	60.03	18.29	62.99	29.82	73.58	49.82	46.90	-7.84
LwF ( Li and Hoiem, 2018 )	18.80	17.53	0.00	0.00	0.00	0.00	73.85	14.77	-70.31
w/ correct skill address	+17.73	+15.21	+0.00	+0.00	+0.00	+70.23	-0.35	+13.98	–
EWC ( Kirkpatrick et al., 2017 )	8.20	9.03	0.00	0.00	0.00	0.00	38.52	7.70	-37.64

Table 3: Qwen3-4B Order-1 results. Tool Choice and Arg. F1 are percentages; T1–T5 report final closed-book graph exact match, Avg. is the task-macro mean, and BWT is mean final-minus-acquisition EM over T1–T4. Correct-skill-address rows are percentage-point deltas from the row above.

Order	Task	Oracle	Self-route	EPI	AdapterCL	FSM
Order 1	T1	45.02	42.01	35.18	41.25	44.21
	T2	83.58	58.82	59.41	64.19	78.19
	T3	75.60	56.02	51.09	62.89	59.64
	T4	80.27	77.26	63.94	70.74	78.93
	T5	76.33	72.44	54.84	64.24	70.67
	Macro	72.16	61.31	52.90	60.66	66.33

Table 4: Per-task Graph EM for selector variants across both task orders. EPI and AdapterCL use the same frozen execution banks as FSM. Macro is the task average.

Figure 4: Two-order mean routing accuracy by task for Self-route, EPI, AdapterCL, and FSM. The dashed horizontal line marks Oracle routing at 100%. Macro is the task average.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Family	Skills	Train	Held-out
T1	Daily Life	40	1,000	864
T2	Language/Speech	11	1,000	408
T3	Retrieval/Understanding	21	1,000	332
T4	Vision/Document	9	1,000	299
T5	Generation/Editing	19	1,000	283
Total	Five disjoint families	100	5,000	2,186

Appendix

Table A1: Benchmark inventory. Train and held-out counts are request–graph pairs; the two orders reuse the same held-out sets.

Reproducibility item	Frozen definition in the current candidate
Executor checkpoint	Rank-16 LoRA on $q/k/v/o$ , learning rate $4\times 10^{-4}$ , six epochs, epoch 6 exported.
Execution-memory storage	Shared 8.2B backbone plus 15.34M LoRA parameters per shard; five shards add 76.68M parameters ( $0.94\%$ of the backbone).
Sequential Address	Request text only; fixed-dimensional hashing features plus incremental Naive Bayes; update on current stage only.
Retained Address state	Compact class/feature statistics; no old requests or workflow targets are replayed.
Inference	Hard request-level routing to one frozen executor; one language-model generation per request.
Reference selector	Offline TF–IDF + logistic regression uses all historical requests.

Appendix

Table A2: Protocol choices that define the deployed FSM method and its reference variants.

Task	Requests	Full-task EM	Skill selection
T1	864	7.87	83.22
T2	408	0.00	92.65
T3	332	0.00	87.05
T4	299	0.00	93.65
T5	283	0.00	94.35
Macro	–	1.57	–

Appendix

Table A3: Frozen Qwen3-8B with the correct skill address only (percent).

Order/task	$O=C$	Self-route $R$	All wrong	$C-R$
O1/T1	45.02	42.01	54.98	3.01
O1/T2	83.58	58.82	16.42	24.75
O1/T3	75.60	56.02	24.40	19.58
O1/T4	80.27	77.26	19.73	3.01
O1/T5	76.33	72.44	23.67	3.89
O2/T1	45.49	42.82	54.51	2.66

Appendix

Table A4: Current self-routing coverage and selection decomposition (percent). The table applies only to self-routing; learned Address did not retain per-request candidate indices.

Question	Observed evidence	What it measures
Is execution retained?	Oracle Graph EM is 71.76% across two orders.	Conditional execution under perfect shard access.
Is access unreliable?	Self-route is 61.43% while candidate coverage is 71.76%.	Selection loss is measured only for self-route.
Can request text recover access continually?	FSM reaches 65.95% and 93.38% route accuracy.	Address itself remains sequence-sensitive.
What is available with historical requests?	Offline Address reaches 69.15%.	Reference selector with historical-request access.

Appendix

Table A5: Measurements supporting the Address–Execution decomposition.

Observed pattern	Primary diagnosis	Next measurement
Low oracle and low routing	Conditional execution is weak.	Revisit stage training, capacity, or data before tuning Address.
High oracle and low self-route	Usable skills exist but are selected unreliably.	Compare request-only routing rules.
Sequential below offline	Historical-request reference is stronger, but the cause is confounded by selector design.	Match representation and classifier under online/offline updates.
Offline close to oracle	A stronger historical-request selector leaves a smaller access gap.	Separate update regime from representation/classifier choice.

Appendix

Table A6: Error decomposition induced by Oracle, Self-route, and learned Address.

Address/control	Route acc.	EM	Evaluation scope
FSM	93.38	65.95	replay-free, two-order mean
Lexical TF–IDF (offline)	97.03	69.15	historical requests retained
E5-base-v2	94.22	66.61	final test, two-order mean
Qwen3-4B mean-pool	90.92	64.20	final test, two-order mean
Template-group split	97.09	–	997 held-out train rows
Ordinary five-fold split	97.26	–	training requests

Appendix

Table A7: Representation and routing controls (percent). Training-request classification and final-test route accuracy are distinct quantities.

Task	4B best	8B best	8B batch-32	Selected recipe
T1 EM	47.5	44.5	32.5	$4\times 10^{-4}$ , batch 8
T5 EM	77.5	76.5	71.0	$4\times 10^{-4}$ , batch 8

Appendix

Table A8: Controlled T1/T5 validation screen (percent). The screen used 800/200 training/validation rows per task and epochs 1–3.

Order prefix	Oracle	FSM	Route accuracy	Available tasks
O1/ $n=1$	45.02	45.02	100.00	T1
O1/ $n=2$	64.30	63.77	98.88	T1–T2
O1/ $n=3$	68.07	63.59	94.30	T1–T3
O1/ $n=4$	71.12	66.70	94.67	T1–T4
O1/ $n=5$	72.16	66.33	93.38	T1–T5
O2/ $n=1$	73.80	73.80	100.00	T3

Appendix

Table A9: Sequential Address prefix results (percent). Oracle is the conditional reference at each prefix; route accuracy is the replay-free Address route audit.

Order	Prefix	T1	T2	T3	T4	T5
O1	$n=1$	45.02	–	–	–	–
	$n=2$	42.82	82.60	–	–	–
	$n=3$	42.36	72.06	61.45	–	–
	$n=4$	42.01	64.71	60.54	77.93	–
	$n=5$	42.01	58.82	56.02	77.26	72.44
O2	$n=1$	–	–	73.80	–	–

Appendix

Table A10: Current self-confidence Graph EM by order prefix (percent). Dashes mark task families that had not entered the stream.

Branch	Evidence level	Outcome and disposition
Current-only SFT, GRPO/HC-GRPO, replay, LwF, EWC, OPSD, and adapter variants	Complete five-task endpoints	Most direct updates overwrite old tasks; representative endpoints are reported in the main results and the analyses below.
SKILL0 TaskBench curriculum	Complete lower triangles	Current-task acquisition succeeds, but request-only old-task Graph EM collapses; exact names temporarily reactivate intermediate checkpoints.
One-epoch SFT warm start before GRPO/SKILL0	Completed audited cells	Warm start enables current-task learning but does not prevent cross-task overwrite.
Pure on-policy SKILL0 pilot	Completed T1 pilot	Protocol assistance creates a weak learning signal, but strict closed-book Graph EM remains zero.
No-LoRA skill-information controls	Complete 30-cell grid	Canonical skill metadata exposes latent capability but changes the inference information boundary.
T2 $\rightarrow$ T3 learning-rate sweep	Complete 8B checkpoint grid	Lower learning rates delay the switch but do not yield a useful retention–plasticity operating point.

Appendix

Table A11: Training branches and the evidence they provide for the final design.

Recipe	S1/T1	S2/T2	S3/T3	S4/T4	S5/T5	Old tasks after S5	Prompt
Current-only SFT (8B)	45.02	83.58	75.90	80.60	76.33	all 0	request only
GRPO (8B)	40.86	79.90	73.19	79.93	71.73	all 0	request only
SKILL0	46.06	83.33	77.11	82.27	79.15	all 0	request only
SKILL0	46.88	83.09	76.51	80.60	79.86	all 0	full skill card

Appendix

Table A12: Closed-book or supplied-skill Graph EM (percent) on the acquired task at each stage. The final old-task statement is evaluated after S5. The 4B SKILL0 rows are not capacity-matched to the 8B baselines.

Method	First request-only zero	Skill address at step 126
SFT-8B	8	54.66
GRPO-8B	6	37.01
SKILL0-4B	18	82.84

Appendix

Table A13: Extended T2 retention trajectories (Graph EM, percent). Supplying the correct skill address continues to recover behavior long after autonomous closed-book access reaches zero.

Learning rate	Best observed step	T2 Graph EM	T3 Graph EM	T2–T3 mean	Outcome
$2\times 10^{-5}$	100	0.00	74.40	37.20	T3 learned; T2 erased
$1\times 10^{-5}$	125	0.00	75.60	37.80	T3 learned; T2 erased
$5\times 10^{-6}$	50	0.00	76.81	38.40	best T3; T2 erased
$2\times 10^{-6}$	125	0.00	73.19	36.60	T3 learned; T2 erased
$1\times 10^{-6}$	25	79.17	0.00	39.58	T2 retained before switch
$5\times 10^{-7}$	50	78.43	0.00	39.22	T2 retained before switch

Appendix

Table A14: Best two-task mean observed in each learning-rate arm (percent). For the four higher rates, every checkpoint with nonzero T3 has zero T2. For the two lowest rates, T3 remains zero for the full epoch.

LR	Step	T2 Graph EM	T3 Graph EM	Mean	T2 node F1	T3 node F1
$1\times 10^{-6}$	25	79.17	0.00	39.58	94.45	0.00
	30	79.41	0.00	39.71	95.45	0.00
	35	0.00	18.98	9.49	0.00	28.22
	40	0.00	39.76	19.88	0.00	56.35
	45	0.00	51.20	25.60	0.00	70.15
	50	0.00	57.23	28.61	0.00	77.34

Appendix

Table A15: Five-step zoom around the learning-rate transition (percent). The two overlapping checkpoints occur before T3 is meaningfully acquired.

Continual Learning for LLM Agents

Continual learning for LLM agents studies how agents adapt from experience while retaining skills and knowledge across tasks.

See all 199 papers

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
Apr 2, 2026Zhengxi Lu, Zhiyuan Yao, Jinyang Wu +7LLM Agent Skill Learning RL for Tool Use
Zhejiang University · Meituan · Tsinghua University
SkillSpec: Consensus-Gated Agent Skill Evolution via Representation Specialization
Sep 30, 2026Huancheng Chen, Xiaodi Sun, Zhaoqiong Huang +3LLM Agent Self-Improvement LLM Agent Skill Learning
Microsoft AI
ERSkill: Evolving for Skill-Guided Adaptive Memory Retrieval
Aug 13, 2026Haolong Chen, Liang Zhang, Zhuo Li +2LLM Agent Memory LLM Agent Skill Learning
Shenzhen International Center for Industrial and Applied Mathematics · Shenzhen Research Institute of Big Data · The Chinese University of Hong Kong, Shenzhen +2

Apr 2, 2026J/K move · Enter open · S save

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han+6

Zhejiang University · Meituan · Tsinghua University

LLM Agent Skill Learning RL for Tool Use In-Context RL Curriculum RL

Open paper AskPDF arXiv

Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file's on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7% for ALFWorld, +6.6% for Search-QA, and+10.1% for WebShop), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero.

Abstract

Continual Learning for LLM Agents

Explore similar work

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

SkillSpec: Consensus-Gated Agent Skill Evolution via Representation Specialization

ERSkill: Evolving for Skill-Guided Adaptive Memory Retrieval