Learning What to Keep: Transfer-Aware Personal Skill Learning from User Feedback

Li Wang; Chen Zhang; Yirong Yang; Xingyu Chen; Jinkun Zhao; Tianhao Peng; Wenjun Wu

cs.CLOct 14, 2026

Learning What to Keep: Transfer-Aware Personal Skill Learning from User Feedback

Authors: Li Wang, Chen Zhang, Yirong Yang, Xingyu Chen, Jinkun Zhao, Tianhao Peng, Wenjun Wu

Organizations: Beihang University · Chengdu University of Information Technology · Nanyang Technological University · Hangzhou International Innovation Institute ·

Abstract

User feedback enables LLMs to refine reusable personalized skills, but often mixes persistent preferences with request-specific corrections. Indiscriminately retaining both risks negative transfer, motivating selective retention at the level of individual requirements. We frame this challenge as selective transfer over atomic rules and propose SkillGrain, a framework for learning reusable personalized skills from user feedback. SkillGrain decomposes feedback into independently actionable rules, predicts their transferability, and retains only those suitable for future requests. To learn these decisions, we combine rubric-guided skill rewards with fine-grained credit assignment that separately evaluates rule content and retention decisions. We further jointly optimize skill induction and skill-conditioned generation through reinforcement learning, ensuring that learned skills support downstream personalization. We construct a benchmark spanning five personalized writing tasks to evaluate cross-request generalization. Experiments across three base models in both in-domain and out-of-domain settings show that SkillGrain achieves the best overall average performance among competing methods.

Figures & tables

Figure 1: User feedback contains a mixture of transferable preferences and task-specific details. SkillGrain learns a user-specific skill from feedback to enable consistent personalized responses.

Figure 2: Overview of SkillGrain. We construct user-specific training instances consisting of an adaptation turn and subsequent requests. The model jointly optimizes skill-update and response trajectories using GRPO, with separate rubric-based rewards and independently normalized advantages. At inference time, the learned user skill is used to generate personalized responses.

Qwen2.5-3B-Instruct based
Methods	In-Domain			Out-of-Domain		Average
Methods	Topic Writing	Yelp	Email	Product Review	Abstract	Average
No Skill	73.61	53.23	68.71	75.88	80.47	70.38
No History	71.76	72.96	81.57	71.06	75.91	74.65
General Skill	71.09	63.52	74.36	78.11	81.99	73.82
Adaptive Skill	71.97	52.59	70.10	79.15	82.45	71.25

Table 1: Performance across five personalization tasks using three backbone models. The best and second-best results within each model block are shown in bold and underlined, respectively.

Qwen2.5-3B-Instruct based
Backbone / Method	Topic	Yelp	Email	Product	Abstract	Average
SkillGrain	76.90	85.11	87.77	78.29	81.75	81.96
w/ outcome-based reward	76.75	82.94	86.24	76.84	82.12	80.99
w/o atomization	77.57	79.89	87.11	73.56	76.46	78.93
w/o advantage allocation	77.12	84.90	88.25	77.76	81.45	81.90
w/o skill update	77.61	78.74	84.77	75.31	80.16	79.33

Table 2: Ablation results of SkillGrain. Best and second-best results under each backbone are shown in bold and underline , respectively.

Figure 3: Further analysis of SkillGrain. (Left three): Performance of trained methods across two rounds of feedback adaptation on Qwen2.5-3B-Instruct, Qwen3-1.7B, and Qwen3-4B, respectively. (Right): Performance of SkillGrain with different proportions of training data. Scores are Mean4 (%) averaged over three evaluation seeds.

Figure 4: Sensitivity analysis of

\lambda

Method	Qwen2.5 3B-Instruct	Qwen3 1.7B	Qwen3 4B
Outcome-RL	72.5	65.0	77.5
Rule-RL	80.0	72.5	77.5
SkillGrain	82.5	72.5	82.5

Table 3: Average rule adherence (%) across four alternative writing preferences.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Writing Scenario	Train	Test
Topic Writing	Generate a personalized post from a topic summary.	498	200
Email	Continue an email from a given prefix in the user’s style.	500	200
Yelp	Generate a personalized business review from metadata and a rating.	500	200
Product Review	Generate a personalized review from product information and a rating.	–	198
Abstract	Generate an author-style abstract from a title and keywords.	–	200
Total		1,498	998

Appendix

Table 4: Personalized writing tasks and curated split sizes.

Qwen2.5-3B-Instruct
Method	LEN	FP	EXPR	NL
Outcome-RL	71.29	86.95	68.24	92.94
Rule-RL	70.83	87.39	67.98	95.23
SkillGrain	74.62	88.05	68.85	96.35
Qwen3-1.7B
Outcome-RL	80.47	88.16	70.22	92.70

Appendix

Table 5: Dimension-wise rubric scores of the three RL methods.

	Extraction		Transferable
Backbone	ID	OOD	ID	OOD
Qwen2.5-3B-Instruct	89.31	87.04	84.27	79.60
Qwen3-1.7B	82.63	77.71	86.24	81.41
Qwen3-4B	94.54	90.30	89.81	81.80

Appendix

Table 6: Atomic extraction and transferable-atom selection precision (%). ID and OOD denote in-domain and out-of-domain test sets, respectively.

Method	Judge Score
Outcome-RL	73.44
Rule-RL	75.16
SkillGrain	76.44

Appendix

Table 7: Independent GPT-6 Luna evaluation with Qwen2.5-3B-Instruct on the same 200 test instances.

Method	Training Time (h)
SFT	2.0
STaR	2.0
Answer-Only RL	0.9
Outcome-RL	2.1
Rule-RL	1.7
SkillGrain (Ours)	1.8

Appendix

Table 8: Training time (hours) of methods requiring optimization with Qwen3-4B.

Task	Round 1	Round 2	Test
Abstract	0.905	1.000	1.000
Topic Writing	0.750	1.000	0.903
Email Completion	0.528	0.992	0.889

Appendix

Table 9: Mean four-dimensional scores of the saved case-study trajectories.

Personalized Language Models

Personalized language models adapt responses to individual users’ preferences, histories, writing styles, and changing needs.

See all 89 papers

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution
Mar 1, 2026Yutao Yang, Junsong Li, Qianjun Pan +9LLM Agent Skill Learning Lifelong Learning Agents
School of Computer Science and Technology, East China Normal University · Shanghai AI Laboratory
Beyond Task-Only Matching: Personalized Skill Routing with Counterfactual Evaluation
Aug 28, 2026Tianle Wang, Yanghe Zou, Xiang Liu +3LLM Agent Skill Retrieval LLM Agent Routing
College of Software Engineering, Southeast University · School of Computer Science and Engineering, Southeast University · Department of Computer Science and Engineering, The Chinese University of Hong Kong +1
SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution
Jul 29, 2026Zhiyuan Yao, Yuxin Chen, Zhengxi Lu +13Agentic RL Curriculum RL
1Zhejiang University · 2National University of Singapore · 3Shanghai Jiao Tong University +1

Mar 1, 2026J/K move · Enter open · S save

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Yutao Yang, Junsong Li, Qianjun Pan, Bihao Zhan+8

School of Computer Science and Technology, East China Normal University · Shanghai AI Laboratory

LLM Agent Skill Learning Lifelong Learning Agents Self-Evolving Agents Personal AI Agents LLM Agent Skill Retrieval

Open paper AskPDF arXiv

In practical LLM applications, users repeatedly express stable preferences and requirements, such as reducing hallucinations, following institutional writing conventions, or avoiding overly technical wording, yet such interaction experience is seldom consolidated into reusable knowledge. Consequently, LLM agents often fail to accumulate personalized capabilities across sessions. We present AutoSkill, an experience-driven lifelong learning framework that enables LLM agents to automatically derive, maintain, and reuse skills from dialogue and interaction traces. AutoSkill abstracts skills from user experience, supports their continual self-evolution, and dynamically injects relevant skills into future requests without retraining the underlying model. Designed as a model-agnostic plugin layer, it is compatible with existing LLMs and introduces a standardized skill representation for sharing and transfer across agents, users, and tasks. In this way, AutoSkill turns ephemeral interaction experience into explicit, reusable, and composable capabilities. This paper describes the motivation, architecture, skill lifecycle, and implementation of AutoSkill, and positions it with respect to prior work on memory, retrieval, personalization, and agentic systems. AutoSkill highlights a practical and scalable path toward lifelong personalized agents and personal digital surrogates.

Abstract

Personalized Language Models

Explore similar work

AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution

Beyond Task-Only Matching: Personalized Skill Routing with Counterfactual Evaluation

SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution