Two Birds with One Stone: Skill-Augmented Generation via Unified Retrieval and Utilization

Zihan Luo; Jiahui Kang; Wei Zhou; Qilin Fan; Mingyan Li; Chao Chen

cs.CLOct 28, 2026

Two Birds with One Stone: Skill-Augmented Generation via Unified Retrieval and Utilization

Authors: Zihan Luo, Jiahui Kang, Wei Zhou, Qilin Fan, Mingyan Li, Chao Chen

Organizations: Chongqing University, Chongqing, China · Huazhong University of Science and Technology, Wuhan, China

Abstract

Skills, as reusable external knowledge, have demonstrated substantial potential to enhance the ability of Large Language Models (LLMs) to solve complex tasks. However, existing skill-augmented generation methods typically rely on separate models for skill retrieval and skill utilization, resulting in disconnected optimization of these two processes. To address this gap, we propose Skill-Uni, a unified reinforcement learning framework that jointly optimizes skill retrieval and utilization. Specifically, we adopt a generative skill retrieval paradigm that enables a single model to both learn how to retrieve relevant skills and leverage them for task solving. To effectively train this unified framework, we introduce a two-stage training strategy consisting of warm-up instruction tuning and GRPO, allowing the model to progressively improve its skill retrieval and utilization capabilities. Extensive experiments on five skill-related benchmarks demonstrate the effectiveness of Skill-Uni, yielding average performance gains of 66.59% and 34.82% over the base Qwen3-4B and LLaMA3.1-8B models, respectively. Besides, Skill-Uni is compatible with various skill retrieval engines, demonstrating its flexibility and generalizability. Additional in-depth analyses also provide insights into the effectiveness of Skill-Uni.

Figures & tables

Figure 1. Retrieval and task accuracy gains relative to TF-IDF, measured as the accuracy differences between each retriever and TF-IDF in skill retrieval and downstream task completion, respectively. Although stronger retrievers achieve substantial improvements in retrieval accuracy, only part of these gains translate into downstream task performance.

Figure 2. Overview of the traditional skill-augmented generation and our paradigm.

Figure 3. The overall framework of Skill-Uni. (a) Given a query, Skill-Uni is first prompted to describe its desired skills, which will be further used for skill retrieval, and then generate the final response with both the query and the retrieved skills. The whole training process consists of a warm-up instruction tuning stage (b) and a reinforcement learning stage (c).

Lexical Matching Methods
Method	TheoremQA	LogicBench	CHAMP	MedCalc-Bench	BigCodeBench	Average
Base (Qwen3-4B)	49.66	78.95	51.11	29.55	32.46	44.84 $(\Delta_{\mathrm{base}})$
Oracle Skill	71.81	86.84	68.89	55.91	34.21	57.43 $(\uparrow\,28.08\%)$
BM25	67.11	67.11	71.11	31.36	30.26	46.85 $(\uparrow\,4.48\%)$
TF-IDF	65.10	67.11	48.89	35.91	31.14	46.73 $(\uparrow\,4.21\%)$
Dense Embedding Retrieval Methods

Table 1. Performance comparison on multiple datasets. The base model is Qwen3-4B. Task accuracy (%) is reported, and the best performance is bold . Improvements in the Average column are measured against the Base model.

Lexical Matching Methods
Method	TheoremQA	LogicBench	CHAMP	MedCalc-Bench	BigCodeBench	Average
Base (LLaMA3.1-8B)	31.59	53.42	21.52	26.82	29.21	32.51 $(\Delta_{\mathrm{base}})$
Oracle Skill	47.26	68.03	42.15	60.82	29.47	49.55 $(\uparrow\,52.41\%)$
BM25	39.76	57.24	23.77	35.91	28.95	37.12 $(\uparrow\,14.18\%)$
TF-IDF	35.74	49.61	24.66	38.45	29.39	35.57 $(\uparrow\,9.51\%)$
Dense Embedding Retrieval Methods

Table 2. Performance comparison on multiple datasets. The base model is LLaMA3.1-8B. Task accuracy (%) is reported, and the best performance is bold . Improvements in the Average column are measured against the Base model.

Model Variants						Accuracy
ID	mask	SFT	GRPO	<rec>	<answer>	Accuracy
1	✗	✓	✓	✓	✓	46.47
2	✓	✗	✓	✓	✓	51.89
3	✓	✓	✗	✓	✓	51.64
4	✓	✓	✓	✗	✓	59.57
5	✓	✓	✓	✓	✗	57.93

Table 3. Ablation study of Skill-Uni. Average task accuracy (%) on five datasets is reported. The base model is Qwen3-4B.

Figure 4. The stratified performance based on query correctness before and after applying a given method.

Panel of Figure 5 — Figure 5. The results of in-depth analysis on (a) the effect of generative skill retrieval and (b) the effect of warm start. The base model is Qwen3-4B.

Method	MBPP+	LiveCodeBench
Base (Qwen3-4B)	55.82	17.96
Skill-Uni	62.17	19.16

Table 4. Performance on OOD datasets

Panel of Figure 6 — Figure 6. The performance analysis of Skill-Uni applied with different retrieval methods. The base model is Qwen3-4B.

Figure 7. A case in TheoremQA where Skill-Uni retrieves the correct skill while BM25 fails.

Figure 8. A case in MedCalc-Bench where both BM25 and Skill-Uni retrieve the golden skills, while only Skill-Uni derives the correct solution.

Knowledge Augmentation for Language Models

Knowledge augmentation for language models adds external or newly learned information to improve factuality, reasoning, and domain performance.

See all 180 papers

Co-Evolving Skill Generation and Policy Optimization
Jun 7, 2026Zhiwei Zhang, Yudi Lin, Nikki Lijing Kuang +4LLM Agent Skill Learning RL for LLM Agents
The Pennsylvania State University · Nanyang Technological University · University of California, San Diego +2
Skill-R1: Agent Skill Evolution via Reinforcement Learning
May 10, 2026Yash Vishe, Rohan Surana, Xunyi Jiang +8Agent Skill Learning RL for LLM Agents
UC San Diego · Adobe Research
SkillFM: Generating Skills for LLM Agents via Latent Flow Matching
Sep 30, 2026Zuming Zhang, Jie He, Yizhe Zhang +1Flow Matching LLM Agent Skill Learning
Nanyang Technological University · University of Edinburgh · Meta

Jun 7, 2026J/K move · Enter open · S save

Co-Evolving Skill Generation and Policy Optimization

Zhiwei Zhang, Yudi Lin, Nikki Lijing Kuang, Linlin Wu+3

The Pennsylvania State University · Nanyang Technological University · University of California, San Diego+2

LLM Agent Skill Learning RL for LLM Agents

Open paper AskPDF arXiv

Skill-augmented reinforcement learning improves language agents by storing reusable procedural knowledge acquired from past experience. Existing methods typically use strong language models to analyze trajectories, generate skills, and update a retrievable skill bank during online training. However, they rarely assess whether a newly generated skill is useful before it is stored and reused. We find that this assumption is unreliable: even skills generated by proprietary frontier LLMs exhibit highly mixed utility, with many providing little benefit or even degrading performance. Once such skills enter the bank, their effects are difficult to identify, because subsequent rollout feedback is delayed and usually reflects the combined effect of multiple retrieved skills rather than the marginal contribution of any individual skill. We propose an online reinforcement learning framework for pre-storage skill validation. The framework estimates whether a candidate skill contributes useful information beyond the skills already retrieved for the current task. It uses the standard rollout budget to form two matched groups under the same task and retrieval context: base rollouts conditioned on the currently retrieved skills, and skill-augmented rollouts conditioned on the same skills plus one candidate skill induced from the base trajectories. The reward gap between these two groups estimates the candidate skill's context-dependent marginal utility, enabling the framework to promote useful skills while filtering ineffective or harmful ones without additional rollout overhead. The framework further uses this marginal-utility signal to train the policy itself as a skill generator, reducing reliance on repeated calls to proprietary models. The learned skill-generation likelihood serves as a context-dependent score for retrieval-time reranking and outdated-skill pruning as the policy evolves.

Abstract

Knowledge Augmentation for Language Models

Explore similar work

Co-Evolving Skill Generation and Policy Optimization

Skill-R1: Agent Skill Evolution via Reinforcement Learning

SkillFM: Generating Skills for LLM Agents via Latent Flow Matching