Selm: Self-Evolving Library Management for Multi-Skill Agent Optimization
Authors: Mengxi Hu, Yuxuan Hu, Jinbo Su, Ke Wang, Honghao Shi, Wenli Wu, Cuiping Li, Hong Chen, Jing Zhang
Organizations: School of Information, Renmin University of China, Beijing, China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China · Beijing Academy of Artificial Intelligence, Beijing, China · Beijing Key Laboratory of Intelligent Database Systems and Applications, Beijing, China · Engineering Research Center of Database and Business Intelligence, Beijing, China
Large language model (LLM) agents increasingly use skills as a lightweight way to acquire task-specific knowledge. Recent work enables skills to evolve from task trajectories. However, these methods often lack systematic management of evolved skills, making it difficult for agents to reuse and accumulate skills across domains, thus limiting performance on subsequent tasks.To reliably accumulate, reuse, and maintain useful skills across tasks, we present SELM (Self-Evolving Library Management), a framework that formulates skill evolution as library-level management. SELM jointly manages skill retrieval, updating, and lifecycle by maintaining skill confidence and utility signals. It introduces utility-aware retrieval to select useful skills, a Map-Reduce framework to generate targeted updates from task-specific experience, and lifecycle controls to retain or remove skills based on their effectiveness. These mechanisms enable the skill library to continuously accumulate useful experience while preserving and pruning skills across tasks, improving agent performance on complex tasks across diverse domains. We evaluate SELM on four agent benchmarks covering diverse tasks and execution environments, and SELM consistently outperforms in most settings.
Figures & tables
Figure 1: Prior skill-evolution methods can overwrite useful procedures or keep weak skills retrievable, causing task failures when experience from different tasks conflicts. Selm instead uses utility-aware retrieval, category-wise Map–Reduce updates, and lifecycle validation to preserve useful skills and retire ineffective ones, enabling more reliable task completion.
Figure 2: Overview of Selm . The frozen agent retrieves skills by relevance and utility, executes tasks, and turns trajectories into category-wise Map–Reduce updates. Candidate skills are validated on a held-out set; accepted skills remain in the library, and low-utility skills are retired to a graveyard that blocks their names from returning.
Table 3
Figure 3: (a) Sensitivity to the EMA decay α on SpreadsheetBench (Claude-Opus-4.6, no harness); the marked default is α=0.4 . (b) SkillsBench soft accuracy (%) for each target model M under different optimizers O , including the no-skill baseline (no harness).
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Description
Default
Mmax
Maximum library capacity
30
wu
Utility weight in retrieval scoring
0.6
w
soft score weight in reward calculating
0.5
α
EMA decay factor for utility update
0.4
τsoft
Soft threshold for success classification
0.3
∣G∣max
Maximum graveyard capacity
50
Appendix
Table 4: Complete hyperparameter summary with default values.
Large language model (LLM) agents rely on reusable skills to solve complex tasks, but existing skill creation approaches often treat skills as isolated, static artifacts, limiting reusability, reliability, and long-term improvement. We propose MUSE-Autoskill Agent (Memory-Utilizing Skill Evolution), a skill-centric agent framework that creates, reuses, and refines skills under a unified lifecycle: creation, memory, management, evaluation, and refinement. MUSE creates skills on demand, stores them across tasks, retrieves them through a skill catalog, and accumulates per-skill experience for later reuse and adaptation. Across the main reported settings on SkillsBench and SkillLearnBench, MUSE-Autoskill outperforms Hermes, Codex, and Claude Code. On SkillsBench, its self-created skills surpass human-authored skills on the successfully covered subset (85.24% vs. 81.17%), showing that lifecycle-managed skills can distill agent experience into highly effective reusable assets; MUSE-created skills also transfer to Hermes more effectively than Codex- or Claude-created skills, reaching 51.90% accuracy under transfer. These results highlight the importance of treating skills as long-lived, experience-aware, and testable assets.
A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.
Yaorui Shi, Yuxin Chen, Zhengxi Lu +6
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2
Large language model (LLM) agents can continually improve without parameter updates by converting historical experience into reusable procedural knowledge. However, existing methods often consolidate experience based on semantic similarity or LLM judgments, which may merge superficially related but behaviorally incompatible strategies and thereby degrade performance. To address the issue, we propose SkillCommit, an online skill evolution framework that continuously transforms experience into a hierarchical library of reusable skills. Rather than generalizing from individual experiences immediately, SkillCommit first preserves each experience as an instance-specific patch, retaining its locally validated behavior. As patches accumulate, SkillCommit abstracts those sharing a common behavioral mechanism into higher-level skills. Specifically, for each incoming skill, embedding-based retrieval first identifies candidate related skills. Cross-instance replay and an LLM-based mechanism check determine whether these skills transfer across cases and share a common underlying mechanism. Candidates that pass both checks are abstracted into a higher-level skill and committed only if it preserves the validated behavior of all constituent skills. Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance across diverse domains. Moreover, the learned skills transfer across model scales and families, enabling cross-model experience transfer.
Yu He, Weikai Yang
Nanjing University · The Hong Kong University of Science and Technology (Guangzhou)