cs.IRApr 9, 2026

SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support

Authors: Xingyan Liu, Xiyue Luo, Linyu Li, Ganghong Huang, Jianfeng Liu, Honglin Qiao

Organizations: Alibaba Cloud Computing, Alibaba Group

Abstract

Deploying LLM-powered agents in enterprise scenarios such as cloud technical support demands high-quality, domain-specific skills. However, existing skill creators lack domain grounding, producing skills poorly aligned with real-world task requirements. Moreover, once deployed, there is no systematic mechanism to trace execution failures back to skill deficiencies and drive targeted refinements, leaving skill quality stagnant despite accumulating operational evidence. We introduce SkillForge, a self-evolving framework that closes an end-to-end creation-evaluation-refinement loop. To produce well-aligned initial skills, a Domain-Contextualized Skill Creator grounds skill synthesis in knowledge bases and historical support tickets. To enable continuous self-optimization, a three-stage pipeline -- Failure Analyzer, Skill Diagnostician, and Skill Optimizer -- automatically diagnoses execution failures in batch, pinpoints the underlying skill deficiencies, and rewrites the skill to eliminate them. This cycle runs iteratively, allowing skills to self-improve with every round of deployment feedback. Evaluated on five real-world cloud support scenarios spanning 1,883 tickets and 3,737 tasks, experiments show that: (1) the Domain-Contextualized Skill Creator produces substantially better initial skills than the generic skill creator, as measured by consistency with expert-authored reference responses from historical tickets; and (2) the self-evolution loop progressively improves skill quality from diverse starting points (including expert-authored, domain-created, and generic skills) across successive rounds, demonstrating that automated evolution can surpass manually curated expert knowledge.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revision

    May 31, 2026Yuxuan Liu, Zhaochen Su, Lingyun Xie +11LLM Agent Self-ImprovementLLM Agent Skill Learning

  2. SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing

    Jun 12, 2026Haowen Gao, Haoran Chen, Can Wang +5Algorithmic AuditingLLM Agent Self-Improvement

  3. CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification

    Apr 2, 2026Hanrong Zhang, Shicheng Fan, Henry Peng Zou +11LLM Agent Skill LearningSelf-Evolving Agents