LLM Agent Skill Retrieval

LLM: Large Language Model

Agent skill retrieval studies how an agent can find and use procedural guidance—such as coding workflows or web-task instructions—from a reusable library. Methods include lexical and dense search with reranking, structured or graph-based organization, query decomposition, and context-budgeted selection and adaptation. Evaluations use interactive environments such as ALFWorld and WebShop alongside coding and web benchmarks, measuring both retrieval quality and whether retrieved skills improve task completion. A central challenge is that relevant-looking skills can be mismatched, conflicting, or ignored, so larger libraries require better tests of actual usefulness, safe selection, and performance across models and agent harnesses.

Latest papers

102 papers16 papers in the last four weeks, up 45% on the four weeks before. 0.1% of all new papers.

Skill Retrieval Augmentation for Agentic AI
Apr 27, 2026Weihang Su, Jianming Long, Qingyao Ai +6AI Agent Benchmarks LLM Agent Skill Retrieval
Department of Computer Science and Technology, Tsinghua University · ByteDance Inc.
Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks
Apr 22, 2026Xiyang Wu, Zongxia Li, Guangyao Shi +5Continual Learning for LLM Agents Game-Playing Agents
University of Maryland · University of Southern California · Good Start Labs +2
Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills
Apr 7, 2026Dawei Liu, Zongxia Li, Hongyang Du +4Tool-Augmented Language Model Agents Graph-Based Retrieval
University of Pennsylvania · University of Maryland · Brown University +2
How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings
Apr 6, 2026Yujian Liu, Jiabao Ji, Li An +3LLM Agent Skill Retrieval LLM Agents
UC Santa Barbara · MIT CSAIL · MIT-IBM Watson AI Lab
SkillRouter: Skill Routing for LLM Agents at Scale
Mar 23, 2026YanZhao Zheng, ZhenTao Zhang, Chao Ma +8LLM Agent Skill Retrieval LLM Agent Routing
Alibaba Group, Hangzhou, China
Memento-Skills: Let Agents Design Agents
Mar 19, 2026Huichi Zhou, Siyuan Guo, Anjie Liu +14Agentic AI LLM Agent Skill Learning
Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale
Mar 2, 2026Hao Li, Chunjiang Mu, Jianhao Chen +5LLM Agent Orchestration LLM Agent Skill Retrieval
Shanghai Artificial Intelligence Laboratory
TARSE: Test-Time Adaptation via Retrieval of Skills and Experience for Reasoning Agents
Mar 1, 2026Junda Wang, Zonghai Tao, Hansi Zeng +3Clinical QA LLM Agent Skill Retrieval
University of Massachusetts Amherst, MA, USA · University of Massachusetts Lowell, MA, USA
AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution
Mar 1, 2026Yutao Yang, Junsong Li, Qianjun Pan +9LLM Agent Skill Learning Lifelong Learning Agents
School of Computer Science and Technology, East China Normal University · Shanghai AI Laboratory
SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
Feb 28, 2026Shiqi Chen, Jingze Gai, Ruochen Zhou +13Tool-Using Agents Tool Use
University of Oxford · City University of Hong Kong · Hong Kong University of Science and Technology +3
SkillNet: Create, Evaluate, and Connect AI Skills
Feb 26, 2026Yuan Liang, Ruobin Zhong, Haoming Xu +47LLM Agent Skill Evaluation Agent Skill Learning
Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments
Feb 18, 2026Yangjie Xu, Lujun Li, Lama Sleem +7Small Language Models LLM Agent Skill Retrieval
University of Luxembourg · Foyer S.A. · Princeton University +1

Apr 27, 2026J/K move · Enter open · S save

Skill Retrieval Augmentation for Agentic AI

Weihang Su, Jianming Long, Qingyao Ai, Qiaozhi He+5

Department of Computer Science and Technology, Tsinghua University · ByteDance Inc.

AI Agent Benchmarks LLM Agent Skill Retrieval

Open paper AskPDF arXiv

As large language models (LLMs) evolve into agentic problem solvers, they increasingly rely on external, reusable skills to handle tasks beyond their native parametric capabilities. In existing agent systems, the dominant strategy for incorporating skills is to explicitly enumerate available skills within the context window. However, this strategy fails to scale: as skill corpora expand, context budgets are consumed rapidly, and the agent becomes markedly less accurate in identifying the right skill. To this end, this paper formulates Skill Retrieval Augmentation (SRA), a new paradigm in which agents dynamically retrieve, incorporate, and apply relevant skills from large external skill corpora on demand. To make this problem measurable, we construct a large-scale skill corpus and introduce SRA-Bench, the first benchmark for decomposed evaluation of the full SRA pipeline, covering skill retrieval, skill incorporation, and end-task execution. SRA-Bench contains 5,400 capability-intensive test instances and 636 manually constructed gold skills, which are mixed with web-collected distractor skills to form a large-scale corpus of 26,262 skills. Extensive experiments show that retrieval-based skill augmentation can substantially improve agent performance, validating the promise of the paradigm. At the same time, we uncover a fundamental gap in skill incorporation: current LLM agents tend to load skills at similar rates, regardless of whether a gold skill is retrieved or whether the task actually requires external capabilities. This shows that the bottleneck in skill augmentation lies not only in retrieval but also in the base model's ability to determine which skill to load and when external loading is actually needed. These findings position SRA as a distinct research problem and establish a foundation for the scalable augmentation of capabilities in future agent systems.