SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering
Organizations: King’s College London, London, United Kingdom · Queen’s University Belfast, Belfast, United Kingdom · Nanjing University, Nanjing, China · Johannes Gutenberg University Mainz, Mainz, Germany ·
Abstract
Agent skills are increasingly used to configure coding agents for software engineering (SE) tasks, yet current practice treats them as static, hand-crafted assets, or evolved on pass rate alone. This is insufficient: a skill can improve task success while substantially raising token cost, or introducing misleading guidance. We argue that SE agent skill bundles can be treated as multi-objective search objects and present SkillMOO, a framework that evolves skill bundles through LLM-proposed edits and NSGA-II Pareto selection on pass rate and inference cost. Evaluated across all 16 SkillsBench SE tasks, SkillMOO achieves the top pass rate rank on 11 of 12 non-zero-pass tasks while achieving cost reductions of up to 31.7% over static bundles, with pass rate gains up to 21 percentage points. Analysis of 38 skill edits shows that pruning and substitution dominate successful operations, offering actionable principles for skill bundle design. Thereby, the current practice of deploying skills without cost-aware validation leaves better skill configurations unexplored, motivating a new class of cost-aware, search-based skill engineering.
Figures & tables
| ID | Task name | Category | #Skills | #Tests | ID | Task name | Category | #Skills | #Tests |
|---|---|---|---|---|---|---|---|---|---|
| 1 | citation-check | Info. Retrieval | 1 | 40 | 9 | jax-computing-basics | Sci. Computing | 1 | 43 |
| 2 | data-to-d3 | Data Visualization | 1 | 40 | 10 | parallel-tfidf-search | Perf. Optimization | 3 | 40 |
| 3 | dialogue-parser | NLP/Parsing | 1 | 40 | 11 | python-scala-translation | Code Migration | 6 | 40 |
| 4 | enterprise-information-search | Info. Retrieval | 1 | 40 | 12 | react-performance-debugging | Perf. Optimization | 2 | 40 |
| 5 | fix-build-agentops | Build Repair | 8 | 40 | 13 | simpo-code-reproduction | ML Reproduction | 2 | 40 |
| 6 | fix-build-google-auto | Build Repair | 3 | 40 | 14 | spring-boot-jakarta-migration | Code Migration | 5 | 40 |
| ID | Opt. Cost | Opt. Runtime | HV | HV | HV | Cost/ HV% | ID | Opt. Cost | Opt. Runtime | HV | HV | HV | Cost/ HV% |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (USD) | (s) | ( ori_skill ) | ( SkillMOO ) | (%) | (USD) | (s) | ( ori_skill ) | ( SkillMOO ) | (%) | ||||
| 1 | 4.1539 | 9720 | 0.0016 | 0.0798 | 5031 | 0.0008 | 8 | 1.9664 | 832 | 0.0012 | 0.0044 | 263 | 0.0075 |
| 2 | 4.1437 | 4613 | 0.0015 | 0.0723 | 4580 | 0.0009 | 9 | 2.8053 | 1717 | 0.0016 | 0.0319 | 1903 | 0.0015 |
| 3 | 3.3556 | 6428 | 0.0017 | 0.0489 | 2828 | 0.0012 | 10 | 9.6070 | 5083 | 0.0053 | 0.0480 | 807 | 0.0119 |
| 4 | 4.2755 | 5191 | 0.0020 | 0.0612 | 2892 | 0.0015 | 11 | 1.8611 | 1545 | 0.0056 | 0.0296 | 430 | 0.0043 |
| 5 | 2.2676 | 2305 | 0.0081 | 0.1783 | 2110 | 0.0011 | 12 | 9.9575 | 6233 | 0.0055 | 0.1342 | 2322 | 0.0043 |
| Operation | Description | #Edits | Pass rate | Cost | Runtime |
| Remove Skills | Drop peripheral or misaligned skills from bundle; observed with pass/cost gain | 11 | 8/11 | 9/11 | 5/11 |
| Remove one skill indicated as redundant by evaluation or test evidence | 5 | 5/5 | 2/5 | 2/5 | |
| Replace Skills | Replace lower-priority skills with alternatives selected by the optimizer | 7 | 5/7 | 7/7 | 4/7 |
| Add Skills | Add skills inferred as missing from current bundle | 5 | 0/5 | 5/5 | 5/5 |
| Explicitly add object-oriented programming guidance | 3 | 0/3 | 3/3 | 3/3 | |
| Explicitly add functional programming style guidance | 1 | 0/1 | 1/1 | 1/1 |
Software Engineering Agents
Software Engineering Agents are AI systems that use code, tools, and feedback to perform software development tasks.
See all 323 papers