SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
Organizations: DreamX Team, Alibaba Group
Abstract
Large language model (LLM) agents such as OpenClaw rely on reusable skills to perform complex tasks, yet these skills remain largely static after deployment. As a result, similar workflows, tool usage patterns, and failure modes are repeatedly rediscovered across users, preventing the system from improving with experience. While interactions from different users provide complementary signals about when a skill works or fails, existing systems lack a mechanism to convert such heterogeneous experiences into reliable skill updates. To address these issues, we present SkillClaw, a framework for collective skill evolution in multi-user agent ecosystems, which treats cross-user and over-time interactions as the primary signal for improving skills. SkillClaw continuously aggregates trajectories generated during use and processes them with an autonomous evolver, which identifies recurring behavioral patterns and translates them into updates to the skill set by refining existing skills or extending them with new capabilities. The resulting skills are maintained in a shared repository and synchronized across users, allowing improvements discovered in one context to propagate system-wide while requiring no additional effort from users. By integrating multi-user experience into ongoing skill updates, SkillClaw enables cross-user knowledge transfer and cumulative capability improvement, and experiments on WildClawBench show that limited interaction and feedback, it significantly improves the performance of Qwen3-Max in real-world agent scenarios.
Figures & tables
| Category | Example Tasks | Challenges |
| Productivity Flow | arXiv classification, scheduling, SCP | multi-step pipelines |
| Code Intelligence | debugging, puzzle solving | execution correctness |
| Social Interaction | negotiation, chat analysis | multi-turn reasoning |
| Search & Retrieval | academic search, conflict resolution | API usage |
| Creative Synthesis | video notes, poster generation | multimodal generation |
| Safety & Alignment | prompt injection, leakage detection | constraint satisfaction |
| Property | Description |
| Execution Environment | Full Linux container with tools |
| Multimodality | Text, code, image, video |
| Evaluation | 3–27 metrics aggregated |
| Hard Constraints | Critical errors zero score |
| Task Length | 15–50 steps |
| External Dependency | APIs and model downloads |
| Category | Day 1 | Day 2 | Day 3 | Day 4 | Day 5 | Day 6 | Abs. Gain | Rel. Gain |
| Social Interaction | 54.01% | 60.34% | 60.34% | 60.34% | 60.34% | 60.34% | +6.33 | +11.72% |
| Search & Retrieval | 22.73% | 30.00% | 30.00% | 34.55% | 34.55% | 34.55% | +11.82 | +52.00% |
| Creative Synthesis | 11.57% | 21.80% | 21.80% | 21.80% | 21.80% | 21.80% | +10.23 | +88.41% |
| Safety & Alignment | 24.00% | 24.00% | 24.00% | 24.00% | 32.00% | 32.00% | +8.00 | +33.33% |
| Day | Candidate Skill | Skill Function | Change Summary | Validator | Next-Day Action |
| 1 | 03_task6 | Cross-dept Slack summarization, data reconciliation, risk identification, board-level brief drafting | Rewrote workflow into strictly-ordered steps; strengthened project keyword filtering, finance priority, change detection, COO contact confirmation | Accept | Day 2: upgrade to new best pool |
| 2 | (none) | Continued using current Social best pool | Same-pool retest; no new skill text landed | Reject | Day 3: keep Day 2 best pool |
| 3 | 03_task1 | Gmail + Calendar meeting coordination | Extended workflow with meeting-param extraction, multi-participant availability check, confirmation loop, reschedule on rejection | Reject | Not admitted; Day 4 keeps current best pool |
| 4 | (none) | Continued using current Social best pool | Same-pool retest; no new skill text landed | Reject | Day 5: keep current best pool |
| 5 | (none) | Continued using current Social best pool | Same-pool retest; no new skill text landed | Reject | Day 6: keep current best pool |
| 6 | 03_task3 | Slack feasibility analysis | Added fallback & grounding constraints; analysis must rely on real API results or user-provided context | Reject | Not admitted to next cycle |
| Day | Candidate Skill | Skill Function | Change Summary | Validator | Next-Day Action |
| 1 | validate-file- existence | Pre-processing file existence check | Before any file parsing / image reading / multimodal call, first confirm the input file actually exists | Accept | Day 2: upgrade to new best pool |
| 2 | debug-missing- file-path | Missing-file path debugging | List parent directory, verify naming, correct path instead of halting on “missing” | Reject | Day 3: keep Day 2 best pool |
| 3 | (none) | Continued using current Search best pool | Same-pool retest; nightly readout was stronger, confirming current pool as best-so-far | Accept | Day 4: continue same best pool |
| 4 | robust-file-validation- before-multimodal | Stronger multimodal pre-validation | Upgraded from “exists?” to “exists + parent-dir search + hard pre-multimodal validation” | Reject | Day 5: keep current best pool |
| 5 | constrained-technical- search-planning | Budget-constrained technical / academic search planning | Added feasibility check, sub-question decomposition, official-source priority, evidence-chain output | Reject | Day 6: keep current best pool |
| 6 | recover-missing- input-file | Recover / locate real input file from workspace | When benchmark’s expected path fails, proactively search the working directory for the actual input file | Reject | Not admitted to next cycle |
| Day | Candidate Skill | Skill Function | Change Summary | Validator | Next-Day Action |
| 1 | validate-tmp- workspace-inputs | Check /tmp_workspace inputs & environment setup | Before creative tasks, verify /tmp_workspace inputs, directories, and symlinks are correct | Accept | Day 2: upgrade to new best pool |
| 2 | multimodal-input- validation-and-setup | Multimodal input validation & output env init | Check video / image / PDF / audio files exist, are readable, and format-correct; prepare output directories | Reject | Day 3: keep current best pool |
| 3 | multimodal-creative- task-pipeline | Multimodal creative pipeline | New unified pipeline: extract content from PDF / video / image and generate posters, webpages, slides, etc. | Reject | Day 4: keep current best pool |
| 4 | multimodal-creative- task-pipeline (impr.) | Multimodal creative pipeline | Added image classification, visual generation, garment synthesis, structured output validation | Reject | Day 5: keep current best pool |
| 5 | multimodal-creative- task-pipeline (impr.); validate-required- input-files | Creative pipeline + per-file fail-fast validation | Pipeline added audio/video fallback & halt on missing input; new skill forces per-file validation for all named inputs | Reject | Day 6: keep current best pool |
| 6 | multimodal-creative- task-pipeline (cand.) | Multimodal creative pipeline | Extended PDF-to-poster / document-to-visual paths; did not yield better deployment results | Reject | Not admitted to next cycle |
| Day | Candidate Skill | Skill Function | Change Summary | Validator | Next-Day Action |
| 1 | git-push-with- auth-fallback | Patch / bundle fallback on git push failure | In no-credential / auth-failure scenarios, provide safe fallback instead of blocking on push | Accept | Day 2: add to Safety best pool |
| 2 | git-push-with- auth-fallback | Git auth-failure fallback | Unified patch / bundle filenames and verification; reduced filename inconsistency during fallback | Accept | Day 3: keep updated best pool |
| 3 | git-push-with- auth-fallback ; git-clone-to- directory | Push fallback + correct clone-to-dir | Push: added auth-alternative paths & secrets audit; Clone: fixed mkdir && cd && git clone subshell pitfalls | Accept | Day 4: keep current best pool |
| 4 | (none) | Continued using current Safety best pool | Same-pool retest; validator read a higher result, confirming current pool as best-so-far | Accept | Day 5: continue same best pool |
| 5 | git-push-with- auth-fallback | Git auth-failure fallback | Added “push hang treated as auth failure” and other non-interactive environment details; no improvement | Reject | Day 6: keep current best pool |
| 6 | git-push-with- auth-fallback | Git auth-failure fallback | Added identity config & filename consistency requirements; did not exceed current best validation result | Reject | Not admitted to next cycle |
| Query | Baseline (%) | Post-Evolve (%) | Gain |
| basic extraction | 21.7% | 69.6% | +47.8% |
| deadline parsing | 41.1% | 48.0% | +6.9% |
| save report | 28.3% | 100.0% | +71.7% |
| Average | 30.4% | 72.5% | +42.1% |
LLM Agent Skill Learning
Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.
See all 379 papers