Multi-agent systems (MAS) for structured data-science tasks externalize analytical control through workflows spanning stages, tools, shared state, verification, and repair. Distilling such workflows into a single-agent skill can reduce orchestration overhead, but it remains unclear which workflow components should cross the control boundary. We distinguish capability resources, which expand what an agent can do, from pipeline guidance, which constrains which solutions it explores. On the same causal-estimation instances, adding task-qualified source pipeline guidance to a capability-matched skill changes normalized utility by +19.6 points under method-selection accuracy but -10.3 points under numerical error. To explain this reversal, we introduce Behavior-Outcome Freedom (F), a pre-synthesis diagnostic of signed behavior-outcome rank mismatch, and formalize its candidate-conditional role through Signed Anchor-Rank Transfer. Motivated by this mechanism, we propose AdaSkill, which preserves validated capability resources, removes runtime orchestration, and conditionally inherits pipeline guidance using a calibrated rule over F. Across 16 capability-matched interventions, the native-scale Full-minus-Discard effect decreases across the continuous F scale (r = -0.80, p < 0.001), while a 15-treatment atomic sweep localizes the reversal to pipeline guidance. Across 11 datasets spanning four structured data-science task families, AdaSkill combines strong task performance with substantially lower deployment overhead.
Figures & tables
Figure 1: Behavior–outcome geometry governs structural lift. Repeated runs share an input. Trajectory direction schematically encodes behavior; endpoints encode outcome rank, inducing the pairwise distance ranks compared by F . Left: behavior- and outcome-distance ranks align, so skill guidance concentrates search among high-rank behaviors. Right: the ordering breaks, so rigid guidance suppresses valid paths, favoring flexibility.
Figure 2: AdaSkill operationalizes signed structural transfer. A task-qualified source proposes a structural candidate and F measures pre-synthesis behavior–outcome rank geometry. The converter preserves eligible capability resources, conditionally inherits source-pipeline control, removes runtime orchestration, and emits a skill for direct deployment by a tool-using agent.
Task
Source
Control architecture
Externalized analytical decisions
T2SQL
APEX-SQL
Staged and parallel workflow modules
Hypothesis planning; dual-path schema pruning; parallel profiling; global synthesis; SQL exploration, execution, and verification.
Retrieved-context augmentation; causal-constraint construction; statistical graph estimation and refinement.
FE
FELA
Role-specialized evolutionary workflow
Idea and code generation; criticism; evaluation; evolutionary selection over shared memory.
Table 1: Published, high-performing source workflows and externalized control. Each source exposes an analytical decision loop through staged execution, specialized modules, shared artifacts, or feedback.
Figure 3: Overall skill utility follows Behavior-Outcome Freedom. Canonical output-space (a) and response-free trace-space (b) estimates recover the same negative ordering with AdaSkill lift ( r=−0.83 and r=−0.79 ) across all 16 dataset–metric tuples. Every vertical coordinate is the directly deployed output of the adaptive converter and includes capability transfer; Figure 6 isolates the matched structural treatment. Circles are individual datasets; stars are metric means.
Figure 4: F quantifies behavior-outcome coupling. Each point pairs two Base Agent runs. Changing only the scorer separates MSA from MRE in output and response-free trace spaces.
Figure 5: Component ablation separates capability augmentation from structural constraint. The performance row uses one scale throughout: each dataset–metric value is the fraction of Final AdaSkill’s improvement over the Base Agent recovered by the displayed configuration, and points are metric means. Tools+Knowledge is the capability-matched Discard candidate; All Components is the matched Full candidate. Capability-only configurations rise with F , while Pipeline and All Components reverse this ordering. The cost row reports component-matched deployment-cost changes for all five configurations; metrics produced by the same execution share one task-level cost.
Figure 6: Behavior-outcome geometry orders the value of pipeline transfer. Each panel removes one complete workflow regime and refits the structural relationship on the remaining contrasts. Gray points form the training fold; open colored points are the excluded regime and do not affect the fitted line. The gray dotted reference is the all-regime boundary tall=0.644 ; colored dashed lines mark the independently calibrated LOFO boundaries. Excluding CE, CD, T2SQL, and FE gives tLOFO=0.644 , 0.426 , 0.692 , and 0.644 . Every fold preserves the negative ordering ( r=−0.86 to −0.73 ). Across all 16 matched contrasts, r=−0.797 ( p=0.00022 ) and ROC AUC =1.000 (exact p=0.0002289 ).
Figure 8
Figure 9: Budget sensitivity of Fout . The gold operating point ( M=N=6 ) costs 6.12;estimatesstabilizebeyondN{\geq}5$ .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Metric
K
Fmethod
Fexec
Textbook
MSA
6
0.098
0.181
Textbook
MRE
6
1.012
1.085
Real
MSA
5
0.236
0.255
Real
MRE
6
0.697
0.762
Synthetic
MRE
6
0.638
0.651
Appendix
Table 5: CE execution-signature audit. The canonical selected-estimator distance is enriched hierarchically with response-free execution configuration parsed from retained tool calls. Different estimator classes have distance one; within-class distance is one half the Jaccard distance over statistical packages, estimator calls, dataframe columns, causal-design tokens, keyword arguments, and whitelisted numeric configurations. Tool results, final responses, and estimated effects are excluded. K is the number of identifiable diagnostic questions.
Audit
n
Full > Discard
Discard > Full
ROC AUC
Exact p
Route-grade
13
5
8
1.000
0.0008
Complete
16
5
11
1.000
0.0002
Appendix
Table 6: Threshold-free ranking of structural value. The positive class is Full > Discard. Scores use −F ; the exact one-sided test enumerates all label assignments conditional on class count.
Task
Dataset
Metric
K
F
Base
Full
Discard
τpipe
CE
Textbook
MSA
6
.098
.615
.846
.771
+19.6
CE
Real
MSA
5
.235
.871
.903
.871
+25.0
CE
Textbook
MRE
6
1.012
.688
.737
.769
-10.3
CE
Real
MRE
5
.697
.817
.827
.831
-2.1
CE
Synthetic
MRE
6
.638
.894
.924
.951
-26.0
CD
AutoMPG
F1
–
.914
.670
.652
.667
-4.5
Appendix
Table 7: Complete matched pipeline interventions Base, Full, and Discard are higher-is-better utilities; MRE is converted as 1−MRE . The final column is the normalized Full-minus-Discard treatment effect. Grey rows have fewer than three identifiable questions.
Figure 10: The pre-skill coupling signal persists under GPT-5.1. Canonical output-space (a) and response-free trace-space (b) estimates preserve the negative ordering with AdaSkill gain ( r=−0.71 and r=−0.79 ) across all 18 dataset–metric tuples. Both Freedom and gain are re-estimated under GPT-5.1 using the same evaluation protocol as Figure 3 . Black solid lines fit GPT-5.1; blue dashed lines transfer the Sonnet 4.6 fits from Figure 3 without refitting. Circles are individual datasets; stars are metric means.
Task
Dataset
Metric
Estimate
95% interval
n
CE
Textbook
MSA
89.7
[76.4,95.9]
39
CE
Textbook
MRE
23.1
[14.7,32.4]
39
CE
Synthetic
MSA
100.0
[92.1,100.0]
45
CE
Synthetic
MRE
4.3
[2.7,6.1]
45
CE
Real
MSA
96.8
[83.8,99.4]
31
CE
Real
MRE
15.9
[8.2,25.4]
31
Appendix
Table 9: Final AdaSkill estimates with 95% question-level intervals. MRE is lower-is-better; all entries are percentages.
Figure 11: Diversity planner ablation for trace-space Freedom. Both panels show FMSAtrace and FMREtrace ( ×100 ) as a function of evaluation budget ( M questions, N runs). At M=N=20 , planner and no-planner protocols give similar displayed estimates ( FMSAtrace:55 vs. 53 ; FMREtrace:85 vs. 85 ). At M=N=6 , the planner-conditioned values are closer to their respective large-budget values than the independent-run values. Figure 9 reports a separate output-space sweep.
Figure 12: Question-level Fout values from the retained diagnostic run for each dataset. Colour encodes domain and shape encodes dataset.
Estimate
Value
Use
Canonical Fout
0.686–1.071
Converter input and structural analysis
Canonical Ftrace
0.147–1.249
Complementary process-space analysis
Appendix
Table 13: Provenance and use of the T2SQL Freedom estimates. Canonical pre-synthesis means determine both the converter input and the matched structural coordinates in Table 7 .
Figure 13: Performance and cost operating points for the six task-level aggregates in Table 2 . Solid segments connect non-dominated points. AdaSkill lies on the Pareto frontier in every panel; T2SQL EX retains a trade-off between accuracy and cost relative to the workflow reference.
Figure 14: Final performance and efficiency breakdown. Row 1 : selected final-skill performance per dataset (EX% / MSA% / F1 ×100 / AUC ×100 ). Rows 2–3 : selected-final efficiency; T2SQL/CE show the task averages reported in Tables 17 and 14 , while CD/FE show per-dataset latency and cache-adjusted cost. Under our accounting protocol, the evaluated source workflows receive little cache credit because staged and role-conditioned calls change prompts across execution. † FELA cost is estimated: Taobao 7.5–15, Dia 10–22 per run.
Figure 15: AdaSkill architecture transformation for CE ( FMSA≈0 ). Left : the published CAIS workflow comprises four stages and eight shared-state micro-tools; method selection and validation are deterministic, and only a subset of the other tools use LLM assistance. Right : the adaptive skill retains callable tools and descriptive domain facts, expresses the low- F method-selection policy as a metric-scoped pipeline, and discards the source workflow’s global ordering and shared-state orchestration. The diagram therefore compares workflow structure, not a fixed number of model calls.
Large language model (LLM) agent systems are increasingly expected to improve after deployment, but existing work often decouples two adaptation targets: skill evolution and multi-agent system (MAS) restructuring. This separation can create organization bottlenecks, context pressure, and mis-specialization. We present SkillMAS, a non-parametric framework for adaptive specialization in multi-agent systems that couples skill evolution with MAS restructuring. SkillMAS uses Utility Learning to assign credit from verified execution traces, bounded skill evolution to refine reusable procedures without unfiltered library growth, and evidence-gated MAS restructuring when retained failures and Executor Utility indicate a structural mismatch. Across embodied manipulation, command-line execution, and retail workflows, SkillMAS is competitive under the reported harnesses while clarifying how post-deployment specialization is attributed, updated, and applied.
Shuai Pan, Yixiang Liu, Jiaye Gao +7
Shanghai Jiao Tong University · Central South University · OPPO
Agent Skills express open-ended task knowledge in natural language, but composing them requires persistent execution order and upstream-output handoffs. We propose selective formalization at Skill boundaries: coordination relations with stable references and machine-checkable constraints become executable, while substantive task judgments remain within Skills. We introduce MetaSkill, a hybrid representation that preserves complete Skill packages and local instructions while making invocation dependencies and predecessor-text bindings executable through a directed acyclic graph (DAG) runtime. This interface supports explicit coordination without prescribing how each Skill performs its local task. Across 20 multi-Skill artifact tasks and two LLMs, we compare natural-language procedures, prompt-visible graphs, and MetaSkill derived from a common frozen plan per task, using the same designated Skill packages. Additional comparisons evaluate AgentSkillOS with the same available Skill packages as MetaSkill, as well as agent systems using their configured tools. Blinded artifact judgments yield favorable preference point estimates across all reported comparisons, with 95% task-bootstrap intervals excluding parity against Prompt Graph under DeepSeek-V4-Flash and OpenClaw under GLM-5.2. In audited GLM-5.2 execution traces, all planned nodes complete and all declared precedence constraints are satisfied. Together, these findings support MetaSkill as a practical interface for composing open-ended Skills, combining local linguistic flexibility with executable dependencies, explicit predecessor-text bindings, and auditable coordination. Code is available at: https://github.com/TokenRhythm/opensquilla/tree/paper/metaskill-lp4fm-2026
Meng Shu, Wei He, Hang Zhou +3
TokenRhythm Technologies · Xidian University · Beijing Institute of Technology
The rapid proliferation of Claude agent skills has raised the central question of how to effectively leverage, manage, and scale the agent skill ecosystem. In this paper, we propose AgentSkillOS, the first principled framework for skill selection, orchestration, and ecosystem-level management. AgentSkillOS comprises two stages: (i) Manage Skills, which organizes skills into a capability tree via node-level recursive categorization for efficient discovery; and (ii) Solve Tasks, which retrieves, orchestrates, and executes multiple skills through DAG-based pipelines. To evaluate the agent's ability to invoke skills, we construct a benchmark of 30 artifact-rich tasks across five categories: data computation, document creation, motion video, visual design, and web interaction. We assess the quality of task outputs using LLM-based pairwise evaluation, and the results are aggregated via a Bradley-Terry model to produce unified quality scores. Experiments across three skill ecosystem scales (200 to 200K skills) show that tree-based retrieval effectively approximates oracle skill selection, and that DAG-based orchestration substantially outperforms native flat invocation even when given the identical skill set. Our findings confirm that structured composition is the key to unlocking skill potential. Our GitHub repository is available at:https://github.com/ynulihao/AgentSkillOS.