From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?

Binyan Xu; Dong Fang; Haitao Li; Kehuan Zhang

cs.AIApr 2, 2026

From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?

Authors: Binyan Xu, Dong Fang, Haitao Li, Kehuan Zhang

Organizations: The Chinese University of Hong Kong, Hong Kong, China · LIGHTSPEED, Shenzhen, China

Abstract

Multi-agent systems (MAS) for structured data-science tasks externalize analytical control through workflows spanning stages, tools, shared state, verification, and repair. Distilling such workflows into a single-agent skill can reduce orchestration overhead, but it remains unclear which workflow components should cross the control boundary. We distinguish capability resources, which expand what an agent can do, from pipeline guidance, which constrains which solutions it explores. On the same causal-estimation instances, adding task-qualified source pipeline guidance to a capability-matched skill changes normalized utility by +19.6 points under method-selection accuracy but -10.3 points under numerical error. To explain this reversal, we introduce Behavior-Outcome Freedom (F), a pre-synthesis diagnostic of signed behavior-outcome rank mismatch, and formalize its candidate-conditional role through Signed Anchor-Rank Transfer. Motivated by this mechanism, we propose AdaSkill, which preserves validated capability resources, removes runtime orchestration, and conditionally inherits pipeline guidance using a calibrated rule over F. Across 16 capability-matched interventions, the native-scale Full-minus-Discard effect decreases across the continuous F scale (r = -0.80, p < 0.001), while a 15-treatment atomic sweep localizes the reversal to pipeline guidance. Across 11 datasets spanning four structured data-science task families, AdaSkill combines strong task performance with substantially lower deployment overhead.

Figures & tables

Figure 1: Behavior–outcome geometry governs structural lift. Repeated runs share an input. Trajectory direction schematically encodes behavior; endpoints encode outcome rank, inducing the pairwise distance ranks compared by $F$ . Left: behavior- and outcome-distance ranks align, so skill guidance concentrates search among high-rank behaviors. Right: the ordering breaks, so rigid guidance suppresses valid paths, favoring flexibility.

Figure 2: AdaSkill operationalizes signed structural transfer. A task-qualified source proposes a structural candidate and $F$ measures pre-synthesis behavior–outcome rank geometry. The converter preserves eligible capability resources, conditionally inherits source-pipeline control, removes runtime orchestration, and emits a skill for direct deployment by a tool-using agent.

Task	Source	Control architecture	Externalized analytical decisions
T2SQL	APEX-SQL	Staged and parallel workflow modules	Hypothesis planning; dual-path schema pruning; parallel profiling; global synthesis; SQL exploration, execution, and verification.
CE	CAIS	Four-stage, eight-micro-tool pipeline	Query parsing; data analysis; variable interpretation; method selection; estimation; validation; refinement; reporting.
CD	MATMCD	Two modules in a centralized workflow	Retrieved-context augmentation; causal-constraint construction; statistical graph estimation and refinement.
FE	FELA	Role-specialized evolutionary workflow	Idea and code generation; criticism; evaluation; evolutionary selection over shared memory.

Table 1: Published, high-performing source workflows and externalized control. Each source exposes an analytical decision loop through staged execution, specialized modules, shared artifacts, or feedback.

Figure 3: Overall skill utility follows Behavior-Outcome Freedom. Canonical output-space (a) and response-free trace-space (b) estimates recover the same negative ordering with AdaSkill lift (

r=-0.83

and

r=-0.79

) across all 16 dataset–metric tuples. Every vertical coordinate is the directly deployed output of the adaptive converter and includes capability transfer; Figure 6 isolates the matched structural treatment. Circles are individual datasets; stars are metric means.

Panel of Figure 4 — Figure 4: $F$ quantifies behavior-outcome coupling. Each point pairs two Base Agent runs. Changing only the scorer separates MSA from MRE in output and response-free trace spaces.

Figure 5: Component ablation separates capability augmentation from structural constraint. The performance row uses one scale throughout: each dataset–metric value is the fraction of Final AdaSkill’s improvement over the Base Agent recovered by the displayed configuration, and points are metric means. Tools+Knowledge is the capability-matched Discard candidate; All Components is the matched Full candidate. Capability-only configurations rise with

F

, while Pipeline and All Components reverse this ordering. The cost row reports component-matched deployment-cost changes for all five configurations; metrics produced by the same execution share one task-level cost.

Figure 6: Behavior-outcome geometry orders the value of pipeline transfer. Each panel removes one complete workflow regime and refits the structural relationship on the remaining contrasts. Gray points form the training fold; open colored points are the excluded regime and do not affect the fitted line. The gray dotted reference is the all-regime boundary

t_{\rm all}=0.644

; colored dashed lines mark the independently calibrated LOFO boundaries. Excluding CE, CD, T2SQL, and FE gives

t_{\rm LOFO}=0.644

0.426

0.692

, and

0.644

. Every fold preserves the negative ordering (

r=-0.86

-0.73

). Across all 16 matched contrasts,

r=-0.797

(

p=0.00022

) and ROC AUC

=1.000

(exact

p=0.0002289

Panel of Figure 9 — Figure 9: Budget sensitivity of $F_{\mathrm{out}}$ . The gold operating point ( $M{=}N{=}6$ ) costs $6.12; estimates stabilize beyond$ N{\geq}5$ .

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Metric	$K$	$F_{\mathrm{method}}$	$F_{\mathrm{exec}}$
Textbook	MSA	6	0.098	0.181
Textbook	MRE	6	1.012	1.085
Real	MSA	5	0.236	0.255
Real	MRE	6	0.697	0.762
Synthetic	MRE	6	0.638	0.651

Appendix

Table 5: CE execution-signature audit. The canonical selected-estimator distance is enriched hierarchically with response-free execution configuration parsed from retained tool calls. Different estimator classes have distance one; within-class distance is one half the Jaccard distance over statistical packages, estimator calls, dataframe columns, causal-design tokens, keyword arguments, and whitelisted numeric configurations. Tool results, final responses, and estimated effects are excluded.

K

is the number of identifiable diagnostic questions.

Audit	$n$	Full $>$ Discard	Discard $>$ Full	ROC AUC	Exact $p$
Route-grade	13	5	8	1.000	0.0008
Complete	16	5	11	1.000	0.0002

Appendix

Table 6: Threshold-free ranking of structural value. The positive class is Full

>

Discard. Scores use

-F

; the exact one-sided test enumerates all label assignments conditional on class count.

Task	Dataset	Metric	$K$	$F$	Base	Full	Discard	$\tau_{\rm pipe}$
CE	Textbook	MSA	6	.098	.615	.846	.771	+19.6
CE	Real	MSA	5	.235	.871	.903	.871	+25.0
CE	Textbook	MRE	6	1.012	.688	.737	.769	-10.3
CE	Real	MRE	5	.697	.817	.827	.831	-2.1
CE	Synthetic	MRE	6	.638	.894	.924	.951	-26.0
CD	AutoMPG	F1	–	.914	.670	.652	.667	-4.5

Appendix

Table 7: Complete matched pipeline interventions Base, Full, and Discard are higher-is-better utilities; MRE is converted as

1-\mathrm{MRE}

. The final column is the normalized Full-minus-Discard treatment effect. Grey rows have fewer than three identifiable questions.

Figure 10: The pre-skill coupling signal persists under GPT-5.1. Canonical output-space (a) and response-free trace-space (b) estimates preserve the negative ordering with AdaSkill gain (

r=-0.71

and

r=-0.79

) across all 18 dataset–metric tuples. Both Freedom and gain are re-estimated under GPT-5.1 using the same evaluation protocol as Figure 3 . Black solid lines fit GPT-5.1; blue dashed lines transfer the Sonnet 4.6 fits from Figure 3 without refitting. Circles are individual datasets; stars are metric means.

Task	Dataset	Metric	Estimate	95% interval	$n$
CE	Textbook	MSA	89.7	$[76.4,95.9]$	39
CE	Textbook	MRE	23.1	$[14.7,32.4]$	39
CE	Synthetic	MSA	100.0	$[92.1,100.0]$	45
CE	Synthetic	MRE	4.3	$[2.7,6.1]$	45
CE	Real	MSA	96.8	$[83.8,99.4]$	31
CE	Real	MRE	15.9	$[8.2,25.4]$	31

Appendix

Table 9: Final AdaSkill estimates with 95% question-level intervals. MRE is lower-is-better; all entries are percentages.

Panel of Figure 11 — Figure 11: Diversity planner ablation for trace-space Freedom. Both panels show $F_{\mathrm{MSA}}^{\mathrm{trace}}$ and $F_{\mathrm{MRE}}^{\mathrm{trace}}$ ( $\times 100$ ) as a function of evaluation budget ( $M$ questions, $N$ runs). At $M{=}N{=}20$ , planner and no-planner protocols give similar displayed estimates ( $F_{\mathrm{MSA}}^{\mathrm{trace}}{:}\;55$ vs. $53$ ; $F_{\mathrm{MRE}}^{\mathrm{trace}}{:}\;85$ vs. $85$ ). At $M{=}N{=}6$ , the planner-conditioned values are closer to their respective large-budget values than the independent-run values. Figure 9 reports a separate output-space sweep.

Figure 12: Question-level

F^{\mathrm{out}}

values from the retained diagnostic run for each dataset. Colour encodes domain and shape encodes dataset.

Estimate	Value	Use
Canonical $F_{\mathrm{out}}$	0.686–1.071	Converter input and structural analysis
Canonical $F_{\mathrm{trace}}$	0.147–1.249	Complementary process-space analysis

Appendix

Table 13: Provenance and use of the T2SQL Freedom estimates. Canonical pre-synthesis means determine both the converter input and the matched structural coordinates in Table 7 .

Panel of Figure 13 — Figure 13: Performance and cost operating points for the six task-level aggregates in Table 2 . Solid segments connect non-dominated points. AdaSkill lies on the Pareto frontier in every panel; T2SQL EX retains a trade-off between accuracy and cost relative to the workflow reference.

Panel of Figure 14 — Figure 14: Final performance and efficiency breakdown. Row 1 : selected final-skill performance per dataset (EX% / MSA% / F1 ${\times}100$ / AUC ${\times}100$ ). Rows 2–3 : selected-final efficiency; T2SQL/CE show the task averages reported in Tables 17 and 14 , while CD/FE show per-dataset latency and cache-adjusted cost. Under our accounting protocol, the evaluated source workflows receive little cache credit because staged and role-conditioned calls change prompts across execution. ${\dagger}$ FELA cost is estimated: Taobao $7.5–$ 15, Dia $10–$ 22 per run.

Panel of Figure 15 — Figure 15: AdaSkill architecture transformation for CE ( $F_{\mathrm{MSA}}\approx 0$ ). Left : the published CAIS workflow comprises four stages and eight shared-state micro-tools; method selection and validation are deterministic, and only a subset of the other tools use LLM assistance. Right : the adaptive skill retains callable tools and descriptive domain facts, expresses the low- $F$ method-selection policy as a metric-scoped pipeline, and discards the source workflow’s global ordering and shared-state orchestration. The diagram therefore compares workflow structure, not a fixed number of model calls.

Agent Skill Learning

Agent skill learning develops reusable procedures and behaviors that help AI agents solve new tasks and adapt from experience.

See all 173 papers

SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent System
May 10, 2026Shuai Pan, Yixiang Liu, Jiaye Gao +7Multi-Agent System Optimization LLM Agent Skill Learning
Shanghai Jiao Tong University · Central South University · OPPO
Where Natural Language Ends in Agents: Executable Skill Composition with MetaSkill
Oct 17, 2026Meng Shu, Wei He, Hang Zhou +3LLM Agent Orchestration LLM Agents
TokenRhythm Technologies · Xidian University · Beijing Institute of Technology
Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale
Mar 2, 2026Hao Li, Chunjiang Mu, Jianhao Chen +5LLM Agent Orchestration LLM Agent Skill Retrieval
Shanghai Artificial Intelligence Laboratory

May 10, 2026J/K move · Enter open · S save

SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent System

Shuai Pan, Yixiang Liu, Jiaye Gao, Te Gao+6

Shanghai Jiao Tong University · Central South University · OPPO

Multi-Agent System Optimization LLM Agent Skill Learning Multi-Agent Systems Self-Evolving Agents

Open paper AskPDF arXiv

Large language model (LLM) agent systems are increasingly expected to improve after deployment, but existing work often decouples two adaptation targets: skill evolution and multi-agent system (MAS) restructuring. This separation can create organization bottlenecks, context pressure, and mis-specialization. We present SkillMAS, a non-parametric framework for adaptive specialization in multi-agent systems that couples skill evolution with MAS restructuring. SkillMAS uses Utility Learning to assign credit from verified execution traces, bounded skill evolution to refine reusable procedures without unfiltered library growth, and evidence-gated MAS restructuring when retained failures and Executor Utility indicate a structural mismatch. Across embodied manipulation, command-line execution, and retail workflows, SkillMAS is competitive under the reported harnesses while clarifying how post-deployment specialization is attributed, updated, and applied.

Abstract

Agent Skill Learning

Explore similar work

SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent System

Where Natural Language Ends in Agents: Executable Skill Composition with MetaSkill

Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale