cs.AISep 29, 2026

Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses

Authors: Ziluowen Luo, Senzhang Wang, Chaozhuo Li, Jun Yin, Hao Yan, Ming Cheng, Chenxu Wang, Songyang Liu, +4 more

Organizations: Central South University, ChangSha, Hunan, China · Beijing Academy of Artificial Intelligence, Beijing, China · Hong Kong Polytechnic University, Hongkong, China · Beijing University of Posts and Telecommunications, Beijing, China · University of Illinois at Chicago, Chicago, Illinois, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Modern agents increasingly rely on memories, tools, and execution logic, so their competence extends beyond model parameters. This shift exposes a limitation of conventional knowledge distillation, which asks how a student model imitates a teacher model. We define Agent Distillation as the persistent transfer of task-solving knowledge from a teacher agent to a student agent. Our study organizes the field by where transferred knowledge is retained: within the model, as artifacts, through the execution harness, or across substrates. This perspective separates transfer evidence from its outcome and clarifies how knowledge moves between agent components. We develop an evaluation framework that relates retention to causal contribution and deployed utility. Together, these contributions establish a foundation for the reliable, maintainable, and safe development of increasingly complex agentic systems.

Figures & tables

Figure 1. From single successful agent to broader reuse of task-solving knowledge.
SurveyYearT →\rightarrow SSubstrate CoverageCross- SubstrateTransfer Evaluation
   ModelArtifactHarness  
Distillation-related Surveys
Gou et al. ( Gou et al., 2021 )2021✓✓✗✗✗⚫
Xu et al. ( Xu et al., 2024 )2024✓✓✗✗✗⚫
Yang et al. ( Yang et al., 2025 )2025✓✓✗✗✗✓
Mansourian et al. ( Mansourian et al., 2025 )2025✓✓✗✗✗⚫
Table 1. Comparison with representative surveys from the distillation and agent literatures. ✓ denotes systematic treatment as a primary organizing perspective, ⚫ denotes partial treatment, and ✗ indicates that the aspect is outside the scope of the survey.
Figure 2. From model-centered to agent-centered knowledge transfer. Agent Distillation treats the complete agent realization as the unit of teacher–student transfer. Our knowledge substrate-centric view organizes existing approaches according to the primary form in which transferred knowledge persists in the student: the parameteric model , reusable artifacts or the dynamic harness .
Figure 3. A running example of Agent Distillation. (a) A teacher coding agent repairs a repository by recovering from an over-broad patch and verifying localized fix. (b) The trace mixes incidental residue, task-specific information and reusable patterns; filtering and abstraction turn selected evidence into a debugging strategy. (c) Then it is realized in student agents and reused on other tasks.
SubstrateEmbodimentRepresentative formsRole in future task solving
ModelLearned model computationParameters, adapters, internal representations, learned policiesInternalizes knowledge to directly shape inference and decisions
ArtifactExplicit reusable objectsMemories, demonstrations, plans, code, skills, and toolsExternalizes knowledge for retrieval and reuse across episodes
HarnessDynamic execution logicWorkflows, routing, verification, recovery, and orchestration logicOperationalizes knowledge by organizing and regulating execution
Table 2. Principal knowledge substrates in Agent Distillation and their roles in future task solving.
Figure 4. The progression of parametric distillation toward environment-grounded agent policy learning.
Figure 5. Representative artifact forms in Agent Distillation. The classification is determined by what persists in RS+\mathcal{R}_{S}^{+} , rather than by the teacher evidence from which it was constructed or the runtime mechanism that later operates it.
BenchmarkSettingSubstrate focusMeasuresDistillation reading / caveat
Parametric reasoning / process
ERASER ( DeYoung et al., 2020 )Static text; offlineMEvidence selection; sufficiency; comprehensivenessProcess supervision and error analysis; faithfulness needs intervention.
ProcessBench ( Zheng et al., 2025 )Static text; offlineMStep validity; first-error localizationReasoning supervision; no closed-loop recovery.
Tool and stateful interaction
APIBank ( Li et al., 2023b )API calls; offlineM/HAPI retrieval; call generation; response productionTool-use transfer; model, tool, and harness effects confounded.
BFCL ( Patil et al., 2025 )Structured/multi-turn function callsM/HRelevance; call validity; multi-turn success; latency/costStructured action supervision; interface version matters.
Table 3. Representative benchmarks for Agent Distillation. M, R, and H denote model, artifact, and harness substrates; these are interpretive substrate foci for attribution experiments, not benchmark-internal ground truth.

Explore similar work

CardsList

  1. Harness-Zero: Harness Distillation via Agent-as-Harness

    Sep 21, 2026Haoran Ye, Yuxing Lu, Haonan Dong +2Agent HarnessSymbolic Distillation

    State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · College of Future Technology, Peking University · Google +1

  2. From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search

    Jul 27, 2026Junlin Liu, Jiangwang Chen, Zixin Song +7Knowledge DistillationAgentic Search

    University of Chinese Academy of Sciences · 2Tsinghua University · 3Peking University +3

  3. Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory

    Aug 7, 2026Taeil Kim, Kangsan Kim, Sung Ju HwangLarge Language Model AgentsFrontier Large Language Model Agents

    KAIST