While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substantial token consumption and inference latency, limiting their practical deployment. To enhance the efficiency of agent systems, recent efforts have focused on transferring agentic capabilities from large-scale models to small language models (SLMs) via supervised fine-tuning-based distillation. Nevertheless, as interaction trajectories grow longer, involving extended reasoning-action chains and extensive environmental feedback, distilled student agents tend to experience progress stagnation: becoming trapped in unproductive loops characterized by persistent failed strategies and difficulty making meaningful progress. Our systematic analysis shows that this phenomenon consistently occurs across diverse SLMs and task domains. To address this issue, we propose Progress-Aware Distillation (ProD), which explicitly detects and penalizes stagnation in student-generated trajectories. ProD encodes the stagnation gap between student and teacher into an adaptive margin for dynamically penalizing possible stagnation. By iteratively applying this procedure, ProD enables student agents to progressively approximate teacher behavior. Extensive experiments involving 8 SLMs, ranging from 0.6B to 8B parameters, on both in-domain and out-of-domain benchmarks, demonstrate that ProD substantially mitigates progress stagnation while enhancing both the success rates and task-completion efficiency of distilled SLM agents.
Figures & tables
Figure 1: Comparison between the teacher agent (GPT-4.1) and the student agent (SFT-distilled Qwen3-1.7B) on a web information-retrieval task. The teacher maintains steady progress through coherent reasoning and actions (left) , while the student suffers from progress stagnation, cycling through similar but ineffective search queries and remaining stuck (right) .
Score
Formulation
Lexical
maxj<iROUGEL(hi⊕ai,hj⊕aj)
Semantic
maxj<iCos(E(hi⊕ai),E(hj⊕aj))
Judgment
JLLM(hi⊕ai∣{hj⊕aj}j<i)
Table 1: Three formulations of the turn-level stagnation score Shi⊕ai .
Figure 2: Comparison of stagnation scores between the teacher agent (GPT-4.1) and SFT-distilled students (Qwen3-4/1.7/0.6B): (1&2) turn-level lexical stagnation on MATH and HotpotQA, (3&4) average semantic and judgment stagnation across turns and datasets.
Pearson r
4B
1.7B
0.6B
r(Slex,Ssem)
0.929
0.958
0.964
r(Slex,Sjudge)
0.946
0.975
0.956
r(Ssem,Sjudge)
0.947
0.971
0.979
Table 2: Pearson correlation among stagnation metrics on Qwen SFT agents, averaged across HotpotQA and MATH.
Method
In-domain
Out-of-domain
MATH
HotpotQA
Avg.
Omni-MATH (Level 1–3)
Omni-MATH (Level 4–5)
2Wiki MultiHopQA
Avg.
Teacher Agent
GPT-4.1
90.4
71.0
80.7
82.0
60.0
59.2
67.1
Student Agents
Qwen3-4B
56.8
53.8
55.3
57.8
23.0
44.8
41.9
SFT Distill
81.9
66.6
74.2
73.6
39.4
49.4
54.1
Table 3: Comparison between teacher and student agents, including base models, SFT distillation, and ProD, on in-domain and out-of-domain agent tasks, evaluated by final answer accuracy (%).
Figure 3: Alleviating progress stagnation. Comparison between SFT and ProD agents: Columns 1&2 show turn-level lexical stagnation on MATH and HotpotQA, Columns 3&4 show average semantic and judgment stagnation across turns and datasets.
Cost
SFT
ProD
Avg Turns
4.144
3.663 ( ↓ 11.6%)
Avg Tokens
16206
12847 ( ↓ 20.7%)
Table 4: Inference efficiency averaged over 8 student agents on ID and OOD settings.
Model
SFT
RP(1.1)
RP(1.2)
RP(1.3)
Concise SFT
ProD
Qwen3-4B
66.6
66.2
65.6
61.6
66.4
70.8
Llama3.2-3B
61.6
62.0
61.2
58.0
62.0
67.0
Table 5: Comparison with repetition-control baselines on HotpotQA. RP (⋅) denotes repetition penalty with the corresponding strength. Concise SFT trains on shortened, less redundant teacher trajectories.
Figure 4: Iterative ProD further alleviates stagnation (1&2) and improves accuracy (3&4) on MATH.
Dataset
SFT
ProD
α=1
α=0.75
α=0.5
α=0.25
α=0
Qwen3-4B
MATH
81.9
85.2
85.8
85.2
84.5
85.3
HotpotQA
66.6
69.4
68.8
70.8
66.8
67.6
Average
74.2
77.3
77.3
78.0
75.7
76.4
Llama3-8B
Table 6: Accuracy (%) under different α values in the adaptive margin.
Figure 5: Ablation of optimization objective components in ProD with two ablated variants, ProD (w/o SFT + AM) and ProD (w/o AM), where AM denotes the adaptive margin.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Hyper.
Qwen3 4B
Qwen3 1.7B
Qwen3 0.6B
Llama-3.1 8B-Inst
Llama-3.2 3B-Inst
Llama-3.2 1B-Inst
Gemma-3 4B-it
Gemma-3 1B-it
MATH
LR
5e-6
5e-6
5e-6
5e-6
1e-6
1e-6
5e-6
1e-6
c0
0.5
0.5
0.5
0.5
0.5
0.5
0.7
0.5
HotpotQA
LR
1e-6
1e-5
1e-6
1e-6
5e-6
5e-6
5e-6
5e-6
c0
0.7
0.5
0.7
0.5
0.5
0.5
0.7
0.3
Appendix
Table 7: Selected learning rates (LR) and initial adaptive margins c0 for ProD.
Task
Hyper.
ProD (w/o SFT + AM)
ProD (w/o AM)
Qwen3-4B
Llama-3.1-8-Inst
Gemma-3-4B-it
Qwen3-4B
Llama-3.1-8-Inst
Gemma-3-4B-it
MATH
LR
5e-6
5e-6
1e-6
5e-6
5e-6
5e-6
c0
0.5
0.5
0.3
0.5
0.5
0.7
HotpotQA
LR
5e-6
1e-6
5e-6
1e-6
1e-6
5e-6
c0
0.5
0.5
0.7
0.7
0.5
0.7
Appendix
Table 8: Selected learning rates (LR) and fixed margins c0 for ProD ablations (w/o SFT+AM and w/o AM) across models and tasks.
Figure 6: Comparison of turn-level lexical stagnation scores on MATH and HotpotQA for the teacher agent (GPT-4.1) and SFT-distilled students (Qwen3-4B, Qwen3-1.7B, Llama3.1-8B-Inst, and Llama3.2-3B-Inst) trained with LoRA or full-parameter fine-tuning (FFT).
Figure 7: Comparison of turn-level lexical stagnation scores between the teacher agent (GPT-4.1) and SFT-distilled students on the training dataset.
Model
SFT
ProD-Semantic
ProD
Qwen3-4B
66.6
69.2
70.8
Llama3.2-3B
61.6
66.6
67.0
Appendix
Table 9: ProD with alternative progress metrics on HotpotQA.
Figure 8: Alleviating progress stagnation in Gemma3 agents. Comparison between SFT and ProD: 1&2 show turn-level lexical stagnation on MATH and HotpotQA, 3&4 show average semantic and judgment stagnation across turns and datasets.
Table 10: Wall-clock time breakdown on HotpotQA (in minutes).
Method
Accuracy (%) ↑
Lexical Stagnation ↓
SFT
55.5
0.678
ProD (Iter-1)
58.6
0.568
ProD (Iter-2)
61.2
0.346
ProD (Iter-3)
62.1
0.345
ProD (Iter-4)
61.0
0.348
Appendix
Table 11: Results of additional ProD iterations on Llama3.2-3B for MATH.
Model
SFT
DPO
SimPO
ProD
Qwen3-4B
74.2
74.4
74.1
78.0
Llama3.1-8B
61.8
62.7
64.9
67.2
Gemma3-4B
56.4
53.8
56.7
61.0
Appendix
Table 12: On-policy preference optimization baselines, averaged over MATH and HotpotQA.
Figure 9: Average task accuracy (%) for SFT, RS-SFT, and ProD on HotpotQA and MATH.
Dataset
Model
SFT
RS-SFT
ProD
HotpotQA
Qwen3-4B
0.544
0.574
0.397
Llama3.1-8B
0.525
0.550
0.459
MATH
Qwen3-4B
0.443
0.525
0.337
Llama3.1-8B
0.638
0.766
0.303
Appendix
Table 13: Average turn-level lexical stagnation scores for SFT, RS-SFT, and ProD.
Model
Evidence-hit Rate ↑
Obs. Redundancy ↓
SFT
ProD
SFT
ProD
Qwen3-4B
38.4
44.3
14.1
7.0
Qwen3-1.7B
31.4
41.1
19.4
6.5
Qwen3-0.6B
31.3
35.5
28.1
14.1
Llama3.1-8B
36.4
42.5
14.6
7.7
Llama3.2-3B
34.6
43.8
16.5
8.0
Appendix
Table 14: Evidence-hit rate and observation redundancy (%) on HotpotQA.
Model
Method
MMLU
ARC-C
HellaSwag
WinoGrande
Avg.
Qwen3-4B
Base
72.7
62.6
62.4
67.2
66.2
SFT
71.8
64.0
63.0
65.8
66.2
ProD
71.3
64.8
64.2
65.8
66.5
Llama3.1-8B
Base
68.3
60.4
73.0
80.4
70.5
SFT
67.6
61.2
72.8
82.0
70.9
ProD
67.9
63.0
74.0
81.6
71.6
Appendix
Table 15: General benchmark accuracy (%) before and after agent distillation.
Dataset
GPT-4o-mini
SFT
ProD
HotpotQA
65.2
56.2
60.4
MATH
75.2
63.4
67.5
Appendix
Table 16: Task accuracy (%) with GPT-4o-mini as the teacher and Qwen3-1.7B as the student.
Figure 10: Qualitative comparison between SFT- and ProD-distilled Qwen3-1.7B student agents on a two-hop HotpotQA question. The SFT-distilled agent repeatedly issues similar search queries that attempt to jointly cover both hops, failing to extract useful intermediate information and exhibiting persistent progress stagnation (left) . In contrast, the ProD-distilled agent decomposes the task into a clear two-hop search process, leveraging intermediate observations to guide subsequent actions and achieving steady progress toward the correct answer (right) .
Figure 11: Qualitative comparison between SFT- and ProD-distilled Qwen3-4B student agents on a HotpotQA evaluation example. Although the SFT-distilled agent recognizes the failure of its initial search attempt, it remains trapped in a loop of near-duplicate queries without acquiring new information (left) . By contrast, the ProD-distilled agent adapts its search strategy after early unsuccessful attempts, escapes the stagnation loop, and successfully retrieves the required information to solve the task (right) .
Learning from large-model demonstrations offers a way to train small agents that can complete recurring tasks without calling a large model at every step. A central design choice is what to retain from teacher trajectories that contain reasoning, actions, and information about task progress. We introduce Task-Progress Distillation (TPD), an offline approach that pairs each demonstrated action with a short label describing the current task stage. The student learns these compact targets and selects actions by jointly scoring admissible stage--action pairs, which a deterministic harness executes in the environment. On ALFWorld, a 1.7B student trained with 404 demonstrations achieves 72.4% mean unseen task success with either TPD or action-only supervision, compared with 48.3% for a reasoning-trained student using constrained action selection. Explicit stages provide an additional benefit at 200 demonstrations, improving success from 48.0% to 67.7% over action-only supervision. With more demonstrations, the action-only student closes the gap, and both approaches reach 76.9% at 808 demonstrations. Shared-history analyses link part of TPD's local advantage to better decisions when moving between subgoals, particularly from object acquisition to processing. These results show that compact supervision can train effective small task agents, while explicit task progress provides additional guidance at an intermediate demonstration budget.
Wenxi Gan
City University of Hong Kong Shenzhen Loop Area Institute
Language model agents are deployed with a harness, the software around the model that manages its context, tools, and feedback. When such an agent is distilled into a smaller one, the harness stays in place, so the student mainly needs the teacher-specific abilities that the harness cannot provide, such as acting correctly on harness information. Standard distillation, however, imitates the teacher's full outputs and treats the harness as part of the input. We propose Harness-Aware Distillation (HAD), which focuses distillation on what the teacher adds beyond the harness. HAD complements on-policy distillation with two components: an action preference that contrasts the same teacher's actions with and without the harness information, scored after the student's own reasoning, and a validity check that drops preference pairs whose preferred action contradicts the harness records. We show that the contrast gives the student information that imitating the teacher alone cannot provide, and HAD needs no task rewards, success labels, or future information. Across multiple long-horizon agent benchmarks and models, HAD outperforms on-policy distillation baselines with the same fixed harness. Our analysis shows that HAD enters fewer unproductive loops and recovers from errors more often than the baselines, and suggests that it adaptively keeps learnable feedback in its weights while reading state information from the harness.
Memory systems have shown promise for improving agent performance, but their potential remains largely unexplored for small language models, which struggle to generate sufficient successful trajectories on their own. We propose Agent Memory Distillation (AMD), a training-free framework that transfers structured knowledge from a large teacher agent to a small student agent through hierarchical memory. AMD constructs three complementary memory types from successful teacher trajectories: Workflow memory encodes task-level strategies, Subtask memory provides concrete behavioral examples at an intermediate granularity, and Function memory captures per-function calling conventions and common pitfalls. Workflow and Subtask memories are injected proactively at the start of each task, while Function memory is retrieved reactively upon tool-calling errors. We evaluate AMD on three tool-use benchmarks using four student models (4B-8B parameters) with GPT-5-mini as the teacher, achieving average accuracy gains of 27.2%p, 11.2%p, and 3.4%p on AppWorld, BFCL V3, and ToolSandbox, while consistently outperforming existing memory-based baselines. Further analysis shows that Subtask memory contributes the largest gains, teacher effectiveness depends on both teacher capability and student compatibility, and 4B-sized students benefit most from AMD.