Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents

Yuanpu Cao; Saket Sathe; Hanyu Wang; Ziyi Yin; Fenglong Ma; Jinghui Chen

cs.LGOct 14, 2026

Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents

Authors: Yuanpu Cao, Saket Sathe, Hanyu Wang, Ziyi Yin, Fenglong Ma, Jinghui Chen

Organizations: The Pennsylvania State University · Microsoft CoreAI

Abstract

While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substantial token consumption and inference latency, limiting their practical deployment. To enhance the efficiency of agent systems, recent efforts have focused on transferring agentic capabilities from large-scale models to small language models (SLMs) via supervised fine-tuning-based distillation. Nevertheless, as interaction trajectories grow longer, involving extended reasoning-action chains and extensive environmental feedback, distilled student agents tend to experience progress stagnation: becoming trapped in unproductive loops characterized by persistent failed strategies and difficulty making meaningful progress. Our systematic analysis shows that this phenomenon consistently occurs across diverse SLMs and task domains. To address this issue, we propose Progress-Aware Distillation (ProD), which explicitly detects and penalizes stagnation in student-generated trajectories. ProD encodes the stagnation gap between student and teacher into an adaptive margin for dynamically penalizing possible stagnation. By iteratively applying this procedure, ProD enables student agents to progressively approximate teacher behavior. Extensive experiments involving 8 SLMs, ranging from 0.6B to 8B parameters, on both in-domain and out-of-domain benchmarks, demonstrate that ProD substantially mitigates progress stagnation while enhancing both the success rates and task-completion efficiency of distilled SLM agents.

Figures & tables

Figure 1: Comparison between the teacher agent (GPT-4.1) and the student agent (SFT-distilled Qwen3-1.7B) on a web information-retrieval task. The teacher maintains steady progress through coherent reasoning and actions (left) , while the student suffers from progress stagnation, cycling through similar but ineffective search queries and remaining stuck (right) .

Score	Formulation
Lexical	$\max_{j<i}\text{ROUGE}_{\text{L}}(\boldsymbol{h}_{i}\oplus\boldsymbol{a}_{i},\boldsymbol{h}_{j}\oplus\boldsymbol{a}_{j})$
Semantic	$\max_{j<i}\text{Cos}(E(\boldsymbol{h}_{i}\oplus\boldsymbol{a}_{i}),E(\boldsymbol{h}_{j}\oplus\boldsymbol{a}_{j}))$
Judgment	$J_{\text{LLM}}(\boldsymbol{h}_{i}\oplus\boldsymbol{a}_{i}\mid\{\boldsymbol{h}_{j}\oplus\boldsymbol{a}_{j}\}_{j<i})$

Table 1: Three formulations of the turn-level stagnation score

S_{\boldsymbol{h}_{i}\oplus\boldsymbol{a}_{i}}

Figure 2: Comparison of stagnation scores between the teacher agent (GPT-4.1) and SFT-distilled students (Qwen3-4/1.7/0.6B): (1&2) turn-level lexical stagnation on MATH and HotpotQA, (3&4) average semantic and judgment stagnation across turns and datasets.

Pearson $r$	4B	1.7B	0.6B
$r(S^{\mathrm{lex}},S^{\mathrm{sem}})$	0.929	0.958	0.964
$r(S^{\mathrm{lex}},S^{\mathrm{judge}})$	0.946	0.975	0.956
$r(S^{\mathrm{sem}},S^{\mathrm{judge}})$	0.947	0.971	0.979

Table 2: Pearson correlation among stagnation metrics on Qwen SFT agents, averaged across HotpotQA and MATH.

Method	In-domain			Out-of-domain
Method	MATH	HotpotQA	Avg.	Omni-MATH (Level 1–3)	Omni-MATH (Level 4–5)	2Wiki MultiHopQA	Avg.
Teacher Agent
GPT-4.1	90.4	71.0	80.7	82.0	60.0	59.2	67.1
Student Agents
Qwen3-4B	56.8	53.8	55.3	57.8	23.0	44.8	41.9
SFT Distill	81.9	66.6	74.2	73.6	39.4	49.4	54.1

Table 3: Comparison between teacher and student agents, including base models, SFT distillation, and ProD, on in-domain and out-of-domain agent tasks, evaluated by final answer accuracy (%).

Figure 3: Alleviating progress stagnation. Comparison between SFT and ProD agents: Columns 1&2 show turn-level lexical stagnation on MATH and HotpotQA, Columns 3&4 show average semantic and judgment stagnation across turns and datasets.

Cost	SFT	ProD
Avg Turns	4.144	3.663 ( $\downarrow$ 11.6%)
Avg Tokens	16206	12847 ( $\downarrow$ 20.7%)

Table 4: Inference efficiency averaged over 8 student agents on ID and OOD settings.

Model	SFT	RP(1.1)	RP(1.2)	RP(1.3)	Concise SFT	ProD
Qwen3-4B	66.6	66.2	65.6	61.6	66.4	70.8
Llama3.2-3B	61.6	62.0	61.2	58.0	62.0	67.0

Table 5: Comparison with repetition-control baselines on HotpotQA. RP

(\cdot)

denotes repetition penalty with the corresponding strength. Concise SFT trains on shortened, less redundant teacher trajectories.

Figure 4: Iterative ProD further alleviates stagnation (1&2) and improves accuracy (3&4) on MATH.

Dataset	SFT	ProD
Dataset	SFT	$\alpha=1$	$\alpha=0.75$	$\alpha=0.5$	$\alpha=0.25$	$\alpha=0$
Qwen3-4B
MATH	81.9	85.2	85.8	85.2	84.5	85.3
HotpotQA	66.6	69.4	68.8	70.8	66.8	67.6
Average	74.2	77.3	77.3	78.0	75.7	76.4
Llama3-8B

Table 6: Accuracy (%) under different

\alpha

values in the adaptive margin.

Figure 5: Ablation of optimization objective components in ProD with two ablated variants, ProD (w/o SFT + AM) and ProD (w/o AM), where AM denotes the adaptive margin.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Hyper.	Qwen3 4B	Qwen3 1.7B	Qwen3 0.6B	Llama-3.1 8B-Inst	Llama-3.2 3B-Inst	Llama-3.2 1B-Inst	Gemma-3 4B-it	Gemma-3 1B-it
MATH	LR	5e-6	5e-6	5e-6	5e-6	1e-6	1e-6	5e-6	1e-6
MATH	$c_{0}$	0.5	0.5	0.5	0.5	0.5	0.5	0.7	0.5
HotpotQA	LR	1e-6	1e-5	1e-6	1e-6	5e-6	5e-6	5e-6	5e-6
HotpotQA	$c_{0}$	0.7	0.5	0.7	0.5	0.5	0.5	0.7	0.3

Appendix

Table 7: Selected learning rates (LR) and initial adaptive margins

c_{0}

for ProD.

Task	Hyper.	ProD (w/o SFT + AM)			ProD (w/o AM)
Task	Hyper.	Qwen3-4B	Llama-3.1-8-Inst	Gemma-3-4B-it	Qwen3-4B	Llama-3.1-8-Inst	Gemma-3-4B-it
MATH	LR	5e-6	5e-6	1e-6	5e-6	5e-6	5e-6
MATH	$c_{0}$	0.5	0.5	0.3	0.5	0.5	0.7
HotpotQA	LR	5e-6	1e-6	5e-6	1e-6	1e-6	5e-6
HotpotQA	$c_{0}$	0.5	0.5	0.7	0.7	0.5	0.7

Appendix

Table 8: Selected learning rates (LR) and fixed margins

c_{0}

for ProD ablations (w/o SFT+AM and w/o AM) across models and tasks.

Figure 6: Comparison of turn-level lexical stagnation scores on MATH and HotpotQA for the teacher agent (GPT-4.1) and SFT-distilled students (Qwen3-4B, Qwen3-1.7B, Llama3.1-8B-Inst, and Llama3.2-3B-Inst) trained with LoRA or full-parameter fine-tuning (FFT).

Figure 7: Comparison of turn-level lexical stagnation scores between the teacher agent (GPT-4.1) and SFT-distilled students on the training dataset.

Model	SFT	ProD-Semantic	ProD
Qwen3-4B	66.6	69.2	70.8
Llama3.2-3B	61.6	66.6	67.0

Appendix

Table 9: ProD with alternative progress metrics on HotpotQA.

Figure 8: Alleviating progress stagnation in Gemma3 agents. Comparison between SFT and ProD: 1&2 show turn-level lexical stagnation on MATH and HotpotQA, 3&4 show average semantic and judgment stagnation across turns and datasets.

Component	Qwen3-4B	Qwen3-1.7B
SFT training (3 epochs)	34.7	19.7
Rollout generation + stagnation score (pre-computed)	73.8	44.4
ProD training (1 epoch)	22.2	12.1

Appendix

Table 10: Wall-clock time breakdown on HotpotQA (in minutes).

Method	Accuracy (%) $\uparrow$	Lexical Stagnation $\downarrow$
SFT	55.5	0.678
ProD (Iter-1)	58.6	0.568
ProD (Iter-2)	61.2	0.346
ProD (Iter-3)	62.1	0.345
ProD (Iter-4)	61.0	0.348

Appendix

Table 11: Results of additional ProD iterations on Llama3.2-3B for MATH.

Model	SFT	DPO	SimPO	ProD
Qwen3-4B	74.2	74.4	74.1	78.0
Llama3.1-8B	61.8	62.7	64.9	67.2
Gemma3-4B	56.4	53.8	56.7	61.0

Appendix

Table 12: On-policy preference optimization baselines, averaged over MATH and HotpotQA.

Figure 9: Average task accuracy (%) for SFT, RS-SFT, and ProD on HotpotQA and MATH.

Dataset	Model	SFT	RS-SFT	ProD
HotpotQA	Qwen3-4B	0.544	0.574	0.397
HotpotQA	Llama3.1-8B	0.525	0.550	0.459
MATH	Qwen3-4B	0.443	0.525	0.337
MATH	Llama3.1-8B	0.638	0.766	0.303

Appendix

Table 13: Average turn-level lexical stagnation scores for SFT, RS-SFT, and ProD.

Model	Evidence-hit Rate $\uparrow$		Obs. Redundancy $\downarrow$
Model	SFT	ProD	SFT	ProD
Qwen3-4B	38.4	44.3	14.1	7.0
Qwen3-1.7B	31.4	41.1	19.4	6.5
Qwen3-0.6B	31.3	35.5	28.1	14.1
Llama3.1-8B	36.4	42.5	14.6	7.7
Llama3.2-3B	34.6	43.8	16.5	8.0

Appendix

Table 14: Evidence-hit rate and observation redundancy (%) on HotpotQA.

Model	Method	MMLU	ARC-C	HellaSwag	WinoGrande	Avg.
Qwen3-4B	Base	72.7	62.6	62.4	67.2	66.2
	SFT	71.8	64.0	63.0	65.8	66.2
	ProD	71.3	64.8	64.2	65.8	66.5
Llama3.1-8B	Base	68.3	60.4	73.0	80.4	70.5
	SFT	67.6	61.2	72.8	82.0	70.9
	ProD	67.9	63.0	74.0	81.6	71.6

Appendix

Table 15: General benchmark accuracy (%) before and after agent distillation.

Dataset	GPT-4o-mini	SFT	ProD
HotpotQA	65.2	56.2	60.4
MATH	75.2	63.4	67.5

Appendix

Table 16: Task accuracy (%) with GPT-4o-mini as the teacher and Qwen3-1.7B as the student.

Figure 10: Qualitative comparison between SFT- and ProD-distilled Qwen3-1.7B student agents on a two-hop HotpotQA question. The SFT-distilled agent repeatedly issues similar search queries that attempt to jointly cover both hops, failing to extract useful intermediate information and exhibiting persistent progress stagnation (left) . In contrast, the ProD-distilled agent decomposes the task into a clear two-hop search process, leveraging intermediate observations to guide subsequent actions and achieving steady progress toward the correct answer (right) .

Figure 11: Qualitative comparison between SFT- and ProD-distilled Qwen3-4B student agents on a HotpotQA evaluation example. Although the SFT-distilled agent recognizes the failure of its initial search attempt, it remains trapped in a loop of near-duplicate queries without acquiring new information (left) . By contrast, the ProD-distilled agent adapts its search strategy after early unsuccessful attempts, escapes the stagnation loop, and successfully retrieves the required information to solve the task (right) .

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 581 papers

Learning to Act with Task Progress: Distilling Small Agents from Compact Teacher Supervision
Oct 7, 2026Wenxi GanLearning from Demonstration Agent Skill Learning
City University of Hong Kong Shenzhen Loop Area Institute
Harness-Aware Distillation for Small Language Model Agents
Oct 2, 2026Moonseok Choi, Taehong Moon, Giung Nam +1LLM Agent Harnesses On-Policy Distillation
KAIST AI · Independent researcher
Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory
Aug 7, 2026Taeil Kim, Kangsan Kim, Sung Ju HwangLLM Agent Memory LLM Tool Use
KAIST

Oct 7, 2026J/K move · Enter open · S save

Learning to Act with Task Progress: Distilling Small Agents from Compact Teacher Supervision

Wenxi Gan

City University of Hong Kong Shenzhen Loop Area Institute

Learning from Demonstration Agent Skill Learning Knowledge Distillation Process Supervision

Open paper AskPDF arXiv

Learning from large-model demonstrations offers a way to train small agents that can complete recurring tasks without calling a large model at every step. A central design choice is what to retain from teacher trajectories that contain reasoning, actions, and information about task progress. We introduce Task-Progress Distillation (TPD), an offline approach that pairs each demonstrated action with a short label describing the current task stage. The student learns these compact targets and selects actions by jointly scoring admissible stage--action pairs, which a deterministic harness executes in the environment. On ALFWorld, a 1.7B student trained with 404 demonstrations achieves 72.4% mean unseen task success with either TPD or action-only supervision, compared with 48.3% for a reasoning-trained student using constrained action selection. Explicit stages provide an additional benefit at 200 demonstrations, improving success from 48.0% to 67.7% over action-only supervision. With more demonstrations, the action-only student closes the gap, and both approaches reach 76.9% at 808 demonstrations. Shared-history analyses link part of TPD's local advantage to better decisions when moving between subgoals, particularly from object acquisition to processing. These results show that compact supervision can train effective small task agents, while explicit task progress provides additional guidance at an intermediate demonstration budget.

Abstract

LLM Agent Training

Explore similar work

Learning to Act with Task Progress: Distilling Small Agents from Compact Teacher Supervision

Harness-Aware Distillation for Small Language Model Agents

Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory