Harness-Aware Distillation for Small Language Model Agents

Moonseok Choi; Taehong Moon; Giung Nam; Juho Lee

cs.AIOct 2, 2026

Harness-Aware Distillation for Small Language Model Agents

Authors: Moonseok Choi, Taehong Moon, Giung Nam, Juho Lee

Organizations: KAIST AI · Independent researcher

Abstract

Language model agents are deployed with a harness, the software around the model that manages its context, tools, and feedback. When such an agent is distilled into a smaller one, the harness stays in place, so the student mainly needs the teacher-specific abilities that the harness cannot provide, such as acting correctly on harness information. Standard distillation, however, imitates the teacher's full outputs and treats the harness as part of the input. We propose Harness-Aware Distillation (HAD), which focuses distillation on what the teacher adds beyond the harness. HAD complements on-policy distillation with two components: an action preference that contrasts the same teacher's actions with and without the harness information, scored after the student's own reasoning, and a validity check that drops preference pairs whose preferred action contradicts the harness records. We show that the contrast gives the student information that imitating the teacher alone cannot provide, and HAD needs no task rewards, success labels, or future information. Across multiple long-horizon agent benchmarks and models, HAD outperforms on-policy distillation baselines with the same fixed harness. Our analysis shows that HAD enters fewer unproductive loops and recovers from errors more often than the baselines, and suggests that it adaptively keeps learnable feedback in its weights while reading state information from the harness.

Figures & tables

Figure 1: OPD versus HAD on the three failures of OPD. HAD acts on the harness information in the first two rows and drops the invalid teacher target from its preference term in the last.

Figure 2: Harness utilization and performance of distilled students on ALFWorld. Left: Each OPD method distilled without (light) and with (dark) the harness. Right: The zero-shot student, a distilled student trained without and with the harness, and HAD. Adding the harness to distillation improves harness utilization (65.7%

\rightarrow

73.1%) but not performance (43.1%

\rightarrow

43.5%), while HAD improves both (65.7%

\rightarrow

81.0%; 43.1%

\rightarrow

57.4%).

Figure 3: Overview of HAD. (1) The student acts through the harness, which tracks the environment state and monitors the student’s behavior. (2) At each visited state, the same teacher is queried with and without the harness information. (3) The student imitates the teacher’s response given the harness and learns to prefer the action chosen with the harness over the action chosen without it, which gives a signal only when the two actions differ. The validity mask drops pairs whose preferred action contradicts the harness records.

Component	Computed from
State Tracking
Available actions	Environment action list
Situation	Location and inventory
History	Visit history
Budget	Step counter
Behavior Monitoring

Table 1: Harness components.

	ALFWorld (1.7B $\leftarrow$ 8B)		WebShop (0.6B $\leftarrow$ 30B)		ScienceWorld (0.6B $\leftarrow$ 30B)
Method	Unseen SR	Seen SR	Score	SR	Score	SR
Teacher	51.49 / 38.06	50.00 / 37.86	49.96 / 41.41	18.80 / 13.40	$-$ 20.26 / $-$ 18.47	20.13 / 15.44
Student	0 9.70 / 0 2.99	20.71 / 0 6.43	20.95 / 0 4.29	0 9.80 / 0 1.40	$-$ 16.47 / $-$ 16.97	0 3.36 / 0 3.36
OPD	47.01 / 31.34	40.00 / 31.43	41.59 / 39.05	19.00 / 16.60	0 $-$ 0.82 / 0 $-$ 0.64	0 8.05 / 0 7.38
SAGE-OPD	31.34 / 25.37	41.43 / 34.29	42.21 / 39.15	17.00 / 14.00	0 $-$ 0.77 / 0 $-$ 1.41	0 6.04 / 0 6.71
Guided-OPD	46.27 / 33.58	35.71 / 27.14	43.89 / 40.09	16.60 / 14.40	0 $-$ 2.52 / 0 $-$ 4.17	0 6.04 / 0 6.04

Table 2: Main results. Each cell reports performance with harness / without harness . All distillation methods are trained with the harness, and the teacher and the student are evaluated zero-shot. Bold marks the best student in each column.

Figure 4: An unseen ALFWorld task where heating works only on the object in hand. The baselines loop on actions until the step limit, while HAD leaves its loop after the stall report and heats the cup once the harness reports that it holds it.

Table 7

Method	Unseen SR	Seen SR
Teacher	51.49 / 43.28	49.29 / 42.14
Student	14.93 / 8.21	18.57 / 15.71
OPD	26.87 / 18.66	31.43 / 29.29
SAGE-OPD	24.63 / 20.90	28.57 / 27.14
Guided-OPD	29.10 / 18.66	32.86 / 24.29
SOPD	32.09 / 29.85	34.29 / 25.71

Table 5: Gemma-4 results.

Variant	SR	HU
HAD (ours)	57.4	81.0
w/o validity ( $m_{t}\equiv 1$ )	50.8	78.9
w/o negative (pos-only)	52.4	74.4
w/o preference ( $\lambda=0$ )	43.5	73.1

Table 6: Component ablations.

Method	Unseen SR	Seen SR
OPD	47.0	40.0
OPD, 2 $\times$	47.8	45.7
SOPD	45.5	41.4
SOPD, 2 $\times$	50.8	42.9
HAD (ours)	63.4	51.4

Table 7: Teacher budget ablations.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

	ALFWorld	WebShop	ScienceWorld
Teacher	Qwen3-8B	Qwen3-30B-A3B	Qwen3-30B-A3B
Student	Qwen3-1.7B	Qwen3-0.6B	Qwen3-0.6B
Teacher (other family)	Gemma-4-12B	–	–
Student (other family)	Gemma-4-E2B	–	–
Test set	140 seen, 134 unseen	500 goals	149 tasks (30 types)
Training pool	3,553 tasks	11,587 goals	3,592 tasks

Appendix

Table 8: Environments, models, and evaluation settings.

Line	Function	Template (one paraphrase)	When it fires
Situation	State tracking	You are at {location}. You are holding {holdings}. {m} of {n} places visited; {k} unvisited ({names}).	Every step
Visit	State tracking	This visit to {location}: arrived turn {a}, {c} commands executed since. and, for a revisit, Previous visit: arrived turn {a}, {c} commands executed, left turn {l}.	Every step with a known location
Budget	State tracking	Step {t} of {cap}. and the warning Only {left} steps remain. Commit to finishing the task now.	Counter at every step, warning when fewer than $\theta$ steps remain
Failure	Behavior monitoring	’{action}’ did not work here (this was attempt {N}). It will keep failing at this state - choose a different action from the displayed list.	The same action was refused or had no effect at the same state at least the minimum number of times
Stall	Behavior monitoring	Nothing new has been observed for {k} steps. You are not making progress.	No new state was reached in the last $k$ steps

Appendix

Table 9: Harness lines. One of the three paraphrases is shown for each line.

	ALFWorld	WebShop	ScienceWorld
Stall window $k$	5	4	8
Budget threshold $\theta$	11	5	11
Minimum failed attempts	2	1	2

Appendix

Table 10: Harness thresholds for each environment. They are part of the harness shared by all methods and are fixed before distillation on a validation split held out from the training tasks.

	Available actions	Situation	Budget	Failure	Stall	HU
Teacher	90.1	97.2	87.8	79.4	16.9	74.3
Student	73.7	96.2	79.1	61.2	18.1	65.7
OPD	87.8	97.0	88.4	52.9	17.6	68.8
SAGE-OPD	88.0	97.9	83.7	52.3	11.0	66.6
Guided-OPD	82.5	94.1	85.7	28.6	5.3	59.2
SOPD	84.8	96.5	92.3	67.3	24.7	73.1

Appendix

Table 11: Harness utilization and the rate of each component on ALFWorld with the harness.

	ALFWorld	WebShop	ScienceWorld
Optimizer	AdamW, weight decay 0.01, gradient clipping 1.0
Learning rate	$10^{-5}$	$10^{-5}$	$10^{-5}$
Schedule	cosine to 0, 15 warmup updates
Effective batch size	60	60	60
Updates	250	150	150
Tasks per round	16	16	16

Appendix

Table 12: Training hyperparameters, shared by all methods unless noted.

	With harness	Without harness
Teacher	14.10	12.99
Student	13.86	9.77
OPD	13.16	13.45
SAGE-OPD	14.71	12.85
Guided-OPD	14.14	12.23
SOPD	15.15	12.31

Appendix

Table 13: Mean number of turns over successful ALFWorld episodes.

Method	Unseen SR	Seen SR	Turns	HU
Teacher	51.49 / 43.28	49.29 / 42.14	13.91 / 13.93	75.92
Student	14.93 / 8.21	18.57 / 15.71	12.26 / 12.92	71.38
OPD	26.87 / 18.66	31.43 / 29.29	13.26 / 12.97	68.76
SAGE-OPD	24.63 / 20.90	28.57 / 27.14	14.30 / 13.79	67.35
Guided-OPD	29.10 / 18.66	32.86 / 24.29	14.84 / 11.58	64.05
SOPD	32.09 / 29.85	34.29 / 25.71	13.85 / 12.33	69.04

Appendix

Table 14: Full results on ALFWorld with Gemma models (E2B

\leftarrow

12B). SR cells report performance with harness / without harness . Turns (mean over successful episodes) and harness utilization (HU) are measured with the harness.

Table 18

Figure 9: How often the teacher actions with and without the harness differ at each position in the episode.

LLM Agent Harnesses

LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.

See all 357 papers

Harness-Zero: Harness Distillation via Agent-as-Harness
Sep 21, 2026Haoran Ye, Yuxing Lu, Haonan Dong +2LLM Agent Training Knowledge Distillation
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · College of Future Technology, Peking University · Google +1
ActionOPD: Distilling Decisions, Not Tokens
Oct 14, 2026Huining Yuan, Xiangmin Yi, Zelai Xu +5LLM Agent Training Knowledge Distillation
Tsinghua University
Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses
Sep 29, 2026Ziluowen Luo, Senzhang Wang, Chaozhuo Li +9AI Agent Evaluation Knowledge Distillation
Central South University, ChangSha, Hunan, China · Beijing Academy of Artificial Intelligence, Beijing, China · Hong Kong Polytechnic University, Hongkong, China +2

Sep 21, 2026J/K move · Enter open · S save

Harness-Zero: Harness Distillation via Agent-as-Harness

Haoran Ye, Yuxing Lu, Haonan Dong, Zhaochen Su+1

State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · College of Future Technology, Peking University · Google+1

LLM Agent Training Knowledge Distillation LLM Agent Harnesses

Open paper AskPDF arXiv

Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and available information, so guidance from the optimized harness cannot serve directly as supervision for the target one. We introduce Harness-Zero, which enables harness distillation through agent-as-harness. Guided by the optimized harness, a harnessing agent corrects student responses before execution in the target harness's action space, turning harness guidance into training demonstrations. Fine-tuning on the resulting trajectories internalizes harness-induced behavior into the model, so the specialized harness can be removed at deployment. Our experiments spanning knowledge work, tool use, and science domains show that: (1) For frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness. (2) With the specialized harness removed at deployment, Harness-Zero improves the base model's macro-average task success from 23.3% to 44.3%, even exceeding the 41.7% it reaches with that harness still attached. (3) Harness-Zero recovers harness-induced behaviors absent from the base model, with 82.3% average recovery across 28 patterns in the three domains.

Abstract

LLM Agent Harnesses

Explore similar work

Harness-Zero: Harness Distillation via Agent-as-Harness

ActionOPD: Distilling Decisions, Not Tokens

Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses