Organizations: National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
Deploying a general-purpose model as an agent means running it inside a product-specific runtime harness that decides which actions are executable, how state advances, and when the loop may terminate. Ordinary trajectory supervision shows which action was taken but leaves the runtime semantics behind that action implicit. We introduce Harness Policy Distillation (HPD), which makes these runtime control semantics an explicit training target alongside actions to align the harness policy. HPD replays reference trajectories through the harness to recover the runtime semantics that are implicit in ordinary trajectory data, and represents them as control documents at each model-call boundary. These documents provide privileged information for a teacher to generate actions, while harness verification ensures that the resulting targets are executable under the runtime contract. We jointly train a shared model on the runtime semantics recovered from control-state privileged information and native history, thereby internalizing these semantics into the model policy. At deployment, the trained model uses the original harness interface without requiring the control document or any additional supplement. On TRAJECT-Bench with the Pi harness, Qwen3-8B trained with HPD reaches 22.98% autonomous task success and 49.23% runtime compliance, against 3.09% and 19.90% for the unadapted model. Harness-verified teacher actions substantially improve the native deployment policy, while joint control-state supervision provides further gains in task completion and runtime compliance. HPD turns executable runtime semantics into dense supervision for learning harness-compatible actions at each decision boundary.
Figures & tables
Figure 1: Harness Policy Distillation. Left: agent model–harness policy mismatches. (a) Recorded actions advance replay; AutoProbe compiles control documents; teacher-model proposals are verified and ranked by reference similarity. (b) Three views train one shared model for native action prediction, document-conditioned action prediction, and document reconstruction.
Method
Supervision source
TS
F1
Step-SR
RC
Exact
Action-Acc
Base
—
3.09
26.82
26.45
19.90
2.57
53.41
SFT
Ground truth
2.06
2.40
47.20
55.92
1.72
2.06
DPO
Preference labels
7.20
33.62
11.12
25.56
5.15
72.38
GiGPO
Task reward
4.46
29.65
26.17
21.61
3.43
54.63
GRPO
Task reward
3.43
27.59
25.79
20.41
2.74
54.35
GSPO
Task reward
7.72
37.61
13.46
28.64
6.69
54.30
Table 1: Qwen3-8B comparison on TRAJECT-Bench/Pi. Scores (%) are from pi-runtime ; TS, RC, and Exact use 583 cases. HPD-T and HPD share document-conditioned, verifier-selected teacher targets; HPD-T trains only the Harness-Replay (T) view, while HPD adds I/R. Baselines are detailed in Appendix C .
Figure 2: Zero-shot transfer to BFCL v4 from above checkpoints without specific training.
HPD variant
TS
F1
Step-SR
RC
Exact
Action-Acc
HPD-T
20.58
39.96
59.25
43.74
15.61
61.40
T → I (staged)
18.01
40.23
47.99
34.99
14.24
64.07
T → R (staged)
7.03
17.47
37.29
18.35
5.15
56.64
T+I joint (without R)
20.58
39.66
56.92
41.68
14.92
62.90
T+R joint (without I)
20.58
40.63
63.13
48.03
16.81
62.80
Full HPD (T+I+R joint)
22.98
41.22
63.79
49.23
18.70
64.95
Table 2: Qwen3-8B HPD component and schedule ablation with 20k training batches per active view. T is the replay-action view; I and R are the injected- and runtime-document views. Arrows denote consecutive phases. Scores are percentages from pi-runtime .
Figure 3: Control entry and reference-conditioned continuation. At identical harness states: (a) probability of the reference harness control token; (b) log10 of mean full-vocabulary predictive entropy in bits; (c) geometric mean reference-token probability. Continuation statistics use reference prefixes, exclude the initial control token, and cover at most 63 positions. KDEs share a bandwidth across models within each panel and integrate to one in the plotted coordinate.
Figure 4: TS and F1 across model architectures (left two panels) and separately adapted harnesses (right two panels). All scores are percentages. The Qwen3-8B/Pi Base scores match Table 1 ; its HPD scores use the 20k-optimizer-step checkpoint.
Figure 5: Task success and final-answer F1 for the Qwen3-8B Base model, Pi-Qwen HPD, and API-accessed models in the same Pi agent and evaluation setup. Values are percentages.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Setting or diagnostic
Value
Candidates per sampling attempt
K=2
Attempts to full raw-trajectory coverage
5
Sampling temperature
1
Vocabulary filter
distribution_top_k=0 (full vocabulary)
First-attempt acceptance rate ( K=2 )
87.90%
Cumulative coverage by attempt 5
100%
Appendix
Table 3: Teacher sampling settings and recorded construction diagnostics.
Setting
Value
Accelerators
8× NVIDIA A800-SXM4-80GB (81,920 MiB each)
Host resources
64 CPU cores; 1,030,000 MB RAM
Runtime and precision
CUDA 12.8; BF16 (bfloat16)
Batch size
128
Optimizer
torch.optim.AdamW(foreach=False)
Learning rate
constant 2×10−6
Appendix
Table 4: Training platform and optimization settings shared by HPD and all baseline training runs.
Method
TS
F1
Step-SR
RC
Base
2.68±0.23
26.81±0.11
26.54±0.13
19.62±0.15
HPD-T
18.20±1.67
39.50±0.66
56.21±3.43
42.82±0.81
Full HPD
22.14±1.18
40.79±0.39
60.58±3.54
48.26±0.88
Appendix
Table 5: Supplementary seed-study results from pi-runtime . Values are percentages, reported as mean ± standard deviation over five seeds (715–719), excluding the main-run seed 42. Run-level scores exclude cases interrupted by device crashes. HPD-T retains the full teacher-construction pipeline and trains only the native-action view.
Figure 7: BFCL v4 transfer on 92 cases without retraining. Panels show (a) TS, (b) final-answer F1, and (c) Step-SR for the eight methods. All axes share the same zero-based percentage scale. HPD-T and Full HPD are blue; Full HPD is hatched. Numerical labels give the scores on this subset.
Figure 8: Full HPD optimization over 60k optimizer steps (20k training rounds). Left: 200-cycle mean loss and local 10–90% range. Right: task success and final-answer F1. Base scores are TS 3.09% and F1 26.82%; trained-checkpoint labels are rounded to one decimal.
Optimizer steps
TS
F1
Action-Acc
Step-SR
RC
Exact
40k (intermediate)
19.55
39.79
63.74
58.27
43.40
15.61
60k (completed)
22.98
41.22
64.95
63.79
49.23
18.70
Appendix
Table 6: Intermediate and completed full-HPD checkpoints evaluated by pi-runtime , indexed by optimizer steps. The 60k endpoint completes 20k training rounds. Scores are percentages.
Figure 9: Single-axis sampling-weight search at 20k optimizer steps. Blue circles vary Injected; orange squares vary Runtime. Dashed lines mark the shared 1:1:1 default. Each panel shows one metric with its own y-scale and labels its maximum. Lines connect evaluated settings.
Varied view
T:I:R
TS
F1
Step-SR
RC
Exact
Default
1:1:1
16.12
39.90
52.76
40.65
13.21
Injected (0.2)
5:1:5
15.95
39.18
56.40
43.57
12.52
Injected (0.6)
5:3:5
16.64
39.48
53.93
41.34
13.55
Injected (1.4)
5:7:5
14.92
39.30
51.21
39.11
12.01
Injected (1.8)
5:9:5
16.12
39.90
49.77
39.28
12.86
Runtime (0.2)
5:5:1
17.67
39.33
53.74
40.48
14.07
Appendix
Table 7: Complete single-axis sampling-weight search. Ratios give the realized T:I:R sampling cycle; all scores are percentages. Bold marks the highest score in each metric.
Model
Variant
TS
F1
Action-Acc
Step-SR
RC
Exact
Qwen3-0.6B
Base
0.00
24.78
52.06
3.64
0.00
0.00
HPD
4.46
25.70
54.07
24.20
16.50
2.40
Qwen3-1.7B
Base
0.00
27.00
49.90
4.53
0.00
0.00
HPD
6.52
30.50
59.00
33.60
20.80
3.30
Qwen3-4B
Base
1.03
20.81
53.40
9.81
7.38
0.86
HPD
15.09
38.50
59.40
48.60
38.90
11.50
Appendix
Table 8: Qwen3 model-size results. Action-Acc uses the native gold-history boundary protocol separately from end-to-end evaluation. Exact retains the historical strict trajectory-success definition, not answer-string exact match. All values are percentages.
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · College of Future Technology, Peking University · Google +1