Organizations: The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong · Harbin Institute of Technology · ERNIE Team, Baidu
In this work, we study on-policy distillation (OPD) for transferring agentic coding capabilities from a large teacher model to a smaller student model. Prior work has focused primarily on reasoning and short-horizon agentic tasks. We present SWE-OPD, a minimal OPD study for software engineering, where the student interacts with executable repositories and the teacher supervises the prefixes visited by the student. Across three teacher--student pairs, two training sources, and three SWE benchmarks, we find that on-policy supervision transfers coding capability: supervision from Qwen3.6-27B increases the SWE-bench Pro resolve rate of Qwen3.5-9B from 35.9% to 39.4%. Refreshing trajectories is not strictly required either, as an offline variant with cached teacher scores still improves over the student baselines while cutting end-to-end GPU-hours by 59.4%--72.6% and halving peak GPU allocation, at benchmark-dependent accuracy. Furthermore, we extend the offline variant to corss-tokenizer scenarios: distilling GLM-4.5-Air from DeepSeek-V4-Flash-0731 raises its SWE-bench Pro and Multilingual resolve rate from 35.8% to 39.7% and from 41.8% to 46.2%, around 4% and 4.6% improvement, respectively. We also find that a decreasing loss, however, does not necessarily indicate improved agent performance: under top-K reverse-KL loss, one student exhibits severe repetition despite a decreasing loss. We identify a support-mass gradient term as a potential mechanism underlying this failure.
Figures & tables
Figure 1: Study design of SWE-OPD . Left: the student interacts with an executable repository; tool observations enter the conditioning context, while only student-generated tokens receive supervision. Middle & Right: The two design axes we compare are the trajectory source (current student in SWE-OPD vs. initial student in Offline SWE-OPD) and the teacher information used per position (top- K candidates in the RKL loss vs. the sampled token in the ST loss objective).
Method
Verified
Pro
Multilingual
Score
Δ
Score
Δ
Score
Δ
Student: Qwen3.5-9B Teacher: Qwen3.6-27B
Student baseline
57.9
35.9
43.0
Teacher Rollout Distillation
54.8
−3.1
34.5
−1.4
40.5
−2.5
RL (UniAgent ( Ding et al., 2026 ) )
59.2
+1.3
–
–
SWE-OPD (RKL)
60.5
+2.6
39.4
+3.5
45.8
+2.8
Table 1: Results after training on 5K ScaleSWE queries. Avg@4 is the mean resolve rate (%) over four independent runs on SWE-bench Verified ( Jimenez et al., 2024 ) , Pro ( Deng et al., 2025 ) , and Multilingual ( Zan et al., 2025 ) . Δ is the change from the student baseline in percentage points; bold marks the best score per benchmark. Subsequent benchmark tables use these conventions within each student--teacher pair. 3 3 3 The experimental setup, training configurations, and agent scaffold are detailed in Appendix A .
Method
Verified
Pro
Multilingual
Score
Δ
Score
Δ
Score
Δ
Student: Qwen3.5-9B Teacher: Qwen3.6-27B
Student baseline
57.9
35.9
43.0
SWE-OPD (RKL)
61.6
+3.7
38.4
+2.5
44.3
+1.3
SWE-OPD (ST)
61.3
+3.4
38.7
+2.8
43.1
+0.1
Student: GLM-4.5-Air Teacher: GLM-5.2
Table 2: Results on the internal dataset. We evaluate SWE-OPD with Qwen3.5-9B and GLM-4.5-Air as students, supervised by Qwen3.6-27B and GLM-5.2, respectively.
Figure 2: Top- K ablation. Gains in percentage points.
Figure 3: Top- K selection and nested support diagnostics. Qwen3.5-9B distilled from Qwen3.6-27B: (a,b) support-size sweep; (c,d) nested readouts for the K=128 ScaleSWE run. All horizontal axes show training steps. In (b), mean overlap over steps 62–71 is 62.5% at K=128 and 55.0% at K=1024 .
Method
Verified
Pro
Multilingual
Score
Δ
Score
Δ
Score
Δ
Student: Qwen3.5-9B Teacher: Qwen3.6-27B
Student baseline
57.9
35.9
43.0
SWE-OPD (RKL)
61.6
+3.7
38.4
+2.5
44.3
+1.3
SWE-OPD (ST)
61.3
+3.4
38.7
+2.8
43.1
+0.1
Offline SWE-OPD (RKL)
60.2
+2.3
38.0
+2.1
45.3
+2.3
Table 3: Results of SWE-OPD and Offline SWE-OPD on the internal dataset. We compare both variants for Qwen3.5-9B distilled from Qwen3.6-27B and GLM-4.5-Air distilled from GLM-5.2, and additionally evaluate Offline SWE-OPD (ST) for GLM-4.5-Air distilled from DeepSeek-V4-Flash-0731, a teacher with a different tokenizer.
Method
Qwen3.5-9B
GLM-4.5-Air
SWE-OPD
1024
2808
Offline SWE-OPD
416
768
GPU-hour reduction
2.46 ×
3.65 ×
Offline SWE-OPD breakdown
Student rollout
176
368
Teacher scoring
48
160
Table 4: Training cost on the internal dataset (GPU-hours) for our online and offline implementations.
Figure 4: GLM training dynamics: ( 4 ) Repetition fraction; ( 4 ) student entropy; ( 4 ) training loss. RKL and ST training are shown in orange and blue, respectively.
Figure 5: teacher-student probability divergence.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Training data
Model pair
Training variant
Loss
K
Kcache
ScaleSWE (5K)
Qwen
Teacher rollouts
SFT
–
–
Qwen
Online
RKL
128
–
Qwen
Online
ST
–
–
Qwen
Online (ablation)
RKL
256/512/1024
–
Internal ( ∼ 4K)
Qwen
Online
RKL
128
–
Qwen
Online
ST
–
–
Appendix
Table 5: Training configurations. Qwen denotes Qwen3.5-9B ← Qwen3.6-27B, GLM denotes GLM-4.5-Air ← GLM-5.2, and GLM/DS denotes GLM-4.5-Air ← DeepSeek-V4-Flash-0731, with student ← teacher. K is the RKL student-support size; Kcache is the offline teacher-cache size. A dash denotes a setting that does not apply.
Figure 6: Constructed comparison of independent-logit updates. Upper panels show initial probabilities (gray) and final probabilities (arrow tips) of two retained tokens and one omitted token. Lower panels track total outside-support mass; dashed lines mark the teacher. ST updates use the exact expected gradient.