cs.AINov 16, 2026

SWE-OPD: On-Policy Distillation for Agentic Coding

Authors: Yuhang He, Yanyu Chen, Chuanzhe Guo, Jie Zhu, Zichen Tian, Hongyu Ge, Zixuan Ren, Huang He, 

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong · Harbin Institute of Technology · ERNIE Team, Baidu

Abstract

In this work, we study on-policy distillation (OPD) for transferring agentic coding capabilities from a large teacher model to a smaller student model. Prior work has focused primarily on reasoning and short-horizon agentic tasks. We present SWE-OPD, a minimal OPD study for software engineering, where the student interacts with executable repositories and the teacher supervises the prefixes visited by the student. Across three teacher--student pairs, two training sources, and three SWE benchmarks, we find that on-policy supervision transfers coding capability: supervision from Qwen3.6-27B increases the SWE-bench Pro resolve rate of Qwen3.5-9B from 35.9% to 39.4%. Refreshing trajectories is not strictly required either, as an offline variant with cached teacher scores still improves over the student baselines while cutting end-to-end GPU-hours by 59.4%--72.6% and halving peak GPU allocation, at benchmark-dependent accuracy. Furthermore, we extend the offline variant to corss-tokenizer scenarios: distilling GLM-4.5-Air from DeepSeek-V4-Flash-0731 raises its SWE-bench Pro and Multilingual resolve rate from 35.8% to 39.7% and from 41.8% to 46.2%, around 4% and 4.6% improvement, respectively. We also find that a decreasing loss, however, does not necessarily indicate improved agent performance: under top-KK reverse-KL loss, one student exhibits severe repetition despite a decreasing loss. We identify a support-mass gradient term as a potential mechanism underlying this failure.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents

    Oct 15, 2026Yuechen Luo, Zhiyi Lai, Miao Ziqi +9AI Coding AgentsLLM Agent Training

  2. ActionOPD: Distilling Decisions, Not Tokens

    Oct 14, 2026Huining Yuan, Xiangmin Yi, Zelai Xu +5LLM Agent TrainingKnowledge Distillation

  3. Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation

    Oct 18, 2026Jike Zhong, Ming Li, Yuxiang Lai +9On-Policy DistillationAgentic RL