Overview

On-policy distillation studies how to transfer a teacher’s capabilities by training a student on trajectories the student generates, rather than only on fixed teacher-written examples. Common approaches match teacher and student token distributions with KL-style objectives, often combining this dense feedback with verifier rewards, reinforcement learning, or feedback from successful peer attempts. Work spans language-model reasoning and tool-using agents as well as multimodal, speech, diffusion, and embodied models; methods increasingly select useful tokens, turns, or rollout lengths to reduce compute. A central tension is that student-generated states can make supervision more relevant, but teacher signals may be unreliable or mismatched, and repeated rollouts can be costly or destabilize training. 4 sentences total.

Latest papers

CardsList

557 papers70 papers in the last four weeks, down 63% on the four weeks before. 0.5% of all new papers.

  1. Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation

    Oct 20, 2026Guangyu Liu, Siyuan Gan, Shilong Zhou +3Token-Level Loss WeightingLanguage Model Distillation

  2. On-Policy Transition Distillation

    Oct 20, 2026Runyang You, Yongqi Li, Tiezheng Yu +1Language Model DistillationOn-Policy Distillation

  3. SAPD: Learning from Agent Experience without Environment Interaction

    Oct 20, 2026Juncheng Diao, Haibin Wang, Wei Shi +4LLM Agent TrainingLanguage Model Distillation

  4. Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets

    Oct 20, 2026Shripad V. Deshmukh, Christian Bock, Kangrui Ruan +4Language Model DistillationOnline Knowledge Distillation

  5. Ask to Distill: Learning to Synthesize Data for On-Policy Distillation

    Oct 20, 2026Jiashuo Sun, Hao Zhang, Xueqiang Xu +9Language Model DistillationOn-Policy Distillation

  6. A Fisher-Geometric View of Multi-Teacher On-Policy Distillation

    Oct 19, 2026Liu Kaiyuan, Zhuang Ziyuan, Ma Haoxuan +4Language Model MergingMulti-Teacher Knowledge Distillation

  7. Bridged Self-Distillation for On-Policy Learning

    Oct 19, 2026Runsheng Chen, Yihong Liu, Raoyuan Zhao +3Reasoning DistillationOn-Policy Distillation

  8. Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation

    Oct 19, 2026Siyuan Gan, Bo Yang, Linjian Meng +4Language Model DistillationMulti-Teacher Knowledge Distillation

  9. ToolOPD: Credit-Aware On-Policy Distillation for Tool-Use Agents

    Oct 19, 2026Chu Zhao, Lei Tang, Yanming Wang +5On-Policy DistillationStep-Level Credit Assignment in RL

  10. OPDArena: Understanding Learning and Forgetting in On-Policy Distillation

    Oct 19, 2026Yuanze Xu, Tianzhu Ye, Hui Sun +3On-Policy DistillationCatastrophic Forgetting

  11. CD-OPD: Contextual Digest for On-Policy Distillation

    Oct 18, 2026Mingyuan Li, Yucheng Li, Xiaxia Wang +1On-Policy DistillationKnowledge Distillation

  12. Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation

    Oct 18, 2026Jike Zhong, Ming Li, Yuxiang Lai +9On-Policy DistillationAgentic RL

  13. SimOPD: Relational Knowledge Transfer for On-Policy Distillation

    Oct 16, 2026Kai Tang, Xing Zheng, Haobo Wang +6Language Model DistillationOn-Policy Distillation

  14. A Stronger Teacher Within: Strengthening MoE Models via On-Policy Bidirectional Distillation

    Oct 16, 2026Haoyu Chen, Jun Bai, Yang Liu +2Mixture-of-Experts Language ModelsLanguage Model Distillation

  15. GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation

    Oct 16, 2026Yushu Wang, Renkai Yang, Jingcao Xu +4On-Policy DistillationKnowledge Distillation

  16. Training-Free On-Policy Distillation

    Oct 15, 2026Bangji Yang, Jiajun Fan, Zhaoyu Zhu +2On-Policy DistillationKnowledge Distillation

  17. Skill-Decoupled On-Policy Expert Distillation for Unified Vision-Language Models

    Oct 15, 2026Yuqi Xiong, Chunyi Peng, Zhipeng Xu +5VLM DistillationRL for VLMs

  18. Multi-Teacher On-Policy Distillation for Flow-Based Vision-Language-Action Models

    Oct 15, 2026Lichao Ni, Yuhan Liu, Pengfei Li +1Flow-Based VLA ModelsVision-Language-Action Models

  19. OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents

    Oct 15, 2026Yuechen Luo, Zhiyi Lai, Miao Ziqi +9AI Coding AgentsLLM Agent Training

  20. Learning from Interaction Experience via Localized Context Distillation

    Oct 15, 2026Zhezheng Hao, Yongkang Yang, Xilin Xia +9Context DistillationContinual Learning for LLMs

  21. Does On-Policy Distillation Need to Complete Every Rollout?

    Oct 15, 2026Zhengxin Zhang, Maosen Tang, Ning Wang +6On-Policy DistillationLanguage Model Distillation

  22. Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models

    Oct 15, 2026Wenxuan Luo, Chenxi Li, Haoyue Liu +2contrastive knowledge distillationLanguage Model Distillation

  23. Do As I Do, Not As I Say: On-Policy Distillation Inherits the Teacher in Vision-Language Models

    Oct 14, 2026Frank F. Yang, Kefei Yan, Yiren Wang +1VLM DistillationOn-Policy Distillation