Overview

On-policy distillation studies how to transfer a teacher’s capabilities by training a student on trajectories the student generates, rather than only on fixed teacher-written examples. Common approaches match teacher and student token distributions with KL-style objectives, often combining this dense feedback with verifier rewards, reinforcement learning, or feedback from successful peer attempts. Work spans language-model reasoning and tool-using agents as well as multimodal, speech, diffusion, and embodied models; methods increasingly select useful tokens, turns, or rollout lengths to reduce compute. A central tension is that student-generated states can make supervision more relevant, but teacher signals may be unreliable or mismatched, and repeated rollouts can be costly or destabilize training. 4 sentences total.

Latest papers

CardsList

557 papers70 papers in the last four weeks, down 63% on the four weeks before. 0.5% of all new papers.

  1. On-Policy Distillation of Language Models for Autonomous Vehicle Motion Planning

    Apr 9, 2026Amirhossein Afsharrad, Amirhesam Abedsoltan, Ahmadreza Moradipari +1Language Model DistillationAutonomous Driving Planning

  2. DP-OPD: Differentially Private On-Policy Distillation for Language Models

    Apr 6, 2026Fatemeh Khadem, Sajad Mousavi, Yi Fang +1Privacy-Preserving Language ModelsPrivacy-Preserving ML

  3. SODA: Semi On-Policy Black-Box Distillation for Large Language Models

    Apr 4, 2026Xiwen Chen, Jingjing Wang, Wenhui Zhu +7Black-Box Knowledge DistillationLanguage Model Distillation

  4. Self-Distilled RLVR

    Apr 3, 2026Chenxu Yang, Chuanyu Qin, Qingyi Si +7On-Policy Self-DistillationReinforcement Learning with Verifiable Rewards

  5. A Survey of On-Policy Distillation for Large Language Models

    Apr 1, 2026Mingyang Song, Mao ZhengExposure BiasLanguage Model Distillation

  6. Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes

    Mar 26, 2026Yuqian Fu, Haohuan Huang, Kaiwen Jiang +4Language Model DistillationOn-Policy Distillation

  7. Entropy-Aware On-Policy Distillation of Language Models

    Mar 7, 2026Woogyeol Jin, Taywon Min, Yongjin Yang +5Language Model DistillationOn-Policy Distillation

  8. X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

    Mar 6, 2026Di Cao, Dongjie Fu, Hai Yu +3Speech Language ModelsLLM Alignment

  9. OVD: On-policy Verbal Distillation

    Jan 29, 2026Jing Xiong, Hui Shen, Shansan Gong +7Black-Box Knowledge DistillationLanguage Model Distillation