Overview

On-policy distillation studies how to transfer a teacher’s capabilities by training a student on trajectories the student generates, rather than only on fixed teacher-written examples. Common approaches match teacher and student token distributions with KL-style objectives, often combining this dense feedback with verifier rewards, reinforcement learning, or feedback from successful peer attempts. Work spans language-model reasoning and tool-using agents as well as multimodal, speech, diffusion, and embodied models; methods increasingly select useful tokens, turns, or rollout lengths to reduce compute. A central tension is that student-generated states can make supervision more relevant, but teacher signals may be unreliable or mismatched, and repeated rollouts can be costly or destabilize training. 4 sentences total.

Latest papers

CardsList

551 papers88 papers in the last four weeks, down 49% on the four weeks before. 0.6% of all new papers.

  1. Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes

    Mar 26, 2026Yuqian Fu, Haohuan Huang, Kaiwen Jiang +4Language Model DistillationOn-Policy Distillation

  2. Entropy-Aware On-Policy Distillation of Language Models

    Mar 7, 2026Woogyeol Jin, Taywon Min, Yongjin Yang +5Language Model DistillationOn-Policy Distillation

  3. X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

    Mar 6, 2026Di Cao, Dongjie Fu, Hai Yu +3Speech Language ModelsLLM Alignment

  4. OVD: On-policy Verbal Distillation

    Jan 29, 2026Jing Xiong, Hui Shen, Shansan Gong +7Black-Box Knowledge DistillationLanguage Model Distillation