Overview

On-policy distillation studies how to transfer a teacher’s capabilities by training a student on trajectories the student generates, rather than only on fixed teacher-written examples. Common approaches match teacher and student token distributions with KL-style objectives, often combining this dense feedback with verifier rewards, reinforcement learning, or feedback from successful peer attempts. Work spans language-model reasoning and tool-using agents as well as multimodal, speech, diffusion, and embodied models; methods increasingly select useful tokens, turns, or rollout lengths to reduce compute. A central tension is that student-generated states can make supervision more relevant, but teacher signals may be unreliable or mismatched, and repeated rollouts can be costly or destabilize training. 4 sentences total.

Latest papers

CardsList

551 papers88 papers in the last four weeks, down 49% on the four weeks before. 0.6% of all new papers.

  1. Teach with Blinks: Breaking Linguistic Momentum in On-Policy Distillation for Fine-Grained Multimodal Perception

    Nov 12, 2026Tangzheng Lian, Hang Zhang, Guanyu Hu +4VLM DistillationOn-Policy Distillation

  2. Turning Multi-Agent Communication into Token-Level Credit with On Policy Self Distillation

    Nov 12, 2026Zixu Wang, Yu Zhao, Xing Zheng +7RL for Language Model ReasoningMulti-Agent LLMs

  3. Janus-MOPD: Tracing the Student Learning Dynamics in Multi-Teacher On-Policy Distillation

    Nov 11, 2026Zongji Yu, Ming Zhang, Dongcheng Zhang +7Multi-Teacher Knowledge DistillationKnowledge Distillation

  4. Beyond Passive Imitation: On-Policy Mid-Training Distillation for Language Models

    Nov 10, 2026Ruize Zheng, Xuyang Liu, Siyu Ren +5Language Model DistillationLanguage Model Pretraining

  5. Bootstrapped On-Policy Distillation

    Nov 10, 2026Yue Wu, Xueyu Zhou, Yangrong Hu +1Language Model DistillationOn-Policy Distillation

  6. Domain-Routed Delta Optimization for Multi-Teacher On-Policy Distillation

    Nov 9, 2026Lin YaoMulti-Teacher Knowledge DistillationOn-Policy Distillation

  7. Which Successes Make Good References? Action-Sequence Alignment for Multi-Turn On-Policy Distillation

    Nov 9, 2026Mufei Li, Nikki Lijing Kuang, Zhuoer Wang +9On-Policy DistillationTool-Using Language Agents

  8. Agentic-OPD: Recovering Forgotten Agent Capabilities via On-Policy Distillation for Agentic Tasks

    Nov 9, 2026Tingcheng Bian, Kangye Gong, Jinchang Luo +6On-Policy DistillationRL Post-Training

  9. VLA-MOPD: On-Policy Distillation for Multi-Task VLA Policy Consolidation

    Nov 9, 2026Yongxuan Lv, Zhen Fang, Lichen Man +10VLMs for RoboticsOn-Policy Distillation

  10. ECOS: Environment-Calibrated On-Policy Self-Distillation for Language Agents

    Nov 9, 2026Chenhao Zhang, Zhongan Wang, Mingxuan Li +3Self-DistillationOn-Policy Distillation

  11. Robust Domain-Mixture Scheduling for Multi-Teacher On-Policy Distillation

    Nov 9, 2026Yankai Chen, Sida Lin, Yifan Li +10Multi-Teacher Knowledge DistillationOn-Policy Distillation

  12. How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate?

    Nov 8, 2026Cong Yu, Ziting Wang, Songlin Jiang +5Training-Inference MismatchOn-Policy Distillation

  13. Learning Dynamics of On-Policy Distillation

    Nov 5, 2026Guan Zhe Hong, Sumeet Ramesh Motwani, Liang Chen +5On-Policy DistillationLanguage Model Distillation

  14. CoRD: Coupled Rollouts Fill the Blind Spot of On-Policy Distillation

    Nov 5, 2026Guhan Chen, Bohan Li, Jun Rao +8Language Model DistillationKnowledge Distillation

  15. O2^2PD: Overlap-Gated On-Policy Distillation on Reliable Prefix

    Nov 5, 2026Shuaishuai Zu, Jihao Zhao, Pengfei Li +5Language Model DistillationOn-Policy Distillation

  16. Reinforcing What the Student Has Learned: Teacher-Guided On-Policy Distillation

    Nov 5, 2026Bozhen Fang, Linhao Zhong, Xuehao Wang +3Language Model DistillationOn-Policy Distillation

  17. D2D^2-OPD: Separating Direction and Distance in Extrapolative On-Policy Distillation

    Nov 4, 2026Bowen Fang, Dingmin Wang, Hengzhi PeiLanguage Model DistillationOn-Policy Distillation

  18. EvoOPD: Evolving Privileged Information Strategies for Black-Box On-Policy Distillation

    Nov 4, 2026Ximo Zhu, Ruiqi Liu, Yi Han +6Black-Box Knowledge DistillationLanguage Model Distillation

  19. On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation

    Nov 3, 2026Yichen Wang, Yufa Zhou, Siqi Zhu +4Language Model DistillationOn-Policy Distillation

  20. Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning

    Nov 3, 2026Kevin Frans, Alexander M Rush, Sergey Levine +2RL for LLM AgentsAgentic RL

  21. P2^2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors

    Nov 2, 2026Sinuo Wang, Jiacheng Lu, Zeyu Zhang +8Language Model DistillationMulti-Teacher Knowledge Distillation

  22. Meet Students Where They Are: Student-Prefix Teacher Adaptation for On-Policy Distillation

    Nov 2, 2026Xue Zhang, Songming Zhang, Yunlong Liang +3On-Policy DistillationKnowledge Distillation

  23. SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding

    Nov 2, 2026Tianyu Liu, Jun Zhang, Wenjie Huang +12LLM Inference AccelerationOn-Policy Distillation

  24. OmniOPD: Stable On-Policy Distillation for Joint Audio-Video Generation

    Nov 2, 2026Zihao Fan, Xin Lu, Jie Huang +3Audio-Video GenerationOn-Policy Distillation

  25. Order-Only On-Policy Distillation

    Nov 1, 2026Guangzeng Han, Yu Wu, Xiaolei HuangLanguage Model DistillationOn-Policy Distillation

  26. Bernoulli On-Policy Distillation: Don't Ignore High-Confidence Student Tokens

    Nov 1, 2026Mingyu Cao, Yaokai Jin, Shiwei Liu +1Language Model DistillationOn-Policy Distillation