Overview

On-policy distillation studies how to transfer a teacher’s capabilities by training a student on trajectories the student generates, rather than only on fixed teacher-written examples. Common approaches match teacher and student token distributions with KL-style objectives, often combining this dense feedback with verifier rewards, reinforcement learning, or feedback from successful peer attempts. Work spans language-model reasoning and tool-using agents as well as multimodal, speech, diffusion, and embodied models; methods increasingly select useful tokens, turns, or rollout lengths to reduce compute. A central tension is that student-generated states can make supervision more relevant, but teacher signals may be unreliable or mismatched, and repeated rollouts can be costly or destabilize training. 4 sentences total.

Latest papers

CardsList

557 papers70 papers in the last four weeks, down 63% on the four weeks before. 0.5% of all new papers.

  1. Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation

    Oct 8, 2026Yilong Yang, Wenzhuo Shang, Yule Liu +2LLM AuditingMembership Inference Attacks

  2. ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery

    Oct 8, 2026Houcheng Jiang, Mao Zheng, Mingyang Song +4LLM PruningLLM Reasoning

  3. Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals

    Oct 8, 2026Lei Zhao, Qichao Zhao, Bowen Zuo +1On-Policy DistillationLanguage Model Distillation

  4. PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation

    Oct 8, 2026Heng Li, Yong Zhang, Ning Cheng +5Few-Shot Text ClassificationAdaptive Knowledge Distillation

  5. Stochastic Teacher Intervention for Agentic On-Policy Distillation

    Oct 7, 2026Junnan Liu, Linhao Luo, Zhijun Chen +3On-Policy DistillationTeacher-Student Learning

  6. Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy Learning

    Oct 7, 2026Yuanda Xu, Hejian Sang, Zhengze Zhou +4Multi-Teacher Knowledge DistillationOn-Policy Distillation

  7. Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models

    Oct 7, 2026Yi Wang, Rui Qian, Yu Li +2Language Model Self-ImprovementOn-Policy Distillation

  8. On-Policy Distillation Teaches New Skills but Not New Knowledge

    Oct 7, 2026Yixuan Tang, Yi YangOn-Policy DistillationKnowledge Distillation

  9. OnlineQAT: On-Policy Distillation for Ultra-Low-Bit Large Language Models

    Oct 7, 2026Wenjun Wang, Heng Li, Yanggan Gu +1LLM QuantizationQuantization

  10. TAP: Efficient Long-Horizon Agent Pruning via Trajectory-Anchored Recovery

    Oct 6, 2026Yuanzhe Li, Pengxin Wang, Yuxin Ren +5LLM PruningLLM Inference Acceleration

  11. GraphOPD: Graph-Augmented On-Policy Distillation for LLM Agents

    Oct 6, 2026Bohan Lin, Liyi Chen, Zhuoning Guo +5LLM Agent TrainingOn-Policy Distillation

  12. UP-MOPD: Update Projection in Multi-Teacher On-Policy Distillation

    Oct 6, 2026Taojie Zhu, Jing Jin, Yuan Xia +8Language Model DistillationGradient Projection

  13. Does On-Policy Distillation for Safety Pose Backdoor Risks?

    Oct 6, 2026Jian Luo, Kehan Qi, Qingqiao Hu +5LLM Backdoor AttacksLLM Security

  14. Learning What to Distill: Bilevel Top-K Token Selection for Self-Distillation in Large Language Models

    Oct 5, 2026Heng Liang, Xinwen Zhang, Hongchang GaoSelective Knowledge DistillationLanguage Model Distillation

  15. Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distribution

    Oct 5, 2026Erfan Baghaei Potraghloo, Seyedarmin Azizi, Arya Fayyazi +4Numerical Reasoning in Language ModelsEfficient Language Model Reasoning

  16. Learning What to Imitate: Entropy-Aware Distribution Mixing

    Oct 5, 2026Juan Garcia Giraldo, Matteo Santelmo, Eduard Durech +3Language Model DistillationUncertainty-Aware Knowledge Distillation

  17. Flash-OPD: Fast On-Policy Distillation

    Oct 5, 2026Wei Chen, Junle Chen, Yitong Yang +6Trajectory GenerationOn-Policy Distillation

  18. Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning Teachers

    Oct 5, 2026Xiaoyu Chen, Bo Shao, Tiangang Zhu +5RL for Language Model ReasoningSelective Knowledge Distillation

  19. ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement Learning

    Oct 5, 2026Seil Kang, Hangoo Kang, Tarun Suresh +4Agentic RLOn-Policy Distillation

  20. Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning

    Oct 5, 2026Chen Henry Wu, Thomas Zhang, Aditi RaghunathanContinual Learning for LLMsOn-Policy Distillation

  21. Beyond In-Distribution Preservation: Recovering Generalization in Quantized VLAs via Vulnerability-Oriented Tuning

    Oct 5, 2026Shen Ruan, Wenchang Gao, Jin Wang +4Vision-Language-Action ModelsOOD Generalization

  22. Towards Unbiased On-Policy Distillation for Block Diffusion Language Models

    Oct 4, 2026Zaiquan Yang, Fei Wei, Yong Wang +6Blockwise Diffusion Language ModelsOn-Policy Distillation

  23. How Should Teachers Be Prepared? RL on Student-Induced States for On-Policy Distillation

    Oct 4, 2026Xiaoyu Ma, Haoyue Liu, Zhichao Wang +2On-Policy DistillationTeacher-Student Learning

  24. ResOPD: Tail Residualization for Sparse On-Policy Distillation

    Oct 4, 2026Penghui Yang, Long Xing, Xuanlang Dai +3Language Model DistillationOn-Policy Distillation

  25. DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation

    Oct 3, 2026Karn Tiwari, Varnith Chordia, Prathosh A POn-Policy DistillationRL for Language Model Reasoning