Overview

On-policy distillation studies how to transfer a teacher’s capabilities by training a student on trajectories the student generates, rather than only on fixed teacher-written examples. Common approaches match teacher and student token distributions with KL-style objectives, often combining this dense feedback with verifier rewards, reinforcement learning, or feedback from successful peer attempts. Work spans language-model reasoning and tool-using agents as well as multimodal, speech, diffusion, and embodied models; methods increasingly select useful tokens, turns, or rollout lengths to reduce compute. A central tension is that student-generated states can make supervision more relevant, but teacher signals may be unreliable or mismatched, and repeated rollouts can be costly or destabilize training. 4 sentences total.

Latest papers

CardsList

551 papers70 papers in the last four weeks, down 63% on the four weeks before. 0.5% of all new papers.

  1. Meet Students Where They Are: Student-Prefix Teacher Adaptation for On-Policy Distillation

    Nov 2, 2026Xue Zhang, Songming Zhang, Yunlong Liang +3On-Policy DistillationKnowledge Distillation

  2. SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding

    Nov 2, 2026Tianyu Liu, Jun Zhang, Wenjie Huang +12LLM Inference AccelerationOn-Policy Distillation

  3. OmniOPD: Stable On-Policy Distillation for Joint Audio-Video Generation

    Nov 2, 2026Zihao Fan, Xin Lu, Jie Huang +3Audio-Video GenerationOn-Policy Distillation

  4. Order-Only On-Policy Distillation

    Nov 1, 2026Guangzeng Han, Yu Wu, Xiaolei HuangLanguage Model DistillationOn-Policy Distillation

  5. Bernoulli On-Policy Distillation: Don't Ignore High-Confidence Student Tokens

    Nov 1, 2026Mingyu Cao, Yaokai Jin, Shiwei Liu +1Language Model DistillationOn-Policy Distillation

  6. Search-Aware On-Policy Self-Distillation with Sibling Alignment for Generative Recommendation

    Nov 1, 2026Rongfeng Guo, Zhuang Zhuang, Ji Dai +6Generative RecommendationSelf-Distillation

  7. DecisionRank: On-Policy Distillation for Decision-Model Rerankers

    Oct 30, 2026Yanran Wu, Dian Yu, Chongyang Bai +10Learning to RankLLM Reranking

  8. Distilling the Shape of Uncertainty: What On-Policy Distillation Learns and How to Predict It

    Oct 28, 2026Shiping Yang, Chuanqi Tang, Hossein Hajimirsadeghi +3Language Model DistillationOn-Policy Distillation

  9. How Much Do Teacher Bridges Matter in Interleaved On-Policy Distillation?

    Oct 28, 2026Leran Chen, Zhiyuan Yao, Lingnan KongLanguage Model DistillationOn-Policy Distillation

  10. ATI-OPD: Adaptive Teacher Intervention for Stable Long-Sequence On-Policy Distillation

    Oct 28, 2026Jiannan Guan, Youxin Jiang, Yutai Hou +9On-Policy DistillationAdaptive Knowledge Distillation

  11. Proposing Better Rollouts for On-Policy Distillation

    Oct 27, 2026Sirui Chen, Yunzhe Qi, Haobo Xu +4Language Model DistillationOn-Policy Distillation

  12. How Much Does the Hint Matter? Scope Conditions for Hindsight Self-Distillation in Financial Reasoning

    Oct 26, 2026Johnny Yu, Cyril GorllaSelf-DistillationKnowledge Distillation

  13. Self-Evolution Reasoner: Continuous Optimization via Policy-Intrinsic Exploration and Exploitation

    Oct 25, 2026Wenhang Shi, Yiren Chen, Zhe Zhao +3Language Model Self-ImprovementRL for Language Model Reasoning

  14. A Distribution-Matching Perspective on LLM Training Algorithms

    Oct 24, 2026Yanxing Huang, Bingzhang Yan, Zihan Tang +3LLM TrainingLanguage Model Distillation

  15. TAD: Distilling Vision-Language-Action Policies from Teacher Trajectories Alone

    Oct 24, 2026Yun Zhu, Da WangVisuomotor Policy LearningKnowledge Distillation

  16. When to Follow the Teacher: Advantage-Based On-Policy Distillation

    Oct 22, 2026Mahdi Sabbaghi, George Pappas, Hamed Hassani +1Language Model DistillationOn-Policy Distillation

  17. Tracing Subliminal Learning Through Pretraining

    Oct 22, 2026Angela Tang, Vishesh Anand, Gonçalo PauloLanguage Model DistillationOn-Policy Distillation

  18. Reverse-KL Policy Distillation: Statistical Guarantees and Limits

    Oct 22, 2026Haolin Liu, Chen-Yu Wei, Julian ZimmertLanguage Model DistillationKnowledge Distillation

  19. Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning

    Oct 21, 2026Wen Ye, Ailin Deng, Zirui Wang +8Reasoning for VLMsVision-Language Models

  20. Asymmetric On-Policy Distillation Learns In-Context Exploration

    Oct 21, 2026Sriyash Poddar, Yanda Bao, Jacob Krantz +5RL ExplorationPartially Observable RL