On-Policy Self-Distillation

Overview

On-policy self-distillation studies how a model can improve from its own generated responses, using a teacher version of the model to provide token-level guidance along the student’s actual trajectories. Common approaches give the teacher extra information, such as solutions, critiques, visual evidence, or task feedback, and use filtering, confidence weighting, or reference anchoring to make its guidance more useful. Methods are being applied to language-model reasoning, tool-using agents, multimodal tasks, and generative models, often alongside verifiers or reinforcement-learning methods such as GRPO and RLVR. A central tension is that privileged guidance can correct errors but also encourage shortcuts, suppress exploration, or harm out-of-distribution generalization, so reliable and transferable supervision remains an open challenge.

Latest papers

CardsList

176 papers32 papers in the last four weeks, down 40% on the four weeks before. 0.2% of all new papers.

  1. HINT-WAM: Hindsight-Informed Training for World Action Models

    Nov 18, 2026Phuong Tuan Dat, Qi Li, Xinchao WangWorld Action ModelsRobotic Manipulation

  2. B-OPSD: On-Policy Self-Distillation with Self-Generated Problem Briefs

    Nov 14, 2026Zhi Xu, Yinqi Wu, Feng Zhang +3On-Policy Self-DistillationContext Distillation

  3. Bootstrapped On-Policy Distillation

    Nov 10, 2026Yue Wu, Xueyu Zhou, Yangrong Hu +1Language Model DistillationOn-Policy Distillation

  4. PhysOPSD: Code as Privilege for Physically Plausible Video Generation

    Nov 9, 2026Harold Haodong Chen, Rui Peng, Lingteng Qiu +3Physical Consistency in Video GenerationControllable Video Generation

  5. Watching Itself Watch: Self-Auditing Visual Reliance for Video Reasoning

    Nov 9, 2026Rohan Dugad, Mi Luo, Kristen GraumanVideo ReasoningVisual Evidence Sensitivity in VLMs

  6. Tiny Evidence Region Localization with Selective Contrastive On-Policy Self-Distillation

    Nov 7, 2026Yujuan Pang, Wenhao Xu, Yinglei TengVisual Evidence Sensitivity in VLMsFine-Grained Visual Perception

  7. Route to Remember: Adaptive Memory Selection for LLM Agents

    Nov 3, 2026Zeyu Fang, Jun Yin, Yongsheng Mei +8Memory Management for LLM AgentsMemory-Augmented Language Models

  8. MemRole: From Memories to Decisions for Long-Horizon Reasoning in Role-Playing Agents

    Nov 1, 2026Mengting Li, Xilong Cheng, Zhihao Xiao +6Large Language Model-Based Role-Play SimulationMemory-Augmented Language Model Agents

  9. Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation

    Oct 29, 2026Yifu Luo, Chuang Ma, Zeyu Chen +3On-Policy Self-DistillationLanguage Model Distillation

  10. DTP-OPSD: Reinforcement Learning with Distractor-Aware Temporal-Privilege On-Policy Self-Distillation for Temporal Video Grounding

    Oct 29, 2026Zhaoyu Chen, Yongwei NieTemporal Video GroundingRL for Video Reasoning

  11. On-Policy Self-Distillation with Teacher Evolution

    Oct 28, 2026Shufan Shen, Zhongni Hou, Chao Bi +3On-Policy Self-DistillationLanguage Model Distillation

  12. MASS: Multi-mask Selective Self-Distillation for Leakage-Resilient Reasoning

    Oct 28, 2026Xiuqi Liu, Oveys Delafrooz Noroozi, Upamanyu MadhowLanguage Model DistillationSelective Knowledge Distillation

  13. RPSD: Retrieval-Privileged On-Policy Self-Distillation with Optimal Transport

    Oct 27, 2026Dongyue Li, Zebang Cheng, Zhongyi Deng +3Sequential RecommendationRL for Recommendation

  14. When to Follow the Teacher: Advantage-Based On-Policy Distillation

    Oct 22, 2026Mahdi Sabbaghi, George Pappas, Hamed Hassani +1Language Model DistillationOn-Policy Distillation

  15. One Teacher, Many Views: Reducing Reference Sensitivity in On-Policy Self-Distillation

    Oct 22, 2026Khouloud Saadi, Di WangOn-Policy Self-DistillationKnowledge Distillation

  16. TeachBack: Learning to Teach via Bidirectional Preference Optimization for On-Policy Self-Distillation

    Oct 21, 2026Linfei Ji, Jiayun Li, Haozhao Wang +5Language Model DistillationOn-Policy Self-Distillation

  17. Bridged Self-Distillation for On-Policy Learning

    Oct 19, 2026Runsheng Chen, Yihong Liu, Raoyuan Zhao +3Reasoning DistillationOn-Policy Distillation