On-Policy Self-Distillation

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 233% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 108

All topics

Geometric Self-Distillation for Reasoning Generalization
Jul 7, 2026Josip Jukić, Ivan TitovOn-Policy Self-Distillation OOD Generalization
ILLC, University of Amsterdam · ILCC, University of Edinburgh
Rethinking On-Policy Self-Distillation for Thinking Models
Jul 6, 2026Simran Kaur, Narutatsu Ri, Yinghui He +2LLM Self-Correction On-Policy Self-Distillation
Princeton Language and Intelligence, Princeton University
dOPSD: On-Policy Self-Distillation for Diffusion Language Models
Jul 5, 2026Phuong Tuan Dat, Qi Li, Xinchao WangOn-Policy Self-Distillation Language Model Distillation
National University of Singapore
DemoPSD: Disagreement-Modulated Policy Self-Distillation
Jul 2, 2026Yunhe Li, Hao Shi, Wenhao Liu +5On-Policy Self-Distillation RL for Language Model Reasoning
1City University of Hong Kong · 2Tsinghua University · 3Shenzhen University of Advanced Technology +1
Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation
Jul 2, 2026Zhuowei Chen, Xiang Lorraine LiOn-Policy Self-Distillation Training Data Selection
University of Pittsburgh, United States
Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
Jul 2, 2026Zhanming Shen, Jintao Tong, Shaotian Yan +9CoT Distillation On-Policy Self-Distillation
1Zhejiang University · 2Tongyi Lab, Alibaba Group · 3Huazhong University of Science and Technology +1
Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training
Jul 2, 2026Meng Wang, Haohan Zhao, Wenzhuo Liu +7Continual Learning for LLMs On-Policy Self-Distillation
Centre for Artificial Intelligence and Robotics, HKISI, CAS · Institute of Automation, CAS · University of Chinese Academy of Sciences +1
DOPD: Dual On-policy Distillation
Jun 29, 2026Xinlei Yu, Gen Li, Qingyi Si +13On-Policy Self-Distillation VLM Distillation
NUS · MMLab, CUHK · Explore Academy, JD +1
PHF: Privileged Hidden Flow for On-Policy Self-Distillation
Jun 28, 2026Yuhan Li, Mingxu Zhang, Dazhong Shen +1On-Policy Self-Distillation Language Model Distillation
1The Hong Kong University of Science and Technology (Guangzhou) · 2Nanjing University of Aeronautics and Astronautics · 3The 63rd Research Institute, National University of Defense Technology, Nanjing
On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity
Jun 24, 2026Andrei Liviu Nicolicioiu, Mohammad Pezeshki, Aaron CourvilleOn-Policy Self-Distillation Language Model Distillation
Mila · Université de Montréal · FAIR at Meta +1
Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation
Jun 17, 2026Sihan Wang, Xiyao Liu, Lianqing Liu +1Vision-Language Models On-Policy Self-Distillation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences
Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
Jun 17, 2026Pengyu Li, Zhitao Gao, Lingling Zhang +5Cross-Modal Knowledge Distillation Efficient Multimodal Inference
Xi’an Jiaotong University · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · MOE KLINNS Lab, Xi’an Jiaotong University +1
Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillation
Jun 17, 2026Zhilin Huang, Hang Gao, Ziqiang Dong +6LLM Self-Correction CoT Distillation
Qwen Business Unit of Alibaba · Tsinghua University · Peking University
Learning from the Self-future: On-policy Self-distillation for dLLMs
Jun 16, 2026Yifu Luo, Zeyu Chen, Haoyu Wang +4On-Policy Self-Distillation Diffusion Language Models
Tsinghua University · Technical University of Munich · Nanyang Technological University +5
Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding
Jun 16, 2026Jingyuan Huang, Zuming Huang, Yucheng Shi +4On-Policy Self-Distillation GUI Grounding
University of Georgia · 2INFLY Tech · *This work was done during an internship at INFLY Tech. +2
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
Jun 14, 2026Zebang Cheng, Shuimu Chen, Boxue Yang +7On-Policy Self-Distillation Multimodal Large Language Models
Shenzhen University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Tsinghua University +6
ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models
Jun 9, 2026Yuxiang Wang, Qinke Ni, Shengbo Cai +3On-Policy Self-Distillation Spoken Dialogue Systems
1The Chinese University of Hong Kong, Shenzhen · 2Tencent Hunyuan · 5Tsinghua University +2
Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation
Jun 7, 2026Yishuo Cai, Jiahui Liu, Yuanxin Liu +9Tool Use in VLMs Efficient VLM Inference
1State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · 2Central South University · University of Science and Technology of China +2
Physics-Guided Policy Optimization with Self-Distillation
Jun 2, 2026Ke Wang, Yuning Wu, Haoran Liu +3On-Policy Self-Distillation Policy Optimization
Amazon, Arlington, USA · Amazon, New York, USA · Amazon, San Jose, USA +1
World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning
Jun 2, 2026Yucheng Zhou, Wei Tao, Yiwen Guo +1On-Policy Self-Distillation World Models
University of Macau · LIGHTSPEED · Independent Researcher
When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation
Jun 2, 2026Haowei Guo, Baolong Bi, Ruicheng Zhang +2On-Policy Self-Distillation Language Model Distillation
Peking University · University of Chinese Academy of Sciences · Tsinghua University
Constitutional On-Policy Safe Distillation
Jun 2, 2026Ming Wen, Yuxuan Liu, Kun Yang +8On-Policy Self-Distillation LLM Safety Alignment
Institute of Trustworthy Embodied AI, Fudan University · 2Shanghai Innovation Institute · 3Ant Group +3
Self-Distilled Policy Gradient
Jun 2, 2026Yifeng Liu, Shiyuan Zhang, Yifan Zhang +1Policy Gradient On-Policy Self-Distillation
Department of Computer Science, University of California, Los Angeles, CA, USA · Princeton AI Laboratory, Princeton University, Princeton, NJ, USA
Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning
May 30, 2026Xuewei Yang, Jiachen Yu, Jie Wu +3On-Policy Self-Distillation RL for Language Model Reasoning
Tsinghua University
A Predictive Law for On-Policy Self-Distillation From World Feedback
May 28, 2026Tommy He, Jerome Sieber, Matteo SaponatiOn-Policy Self-Distillation RL Post-Training
Tufa Labs, Zürich, Switzerland
Beyond Imitation: Reflective On-Policy Self-Distillation for LLM Reasoning
May 27, 2026Ziqi Zhao, Xinyu Ma, Liu Yang +6On-Policy Self-Distillation Language Model Distillation
The Hong Kong Polytechnic University · Baidu Inc. · Shandong University +1
Restoring the Sweet Spot: Pass-Rate Weighted Self-Distillation for LLM Reasoning
May 26, 2026Zehao Liu, Yuanpu Cao, Jinghui Chen +1Reinforcement Learning On-Policy Self-Distillation
College of Information Sciences and Technology Pennsylvania State University
EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation
May 22, 2026Aristotelis Lazaridis, Dylan Bates, Aman Sharma +3On-Policy Self-Distillation On-Policy Distillation
EdgeRunner AI
Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning
May 21, 2026Zihan Liang, Yufei Ma, Ben Chen +4Tool-Augmented Language Model Agents On-Policy Self-Distillation
Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning
May 21, 2026Hongbin Zhang, Chaozheng Wang, Kehai Chen +4On-Policy Self-Distillation Mathematical Reasoning
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Keeta AI, Meituan, Beijing, China · Peng Cheng Laboratory, Shenzhen, China

Jul 7, 2026, cs.LGJ/K move · Enter open · S save

Geometric Self-Distillation for Reasoning Generalization

Josip Jukić, Ivan Titov

ILLC, University of Amsterdam · ILCC, University of Edinburgh

On-Policy Self-Distillation OOD Generalization Self-Distillation

Open paper AskPDF arXiv

On-policy distillation provides dense teacher supervision on a language model's own trajectories. In self-distillation with privileged context, this supervision comes from the model itself, conditioned on a hint or solution trace hidden from the student. When the teacher's preferences hinge on privileged information, it can assign higher probability to continuations the student cannot infer from its own context. Matching these preferences throughout training can induce predictive drift and degrade out-of-distribution (OOD) reasoning. We propose GeoSD, a self-distillation method that controls this drift through two complementary geometric terms. A Hellinger loss weights each teacher preference by the student--teacher overlap, reducing the influence of tokens to which the student assigns low probability. Because these influences can still accumulate, a Fisher--Rao penalty regulates predictive distance from a copy of the student refreshed periodically during training. Both terms compare next-token distributions in Fisher--Rao geometry and are jointly optimized with a preconditioner motivated by the natural gradient. Across three model families, GeoSD retains strong in-distribution gains while improving average mathematical OOD accuracy by 5.7--8.6 points over the base model. OOD gains hold across five model scales from 1.7B to 32B and transfer to code generation, where GeoSD improves code accuracy by 1.9 points on average despite distilling on mathematics alone. Our analysis of mathematical reasoning shows that standard matching rapidly concentrates probability mass at high-entropy states and that its samples confidently agree on incorrect answers. In contrast, GeoSD preserves alternative token mass and reduces false consensus.

Momentum

Related topics

Latest papers 108