cs.CLNov 2, 2026

Meet Students Where They Are: Student-Prefix Teacher Adaptation for On-Policy Distillation

Authors: Xue Zhang, Songming Zhang, Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu

Organizations: Key Laboratory of Big Data & Artificial Intelligence in Transportation, Beijing Jiaotong University · School of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · Weixin AI, Tencent, China

Abstract

On-policy distillation (OPD) mitigates the student's exposure bias by optimizing on student-generated trajectories. However, student-generated prefixes may contain reasoning errors or follow patterns unfamiliar to the teacher, undermining the reliability of its token-level supervision. This degradation becomes more pronounced as prefixes grow longer. To address this teacher-side distribution mismatch, we propose Student-Prefix Teacher Adaptation (SPTA), which adapts the teacher to the states on which it is queried during distillation. Specifically, before OPD starts, we collect student rollout prefixes with random lengths and continually train the teacher to continue from the prefixes using reinforcement learning with verifiable rewards (RLVR). By treating each prefix as context and rewarding correct final answers, the teacher learns to recover from student-induced states rather than imitate the student's reasoning. Then, the adapted teacher is used for standard OPD, with no changes to the distillation objective. Experiments on mathematical reasoning benchmarks show that our SPTA improves average student performance over standard OPD and OPD with task-level RLVR-adapted teachers. These results highlight the importance of adapting teachers not only to solve problems independently, but also to provide reliable guidance on the states their students actually visit.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. How Should Teachers Be Prepared? RL on Student-Induced States for On-Policy Distillation

    Oct 4, 2026Xiaoyu Ma, Haoyue Liu, Zhichao Wang +2On-Policy DistillationTeacher-Student Learning

  2. ATI-OPD: Adaptive Teacher Intervention for Stable Long-Sequence On-Policy Distillation

    Oct 28, 2026Jiannan Guan, Youxin Jiang, Yutai Hou +9On-Policy DistillationAdaptive Knowledge Distillation