Policy Distillation

Policy distillation studies how to transfer useful behavior from a teacher policy to a student that may be smaller, less informed, or easier to deploy. Common approaches match teacher actions or probability distributions using supervised learning, KL objectives, on-policy rollouts, offline data, or reinforcement-learning feedback; multi-teacher mixtures, generative students, and interpretable models extend this to diverse skills and compact controllers. The methods target lower inference costs, specialist-skill integration, and transfer from privileged state to robot sensors or smaller language models, with evaluation in settings such as offline control, multi-agent tasks, reasoning benchmarks, and real-robot manipulation and navigation. A central tension is that imitation can copy flawed or inaccessible teacher behavior and lose valid action modes, so students need guidance that matches their deployment information while retaining enough exploration and feedback to generalize or improve on the teacher.