Policy Distillation

Policy distillation in reinforcement learning aims to transfer knowledge from a complex, often computationally expensive "teacher" policy to a simpler, more efficient "student" policy. Current research focuses on improving sample efficiency, enhancing robustness to imperfect teacher policies, and achieving interpretability through distillation into models like decision trees, gradient boosting machines, or neuro-fuzzy systems. This technique is proving valuable across diverse applications, including robotics (manipulation, locomotion, grasping), finance (portfolio management), and healthcare (drug dosing), by enabling faster training, reduced computational cost, and improved explainability of learned behaviors.

Papers

May 23, 2022

Data augmentation for efficient learning from parametric experts
Alexandre Galashov, Josh Merel, Nicolas Heess
Data Augmentation Imitation Learning Behavior Cloning Efficient Learning Parametric Model Policy Distillation Policy Cloning

May 18, 2022

Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretability
Jinwei Xing, Takashi Nagata, Xinyun Zou, Emre Neftci, Jeffrey L. Krichmar
Deep Reinforcement Learning Saliency Map Better Interpretability Policy Distillation

March 2, 2022

Keeping Minimal Experience to Achieve Efficient Interpretable Policy Distillation
Xiao Liu, Shuyang Liu, Wenbin Li, Shangdong Yang, Yang Gao
Deep Reinforcement Learning Policy Distillation Experience Pool Knowledge Retention Policy Similarity

December 6, 2021

Distilled Domain Randomization
Julien Brosseit, Benedikt Hahner, Fabio Muratore, Michael Gienger, Jan Peters
Reinforcement Learning Deep Reinforcement Learning Domain Randomization Policy Distillation Learned Controller

November 11, 2021

Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation
I-Chun Arthur Liu, Shagun Uppal, Gaurav S. Sukhatme, Joseph J. Lim, Peter Englert, Youngwoon Lee
Reinforcement Learning Motion Planning Behavior Cloning Robot Manipulation Policy Distillation Vision Based Reinforcement Learning Visual Policy End to End Motion Planner

Policy Distillation

Papers

Data augmentation for efficient learning from parametric experts

Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretability

Keeping Minimal Experience to Achieve Efficient Interpretable Policy Distillation

Distilled Domain Randomization

Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation