Policy Distillation

Policy distillation in reinforcement learning aims to transfer knowledge from a complex, often computationally expensive "teacher" policy to a simpler, more efficient "student" policy. Current research focuses on improving sample efficiency, enhancing robustness to imperfect teacher policies, and achieving interpretability through distillation into models like decision trees, gradient boosting machines, or neuro-fuzzy systems. This technique is proving valuable across diverse applications, including robotics (manipulation, locomotion, grasping), finance (portfolio management), and healthcare (drug dosing), by enabling faster training, reduced computational cost, and improved explainability of learned behaviors.

Papers

October 31, 2024

Language-Driven Policy Distillation for Cooperative Driving in Multi-Agent Reinforcement Learning
Jiaqi Liu, Chengkai Xu, Peng Hang, Jian Sun, Mingyu Ding, Wei Zhan, Masayoshi Tomizuka
Autonomous Vehicle Multi Agent Reinforcement Learning Cooperative Driving Cooperative Agent Policy Distillation Collaborative Decision

July 21, 2024

Proximal Policy Distillation
Giacomo Spigler
Policy Distillation

June 8, 2024

Online Policy Distillation with Decision-Attention
Xinqiang Yu, Chuanguang Yang, Chengqing Yu, Libo Huang, Zhulin An, Yongjun Xu
Reinforcement Learning Task Online Knowledge Distillation Policy Distillation

May 8, 2024

Markowitz Meets Bellman: Knowledge-distilled Reinforcement Learning for Portfolio Management
Gang Hu, Ming Gu
Knowledge Distillation Portfolio Management Policy Distillation Bellman Operator Portfolio Performance

April 26, 2024

An Explainable Deep Reinforcement Learning Model for Warfarin Maintenance Dosing Using Policy Distillation and Action Forging
Sadjad Anzabi Zadeh, W. Nick Street, Barrett W. Thomas
Deep Reinforcement Learning Proximal Policy Optimization Action Feature Policy Distillation Dosing Model Explainable Deep Reinforcement Learning Warfarin Dosing

April 5, 2024

Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation
Lanpei Li, Enrico Donato, Vincenzo Lomonaco, Egidio Falotico
Reinforcement Learning Robotic Hand Hand Manipulation Policy Distillation Soft Robotic Hand

March 21, 2024

Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression
Fernando Acero, Zhibin Li
Reinforcement Learning Symbolic Regression Gradient Boosting Robot Locomotion Policy Distillation Reinforcement Learning Policy Neural Network Policy

March 15, 2024

Grasp Anything: Combining Teacher-Augmented Policy Gradient Learning with Instance Segmentation to Grasp Arbitrary Objects
Malte Mosbach, Sven Behnke
Reinforcement Learning Instance Segmentation Robot Learning Grasp Anything Object Grasping Policy Distillation

February 9, 2024

Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control
Zheng Xiong, Risto Vuorio, Jacob Beck, Matthieu Zimmer, Kun Shao, Shimon Whiteson
Zero Shot Policy Adaptation Policy Distillation Reinforcement Learning Policy Conditioned Hypernetwork Multi Morphology Controller

November 22, 2023

Curriculum Learning and Imitation Learning for Model-free Control on Financial Time-series
Woosung Koh, Insu Choi, Yuntae Jang, Gimin Kang, Woo Chang Kim
Imitation Learning Curriculum Learning Model Free Financial Time Series Policy Distillation Time Varying Control

June 23, 2023

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, Olivier Bachem
Language Model Knowledge Distillation Policy Distillation Task Agnostic Distillation

June 21, 2023

One Policy to Dress Them All: Learning to Dress People with Diverse Poses and Garments
Yufei Wang, Zhanyi Sun, Zackory Erickson, David Held
Robotics Domain Human Pose Person Name Health Policy Recommendation Policy Distillation Robot Assisted Dressing

May 25, 2023

Counterfactual Explainer Framework for Deep Reinforcement Learning Models Using Policy Distillation
Amir Samadi, Konstantinos Koufos, Kurt Debattista, Mehrdad Dianati
Deep Reinforcement Learning Counterfactual Explanation Policy Distillation DRL Algorithm

February 1, 2023

Distillation Policy Optimization
Jianfei Ma
Policy Reinforcement Learning Policy Algorithm Policy Distillation Policy Data

September 20, 2022

Soft Action Priors: Towards Robust Policy Transfer
Matheus Centa, Philippe Preux
Reinforcement Learning Robust Policy Policy Distillation Action Prior

September 19, 2022

MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation
Aaron M. Roth, Jing Liang, Ram Sriram, Elham Tabassi, Dinesh Manocha
Reinforcement Learning Deep Reinforcement Learning Robot Navigation Policy Distillation

September 15, 2022

Human-level Atari 200x faster
Steven Kapturowski, Víctor Campos, Ray Jiang, Nemanja Rakićević, Hado van Hasselt, Charles Blundell, Adrià Puigdomènech Badia
Reinforcement Learning Agent Smith Deeper Network Policy Distillation ATARI Game

September 7, 2022

Distilling Deep RL Models Into Interpretable Neuro-Fuzzy Systems
Arne Gevaert, Jonathan Peck, Yvan Saeys
Deep Reinforcement Learning Deep Q Network Fuzzy Logic DQN Agent Policy Distillation Neuro Fuzzy

July 29, 2022

Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomization
Yuki Kadokawa, Lingwei Zhu, Yoshihisa Tsurumine, Takamitsu Matsubara
Deep Reinforcement Learning Sample Efficient Domain Randomization Policy Distillation Sim to Real Reinforcement Learning

June 17, 2022

MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare
Yiqiang Chen, Wang Lu, Xin Qin, Jindong Wang, Xing Xie
Novel Framework Policy Distillation Personalized Healthcare Knowledge Accumulation

Policy Distillation

Papers

Language-Driven Policy Distillation for Cooperative Driving in Multi-Agent Reinforcement Learning

Proximal Policy Distillation

Online Policy Distillation with Decision-Attention

Markowitz Meets Bellman: Knowledge-distilled Reinforcement Learning for Portfolio Management

An Explainable Deep Reinforcement Learning Model for Warfarin Maintenance Dosing Using Policy Distillation and Action Forging

Continual Policy Distillation of Reinforcement Learning-based Controllers for Soft Robotic In-Hand Manipulation

Distilling Reinforcement Learning Policies for Interpretable Robot Locomotion: Gradient Boosting Machines and Symbolic Regression

Grasp Anything: Combining Teacher-Augmented Policy Gradient Learning with Instance Segmentation to Grasp Arbitrary Objects

Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control

Curriculum Learning and Imitation Learning for Model-free Control on Financial Time-series

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes

One Policy to Dress Them All: Learning to Dress People with Diverse Poses and Garments

Counterfactual Explainer Framework for Deep Reinforcement Learning Models Using Policy Distillation

Distillation Policy Optimization

Soft Action Priors: Towards Robust Policy Transfer

MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation

Human-level Atari 200x faster

Distilling Deep RL Models Into Interpretable Neuro-Fuzzy Systems

Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomization

MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare