Diffusion Policy

Diffusion policies leverage the power of diffusion models to generate actions for complex tasks, primarily aiming to improve the robustness, efficiency, and generalization capabilities of reinforcement learning agents and robotic controllers. Current research focuses on refining algorithms like Diffusion Policy Policy Optimization (DPPO) and exploring architectures such as Mixture of Experts (MoE) to enhance multi-task learning and reduce computational costs, often incorporating techniques from reinforcement learning and imitation learning. This approach holds significant promise for advancing robotics, particularly in areas like manipulation, locomotion, and navigation, by enabling more adaptable and data-efficient learning of complex behaviors.

Papers

February 25, 2024

Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
Kaiqi Chen, Eugene Lim, Kelvin Lin, Yiyang Chen, Harold Soh
Diffusion Model Imitation Learning Scratch Project Diffusion Policy

February 13, 2024

Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases
Ziyi Zhang, Sen Zhang, Yibing Zhan, Yong Luo, Yonggang Wen, Dacheng Tao
Diffusion Model Visual Perspective Reward Model Diffusion Policy Downstream Model Reward Overoptimization Inductive Task Primacy Bias

February 6, 2024

Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning
Ruoqi Zhang, Ziwei Luo, Jens Sjölund, Thomas B. Schön, Per Mattsson
Offline Reinforcement Learning Diffusion Policy Entropy Regularized Reverse Time Q Ensemble

January 17, 2024

DiffClone: Enhanced Behaviour Cloning in Robotics with Diffusion-Driven Policy Learning
Sabariswaran Mani, Sreyas Venkataraman, Abhranil Chandra, Adyan Rizvi, Yash Sirvi, Soumojit Bhattacharya, Aritra Hazra
Robotics Domain Robot Learning Behavior Cloning Diffusion Policy Cloning Agent

December 13, 2023

World Models via Policy-Guided Trajectory Diffusion
Marc Rigter, Jun Yamada, Ingmar Posner
World Model Policy Reinforcement Learning Diffusion Policy Synthetic Trajectory

December 8, 2023

RL Dreams: Policy Gradient Optimization for Score Distillation based 3D Generation
Aradhya N. Mathur, Phu Pham, Aniket Bera, Ojaswa Sharma
Generative Modeling Policy Gradient 3D Generation Diffusion Policy Score Distillation

November 22, 2023

Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model
Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Qimai Li, Weihan Shen, Xiaolong Zhu, Xiu Li
Diffusion Model Human Feedback Reward Model Diffusion Policy Robust Reward

October 30, 2023

Asymmetric Diffusion Based Channel-Adaptive Secure Wireless Semantic Communications
Xintian Ren, Jun Wu, Hansong Xu, Qianqian Pan
Semantic Communication Diffusion Policy Semantic Communication System

October 11, 2023

NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration
Ajay Sridhar, Dhruv Shah, Catherine Glossop, Sergey Levine
Environment Exploration Local Navigation Navigation Task Diffusion Policy Agnostic Exploration

October 4, 2023

Learning to Reach Goals via Diffusion
Vineet Jain, Siamak Ravanbakhsh
LeArning Abstract Diffusion Explainer Goal Conditioned Reinforcement Learning Diffusion Policy Sequential Decision Making Pseudo Goal Goal Conditioned Policy

September 26, 2023

Maximum diffusion reinforcement learning
Thomas A. Berrueta, Allison Pinosky, Todd D. Murphey
Reinforcement Learning Maximum Entropy Diffusion Policy Sequential Experience

July 10, 2023

Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning
Suzan Ece Ada, Erhan Oztop, Emre Ugur
Offline Reinforcement Learning Distribution Generalization Behavior Cloning Offline Reinforcement Learning Algorithm Diffusion Policy

July 4, 2023

May 31, 2023

Efficient Diffusion Policies for Offline Reinforcement Learning
Bingyi Kang, Xiao Ma, Chao Du, Tianyu Pang, Shuicheng Yan
Reinforcement Learning Offline Reinforcement Learning Policy Gradient Diffusion Policy Offline RL

May 20, 2023

Diffusion Co-Policy for Synergistic Human-Robot Collaborative Tasks
Eley Ng, Ziang Liu, Monroe Kennedy
Human Robot Collaboration Diffusion Policy Human Robot Collaborative Task Multimodal Behavior

April 20, 2023

IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies
Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, Sergey Levine
Actor Critic Diffusion Policy Implicit Language Semantic Actor Offline RL Implicit Q Learning

August 12, 2022

Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning
Zhendong Wang, Jonathan J Hunt, Mingyuan Zhou
Offline Reinforcement Learning Conditional Diffusion Model Bandit Algorithm Diffusion Policy Expressive Policy Visuomotor Latent Diffusion Policy

Diffusion Policy

Papers

Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion

Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases

Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning

DiffClone: Enhanced Behaviour Cloning in Robotics with Diffusion-Driven Policy Learning

World Models via Policy-Guided Trajectory Diffusion

RL Dreams: Policy Gradient Optimization for Score Distillation based 3D Generation

Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model

Asymmetric Diffusion Based Channel-Adaptive Secure Wireless Semantic Communications

NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration

Learning to Reach Goals via Diffusion

Maximum diffusion reinforcement learning

Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning

Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning

Beyond Conservatism: Diffusion Policies in Offline Multi-agent Reinforcement Learning

Efficient Diffusion Policies for Offline Reinforcement Learning

Diffusion Co-Policy for Synergistic Human-Robot Collaborative Tasks

IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies

Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning