Policy Deep Reinforcement Learning

Policy deep reinforcement learning (DRL) aims to develop efficient algorithms that learn optimal policies from data, particularly focusing on off-policy methods which leverage past experiences. Current research emphasizes improving sample efficiency and stability through techniques like refined experience replay mechanisms (e.g., corrected uniform replay, neighborhood mixup), novel critic updates independent of the actor, and adaptive blending of online and offline learning. These advancements are significant for robotics and other domains requiring efficient learning from limited data, leading to more robust and sample-efficient control policies in complex environments.

Papers

August 4, 2024

RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
Yukinari Hisaki, Isao Ono
Soft Actor Critic Policy Deep Reinforcement Learning Average Reward Average Reward Reinforcement Learning

June 13, 2024

CUER: Corrected Uniform Experience Replay for Off-Policy Continuous Deep Reinforcement Learning Algorithms
Arda Sarp Yenicesu, Furkan B. Mutlu, Suleyman S. Kozat, Ozgur S. Oguz
Experience Replay Policy Deep Reinforcement Learning

May 28, 2024

Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL
Yu Luo, Tianying Ji, Fuchun Sun, Jianwei Zhang, Huazhe Xu, Xianyuan Zhan
Policy Reinforcement Learning Offline Policy Policy Deep Reinforcement Learning Policy Synthesis

May 1, 2024

No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO
Skander Moalla, Andrea Miele, Razvan Pascanu, Caglar Gulcehre
Reinforcement Learning Proximal Policy Optimization Individual Representation Appropriate Trust Policy Optimization Event Collapse Dynamic Representation Policy Deep Reinforcement Learning Representation Collapse

April 24, 2024

AFU: Actor-Free critic Updates in off-policy RL for continuous control
Nicolas Perrin-Gilbert
Q Learning Policy Reinforcement Learning Actor Critic Continuous Control Multiple Critic Upsampling Module Policy Deep Reinforcement Learning

March 7, 2024

Vlearn: Off-Policy Learning with Efficient State-Value Function Estimation
Fabian Otto, Philipp Becker, Ngo Anh Vien, Gerhard Neumann
Policy Learning Policy Reinforcement Learning Policy Deep Reinforcement Learning State Action Value Deep Function

January 29, 2024

SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
Jianlan Luo, Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, Sergey Levine
Reinforcement Learning Sample Efficient Reinforcement Learning Software Suite Robotics Community Robot Policy Learning Policy Deep Reinforcement Learning

November 30, 2023

Handling Cost and Constraints with Off-Policy Deep Reinforcement Learning
Jared Markowitz, Jesse Silverberg, Gary Collins
Reward Function Action Space Participation Constraint Hidden CoST Policy Actor Critic Policy Deep Reinforcement Learning

July 9, 2023

Investigating the Edge of Stability Phenomenon in Reinforcement Learning
Rares Iordan, Marc Peter Deisenroth, Mihaela Rosca
Reinforcement Learning Deep Reinforcement Learning Extreme Edge Policy Reinforcement Learning Dynamic Optimization Policy Deep Reinforcement Learning

June 20, 2023

Evolutionary Strategy Guided Reinforcement Learning via MultiBuffer Communication
Adam Callaghan, Karl Mason, Patrick Mannion
Reinforcement Learning Deep Reinforcement Learning Evolution Strategy Policy Deep Reinforcement Learning

June 5, 2023

Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic
Tianying Ji, Yu Luo, Fuchun Sun, Xianyuan Zhan, Jianwei Zhang, Huazhe Xu
Policy Learning Net Present Value Financial Success Q Function Policy Deep Reinforcement Learning Action Sampling

December 26, 2022

Off-Policy Reinforcement Learning with Loss Function Weighted by Temporal Difference Error
Bumgeun Park, Taeyoung Kim, Woohyeon Moon, Luiz Felipe Vecchietti, Dongsoo Har
Loss Function Policy Reinforcement Learning Memory Replay Policy Algorithm Policy Deep Reinforcement Learning

December 11, 2022

Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks
Altun Rzayev, Vahid Tavakol Aghaei
Manipulation Task Robotic Manipulator Multi Agent MuJoCo Policy Deep Reinforcement Learning Efficient Control

July 27, 2022

Safe and Robust Experience Sharing for Deterministic Policy Gradient Algorithms
Baturay Saglam, Dogan C. Cicek, Furkan B. Mutlu, Suleyman S. Kozat
Action Space eXperience Report Memory Replay Policy Deep Reinforcement Learning Deterministic Policy Gradient

July 3, 2022

Stabilizing Off-Policy Deep Reinforcement Learning from Pixels
Edoardo Cetin, Philip J. Ball, Steve Roberts, Oya Celiktutan
Policy Reinforcement Learning Tetromino Pixel Convolutional Encoder DeepMind Control Suite Catastrophic Overfitting Policy Deep Reinforcement Learning Training Instability

June 25, 2022

Guided Exploration in Reinforcement Learning via Monte Carlo Critic Optimization
Igor Kuznetsov
Reinforcement Learning Exploration Method Guided Exploration Policy Deep Reinforcement Learning Modern Reinforcement Learning Action Correction Unsupervised Calibration

June 9, 2022

Overcoming the Spectral Bias of Neural Value Approximation
Ge Yang, Anurag Ajay, Pulkit Agrawal
Neural Tangent Kernel Policy Reinforcement Learning Kernel Regression Spectral Bias Neural Network Approximation Policy Deep Reinforcement Learning

June 8, 2022

Action Noise in Off-Policy Deep Reinforcement Learning: Impact on Exploration and Performance
Jakob Hollenstein, Sayantan Auddy, Matteo Saveriano, Erwan Renaudo, Justus Piater
Reinforcement Learning Global Impact System Performance Environment Exploration State Space Policy Deep Reinforcement Learning Action Noise

May 18, 2022

Neighborhood Mixup Experience Replay: Local Convex Interpolation for Improved Sample Efficiency in Continuous Control Tasks
Ryan Sander, Wilko Schwarting, Tim Seyde, Igor Gilitschenski, Sertac Karaman, Daniela Rus
Experience Replay Sample Efficiency Continuous Control Task Replay Buffer Policy Deep Reinforcement Learning Class Mixup

Policy Deep Reinforcement Learning

Papers

RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning

CUER: Corrected Uniform Experience Replay for Off-Policy Continuous Deep Reinforcement Learning Algorithms

Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL

No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO

AFU: Actor-Free critic Updates in off-policy RL for continuous control

Vlearn: Off-Policy Learning with Efficient State-Value Function Estimation

SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning

Handling Cost and Constraints with Off-Policy Deep Reinforcement Learning

Investigating the Edge of Stability Phenomenon in Reinforcement Learning

Evolutionary Strategy Guided Reinforcement Learning via MultiBuffer Communication

Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic

Off-Policy Reinforcement Learning with Loss Function Weighted by Temporal Difference Error

Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks

Safe and Robust Experience Sharing for Deterministic Policy Gradient Algorithms

Stabilizing Off-Policy Deep Reinforcement Learning from Pixels

Guided Exploration in Reinforcement Learning via Monte Carlo Critic Optimization

Overcoming the Spectral Bias of Neural Value Approximation

Action Noise in Off-Policy Deep Reinforcement Learning: Impact on Exploration and Performance

Neighborhood Mixup Experience Replay: Local Convex Interpolation for Improved Sample Efficiency in Continuous Control Tasks