Proximal Policy Optimization
Proximal Policy Optimization (PPO) is a reinforcement learning algorithm used to train agents to make optimal decisions in complex environments, with a current research focus on improving its efficiency and robustness. Recent work explores enhancements such as refined credit assignment methods (e.g., VinePPO), incorporation of human feedback and safety mechanisms (e.g., HI-PPO, PRPO), and addressing challenges in high-dimensional spaces and sample efficiency through techniques like diffusion model integration. These advancements are significant for various applications, including robotics, autonomous systems, and large language model alignment, where PPO's ability to learn effective policies from interactions with the environment is crucial.
Papers
Learning Quadrupedal Locomotion via Differentiable Simulation
Clemens Schwarke, Victor Klemm, Jesus Tordesillas, Jean-Pierre Sleiman, Marco Hutter
Solving a Real-World Optimization Problem Using Proximal Policy Optimization with Curriculum Learning and Reward Engineering
Abhijeet Pendyala, Asma Atamna, Tobias Glasmachers
A Safe Reinforcement Learning Algorithm for Supervisory Control of Power Plants
Yixuan Sun, Sami Khairy, Richard B. Vilim, Rui Hu, Akshay J. Dave
Backpropagation Through Agents
Zhiyuan Li, Wenshuai Zhao, Lijun Wu, Joni Pajarinen
Reinforcement Learning for Graph Coloring: Understanding the Power and Limits of Non-Label Invariant Representations
Chase Cummins, Richard Veras