Proximal Policy Optimization
Proximal Policy Optimization (PPO) is a reinforcement learning algorithm used to train agents to make optimal decisions in complex environments, with a current research focus on improving its efficiency and robustness. Recent work explores enhancements such as refined credit assignment methods (e.g., VinePPO), incorporation of human feedback and safety mechanisms (e.g., HI-PPO, PRPO), and addressing challenges in high-dimensional spaces and sample efficiency through techniques like diffusion model integration. These advancements are significant for various applications, including robotics, autonomous systems, and large language model alignment, where PPO's ability to learn effective policies from interactions with the environment is crucial.
Papers
Reward Scale Robustness for Proximal Policy Optimization via DreamerV3 Tricks
Ryan Sullivan, Akarsh Kumar, Shengyi Huang, John P. Dickerson, Joseph Suarez
Learning Regularized Graphon Mean-Field Games with Unknown Graphons
Fengzhuo Zhang, Vincent Y. F. Tan, Zhaoran Wang, Zhuoran Yang
Coalitional Bargaining via Reinforcement Learning: An Application to Collaborative Vehicle Routing
Stephen Mak, Liming Xu, Tim Pearce, Michael Ostroumov, Alexandra Brintrup