Proximal Policy Optimization

Proximal Policy Optimization (PPO) is a reinforcement learning algorithm used to train agents to make optimal decisions in complex environments, with a current research focus on improving its efficiency and robustness. Recent work explores enhancements such as refined credit assignment methods (e.g., VinePPO), incorporation of human feedback and safety mechanisms (e.g., HI-PPO, PRPO), and addressing challenges in high-dimensional spaces and sample efficiency through techniques like diffusion model integration. These advancements are significant for various applications, including robotics, autonomous systems, and large language model alignment, where PPO's ability to learn effective policies from interactions with the environment is crucial.

Papers

May 19, 2022

Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemble
Zhengyu Yang, Kan Ren, Xufang Luo, Minghuan Liu, Weiqing Liu, Jiang Bian, Weinan Zhang, Dongsheng Li
Reinforcement Learning Strong Generalization Proximal Policy Optimization Sample Efficiency Ensemble Deep Reinforcement Learning Vanilla Reinforcement Learning

May 18, 2022

A2C is a special case of PPO
Shengyi Huang, Anssi Kanervisto, Antonin Raffin, Weixun Wang, Santiago Ontañón, Rousslan Fernand Julien Dossa
Proximal Policy Optimization Policy Optimization Game AI Deep Reinforcement Learning Algorithm Advantage Actor Critic Special Case Pseudo Code Apical 2 Chamber

May 16, 2022

A Deep Reinforcement Learning Blind AI in DareFightingICE
Thai Van Nguyen, Xincheng Dai, Ibrahim Khan, Ruck Thawonmas, Hai V. Pham
Artificial Intelligence Proximal Policy Optimization Deep Reinforcement DareFightingICE Platform

May 7, 2022

AdaptiveON: Adaptive Outdoor Local Navigation Method For Stable and Reliable Actions
Jing Liang, Kasun Weerakoon, Tianrui Guan, Nare Karapetyan, Dinesh Manocha
Autonomous Navigation Proximal Policy Optimization Real World Environment Outdoor Navigation Outdoor Robot Reliable Online Method

May 1, 2022

Processing Network Controls via Deep Reinforcement Learning
Mark Gluzman
Deep Reinforcement Learning Policy Gradient Proximal Policy Optimization Policy Improvement Trust Region Policy Optimization Network Control

April 12, 2022

Proximal Policy Optimization Learning based Control of Congested Freeway Traffic
Shurong Mo, Nailong Wu, Jie Qi, Anqi Pan, Zhiguang Feng, Huaicheng Yan, Yueying Wang
External Control Proximal Policy Optimization Lyapunov Function Adaptive Cruise Control Traffic Flow Traffic Dynamic

April 1, 2022

Hysteresis-Based RL: Robustifying Reinforcement Learning-based Control Policies via Hybrid Control
Jan de Priester, Ricardo G. Sanfelice, Nathan van de Wouw
Reinforcement Learning Proximal Policy Optimization Deep Q Network Robust Reinforcement Learning Hybrid Control Hysteresis Compensation

March 22, 2022

Is Vanilla Policy Gradient Overlooked? Analyzing Deep Reinforcement Learning for Hanabi
Bram Grooten, Jelle Wemmenhove, Maurice Poot, Jim Portegies
Deep Reinforcement Learning Multi Agent Proximal Policy Optimization Multi Agent Collaboration Vanilla Reinforcement Learning Game Hanabi

March 18, 2022

Proximal Policy Optimization with Adaptive Threshold for Symmetric Relative Density Ratio
Taisuke Kobayashi
Deep Reinforcement Learning Non Humanoid Robot Proximal Policy Optimization Density Ratio Adaptive Threshold Threshold Optimization

March 11, 2022

Combining imitation and deep reinforcement learning to accomplish human-level performance on a virtual foraging task
Vittorio Giammarino, Matthew F Dunne, Kylie N Moore, Michael E Hasselmo, Chantal E Stern, Ioannis Ch. Paschalidis
Reinforcement Learning Deep Reinforcement Learning Imitation Learning Proximal Policy Optimization Feature Imitation Human Performance Level Based Foraging

March 5, 2022

AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles
Haijun Zhang, Minghui Jiang, Xiangnan Liu, Keping Long, Victor C. M. Leung
Proximal Policy Optimization Vehicle Plan Random Access Internet of Vehicle Mixed Traffic Control Traffic Control

March 1, 2022

Approximating a deep reinforcement learning docking agent using linear model trees
Vilde B. Gjærum, Ella-Lovise H. Rørvik, Anastasios M. Lekkas
Deep Reinforcement Learning Agent Smith Proximal Policy Optimization Autonomous Surface Vehicle Opaque Machine Learning Autonomous Vessel Linear Model Tree

January 31, 2022

You May Not Need Ratio Clipping in PPO
Mingfei Sun, Vitaly Kurin, Guoqing Liu, Sam Devlin, Tao Qin, Katja Hofmann, Shimon Whiteson
Proximal Policy Optimization Policy Optimization Mini Batch Gradient Descent Adaptive Clipping Surrogate Objective Epoch Based

January 29, 2022

ApolloRL: a Reinforcement Learning Platform for Autonomous Driving
Fei Gao, Peng Geng, Jiaqi Guo, Yuan Liu, Dingfeng Guo, Yabo Su, Jie Zhou, Xiao Wei, Jin Li, Xu Liu
Reinforcement Learning Autonomous Driving Proximal Policy Optimization

January 28, 2022

Joint Differentiable Optimization and Verification for Certified Reinforcement Learning
Yixuan Wang, Simon Zhan, Zhilu Wang, Chao Huang, Zhaoran Wang, Zhuoran Yang, Qi Zhu
Reinforcement Learning Proximal Policy Optimization Model Based Reinforcement Learning Verification Task Joint Optimization Learned Controller Novel Control

January 24, 2022

STOPS: Short-Term-based Volatility-controlled Policy Search and its Global Convergence
Liangliang Xu, Daoming Lyu, Yangchen Pan, Aiwen Jiang, Bo Liu
Proximal Policy Optimization Global Convergence Natural Policy Gradient

January 15, 2022

Profitable Strategy Design by Using Deep Reinforcement Learning for Trades on Cryptocurrency Markets
Mohsen Asgari, Seyed Hossein Khasteh
Deep Reinforcement Learning Proximal Policy Optimization Generative Adversarial Imitation Learning Post Trade Allocation Cryptocurrency Market Strategy Synthesis Reinforcement Learning Solution

January 11, 2022

December 27, 2021

Intelligent Traffic Light via Policy-based Deep Reinforcement Learning
Yue Zhu, Mingyu Cai, Chris Schwarz, Junchao Li, Shaoping Xiao
Reinforcement Learning Proximal Policy Optimization Intelligent Traffic Light

Proximal Policy Optimization

Papers

Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemble

A2C is a special case of PPO

A Deep Reinforcement Learning Blind AI in DareFightingICE

AdaptiveON: Adaptive Outdoor Local Navigation Method For Stable and Reliable Actions

Processing Network Controls via Deep Reinforcement Learning

Proximal Policy Optimization Learning based Control of Congested Freeway Traffic

Hysteresis-Based RL: Robustifying Reinforcement Learning-based Control Policies via Hybrid Control

Is Vanilla Policy Gradient Overlooked? Analyzing Deep Reinforcement Learning for Hanabi

Proximal Policy Optimization with Adaptive Threshold for Symmetric Relative Density Ratio

Combining imitation and deep reinforcement learning to accomplish human-level performance on a virtual foraging task

AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles

Approximating a deep reinforcement learning docking agent using linear model trees

You May Not Need Ratio Clipping in PPO

ApolloRL: a Reinforcement Learning Platform for Autonomous Driving

Joint Differentiable Optimization and Verification for Certified Reinforcement Learning

STOPS: Short-Term-based Volatility-controlled Policy Search and its Global Convergence

Profitable Strategy Design by Using Deep Reinforcement Learning for Trades on Cryptocurrency Markets

Benchmarking Deep Reinforcement Learning Algorithms for Vision-based Robotics

Learning Robust Policies for Generalized Debris Capture with an Automated Tether-Net System

Intelligent Traffic Light via Policy-based Deep Reinforcement Learning