Proximal Policy Optimization

Proximal Policy Optimization (PPO) is a reinforcement learning algorithm used to train agents to make optimal decisions in complex environments, with a current research focus on improving its efficiency and robustness. Recent work explores enhancements such as refined credit assignment methods (e.g., VinePPO), incorporation of human feedback and safety mechanisms (e.g., HI-PPO, PRPO), and addressing challenges in high-dimensional spaces and sample efficiency through techniques like diffusion model integration. These advancements are significant for various applications, including robotics, autonomous systems, and large language model alignment, where PPO's ability to learn effective policies from interactions with the environment is crucial.

Papers

February 26, 2023

Reinforcement Learning Based Pushing and Grasping Objects from Ungraspable Poses
Hao Zhang, Hongzhuo Liang, Lin Cong, Jianzhi Lyu, Long Zeng, Pingfa Feng, Jianwei Zhang
Proximal Policy Optimization Variational Method Functional Grasping Model Free Deep Reinforcement Learning Grasping Object Stable Pushing Unusual Pose

February 22, 2023

Behavior Proximal Policy Optimization
Zifeng Zhuang, Kun Lei, Jinxin Liu, Donglin Wang, Yilang Guo
Offline Reinforcement Learning Proximal Policy Optimization Offline Policy Policy Actor Critic

February 17, 2023

Deep Reinforcement Learning for mmWave Initial Beam Alignment
Daniel Tandler, Sebastian Dörner, Marc Gauger, Stephan ten Brink
Deep Reinforcement Learning Proximal Policy Optimization Deep Reinforcement Beam Management

February 1, 2023

Bridging Physics-Informed Neural Networks with Reinforcement Learning: Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO)
Amartya Mukherjee, Jun Liu
Reinforcement Learning Neural Network Proximal Policy Optimization Bridging Text Hamilton Jacobi Bellman

January 26, 2023

January 10, 2023

Imbalanced Classification In Faulty Turbine Data: New Proximal Policy Optimization
Mohammad Hossein Modirrousta, Mahdi Aliyari Shoorehdeli, Mostafa Yari, Arash Ghahremani
Reinforcement Learning Proximal Policy Optimization Fault Detection Imbalanced Classification Data Driven Fault Limited Fault

January 9, 2023

Transformers as Policies for Variable Action Environments
Niklas Zwingenberger
Transformer Megatron Decepticons Pre Trained Model Proximal Policy Optimization Transformer Encoder Prior Policy

December 23, 2022

Proximal Policy Optimization with Graph Neural Networks for Optimal Power Flow
Ángela López-Cardona, Guillermo Bernárdez, Pere Barlet-Ros, Albert Cabellos-Aparicio
Graph Neural Network Proximal Policy Optimization Power Grid Optimal Power Flow Power Network

December 13, 2022

November 29, 2022

Approximating Martingale Process for Variance Reduction in Deep Reinforcement Learning with Large State Space
Charlie Ruan
Reinforcement Learning Deep Reinforcement Learning Proximal Policy Optimization State Space Variance Reduction

November 28, 2022

Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies
Melody Wolk, Andy Applebaum, Camron Dennler, Patrick Dwyer, Marina Moskowitz, Harold Nguyen, Nicole Nichols, Nicole Park, Paul Rachwalski, Frank Rau, Adrian Webster
Reinforcement Learning Strong Generalization Proximal Policy Optimization Commonsense Augmented Dialogue Unknown Attack Autonomous Network

November 27, 2022

Combined Peak Reduction and Self-Consumption Using Proximal Policy Optimization
Thijs Peirelinck, Chris Hermans, Fred Spiessens, Geert Deconinck
Reinforcement Learning Proximal Policy Optimization Self Information Demand Response Demand Response Program

November 23, 2022

Reinforcement learning for traffic signal control in hybrid action space
Haoqing Luo, sheng jin
Reinforcement Learning Action Space Proximal Policy Optimization Traffic Signal Control Asynchronous Optimization

November 15, 2022

Coordination for Connected and Automated Vehicles at Non-signalized Intersections: A Value Decomposition-based Multiagent Deep Reinforcement Learning Approach
Zihan Guo, Yan Wu, Lifang Wang, Junzhi Zhang
Reinforcement Learning Proximal Policy Optimization Automated Driving Multi Agent Deep Reinforcement Learning Prior Coordination Connected Behavior Mixed Autonomy Traffic Signalized Intersection

November 12, 2022

CACTO: Continuous Actor-Critic with Trajectory Optimization -- Towards global optimality
Gianluigi Grandesso, Elisa Alboni, Gastone P. Rosati Papini, Patrick M. Wensing, Andrea Del Prete
Reinforcement Learning Proximal Policy Optimization Trajectory Optimization Actor Critic Global Optimality Cacto Sl

November 11, 2022

Efficient Deep Reinforcement Learning with Predictive Processing Proximal Policy Optimization
Burcu Küçükoğlu, Walraaf Borkent, Bodo Rueckauer, Nasir Ahmad, Umut Güçlü, Marcel van Gerven
Reinforcement Learning Deep Reinforcement Learning Recurrent Neural Network Proximal Policy Optimization Actor Critic Reinforcement Learning

October 20, 2022

Co-Training an Observer and an Evading Target
André Brandenburger, Folker Hoffmann, Alexander Charlish
Reinforcement Learning Multi Agent Reinforcement Learning Proximal Policy Optimization

October 15, 2022

A Multilevel Reinforcement Learning Framework for PDE-based Control
Atish Dixit, Ahmed Elsheikh
Reinforcement Learning External Control Proximal Policy Optimization Model Free Reinforcement Learning Monte Carlo

Proximal Policy Optimization

Papers

Reinforcement Learning Based Pushing and Grasping Objects from Ungraspable Poses

Behavior Proximal Policy Optimization

Deep Reinforcement Learning for mmWave Initial Beam Alignment

Bridging Physics-Informed Neural Networks with Reinforcement Learning: Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO)

Partial advantage estimator for proximal policy optimization

Joint action loss for proximal policy optimization

Imbalanced Classification In Faulty Turbine Data: New Proximal Policy Optimization

Transformers as Policies for Variable Action Environments

Proximal Policy Optimization with Graph Neural Networks for Optimal Power Flow

Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets

PPO-UE: Proximal Policy Optimization via Uncertainty-Aware Exploration

Approximating Martingale Process for Variance Reduction in Deep Reinforcement Learning with Large State Space

Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies

Combined Peak Reduction and Self-Consumption Using Proximal Policy Optimization

Reinforcement learning for traffic signal control in hybrid action space

Coordination for Connected and Automated Vehicles at Non-signalized Intersections: A Value Decomposition-based Multiagent Deep Reinforcement Learning Approach

CACTO: Continuous Actor-Critic with Trajectory Optimization -- Towards global optimality

Efficient Deep Reinforcement Learning with Predictive Processing Proximal Policy Optimization

Co-Training an Observer and an Evading Target

A Multilevel Reinforcement Learning Framework for PDE-based Control