Proximal Policy Optimization

Proximal Policy Optimization (PPO) is a reinforcement learning algorithm used to train agents to make optimal decisions in complex environments, with a current research focus on improving its efficiency and robustness. Recent work explores enhancements such as refined credit assignment methods (e.g., VinePPO), incorporation of human feedback and safety mechanisms (e.g., HI-PPO, PRPO), and addressing challenges in high-dimensional spaces and sample efficiency through techniques like diffusion model integration. These advancements are significant for various applications, including robotics, autonomous systems, and large language model alignment, where PPO's ability to learn effective policies from interactions with the environment is crucial.

Papers

June 14, 2023

A reinforcement learning strategy for p-adaptation in high order solvers
David Huergo, Gonzalo Rubio, Esteban Ferrer
Reinforcement Learning Proximal Policy Optimization Actor Critic Reinforcement Learning High Order Solver Mesh Adaptation

June 4, 2023

Fine-Tuning Language Models with Advantage-Induced Policy Alignment
Banghua Zhu, Hiteshi Sharma, Felipe Vieira Frujeri, Shi Dong, Chenguang Zhu, Michael I. Jordan, Jiantao Jiao
Large Language Model Reinforcement Learning Proximal Policy Optimization Reward Model Language Model Fine Tuning

June 2, 2023

Deep Q-Learning versus Proximal Policy Optimization: Performance Comparison in a Material Sorting Task
Reuf Kozlica, Stefan Wegenkittl, Simon Hirländer
Proximal Policy Optimization Deep Q Learning Performance Comparison Material Separation Policy Based Algorithm

May 30, 2023

Policy Optimization for Continuous Reinforcement Learning
Hanyang Zhao, Wenpin Tang, David D. Yao
Reinforcement Learning Policy Gradient Proximal Policy Optimization Policy OpTimization

May 24, 2023

Neural Lyapunov and Optimal Control
Daniel Layeghi, Steve Tonneau, Michael Mistry
Reinforcement Learning Proximal Policy Optimization Optimal Control Reinforcement Learning Algorithm Lyapunov Function Neural Lyapunov Optimal Value Function

May 23, 2023

May 19, 2023

Learning Diverse Risk Preferences in Population-based Self-play
Yuhua Jiang, Qihan Liu, Xiaoteng Ma, Chenghao Li, Yiqin Yang, Jun Yang, Bin Liang, Qianchuan Zhao
Reinforcement Learning Proximal Policy Optimization Self Play Risk Quadrangle

May 15, 2023

A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processes
Han Zhong, Tong Zhang
Proximal Policy Optimization Theoretical Analysis Adversarial Markov Decision Process Linear Markov Decision Process

May 9, 2023

Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization
Paul Seurin, Koroush Shirvan
Deep Reinforcement Learning Proximal Policy Optimization Direct Assessment Stochastic Optimization Design Optimization Reactor Optimization

May 4, 2023

Simple Noisy Environment Augmentation for Reinforcement Learning
Raad Khraishi, Ramin Okhrati
Reinforcement Learning Data Augmentation Proximal Policy Optimization Augmentation Technique Image Augmentation Augmented Sample

April 26, 2023

Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectories
Li-Cheng Lan, Huan Zhang, Cho-Jui Hsieh
Reinforcement Learning Strong Generalization Agent Smith Proximal Policy Optimization Agent Performance Structural Generalization RELAY Selection

April 4, 2023

Regularization of the policy updates for stabilizing Mean Field Games
Talal Algumaei, Ruben Solozabal, Reda Alami, Hakim Hacid, Merouane Debbah, Martin Takac
Deep Reinforcement Learning Proximal Policy Optimization Mean Field Cooperative Multi Agent Reinforcement Learning Mean Field Game Policy Update

March 31, 2023

Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization
Mohamed-Amine Chadi, Hajar Mousannif
Reinforcement Learning Artificial Intelligence Q Learning Proximal Policy Optimization Reinforcement Learning Algorithm Much Progress State of the Art Algorithm

March 23, 2023

RLOR: A Flexible Framework of Deep Reinforcement Learning for Operation Research
Ching Pui Wan, Tung Li, Jason Min Wang
Reinforcement Learning Deep Reinforcement Learning Proximal Policy Optimization Flexible Framework Operation Research

March 22, 2023

March 15, 2023

March 7, 2023

A Multiplicative Value Function for Safe and Efficient Reinforcement Learning
Nick Bührer, Zhejun Zhang, Alexander Liniger, Fisher Yu, Luc Van Gool
Proximal Policy Optimization Safe Reinforcement Learning Value Function Efficient Reinforcement Learning Reward Learning

Proximal Policy Optimization

Papers

A reinforcement learning strategy for p-adaptation in high order solvers

Fine-Tuning Language Models with Advantage-Induced Policy Alignment

Deep Q-Learning versus Proximal Policy Optimization: Performance Comparison in a Material Sorting Task

Policy Optimization for Continuous Reinforcement Learning

Neural Lyapunov and Optimal Control

Constrained Proximal Policy Optimization

Research on Multi-Agent Communication and Collaborative Decision-Making Based on Deep Reinforcement Learning

Learning Diverse Risk Preferences in Population-based Self-play

A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processes

Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization

Simple Noisy Environment Augmentation for Reinforcement Learning

Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectories

Regularization of the policy updates for stabilizing Mean Field Games

Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization

RLOR: A Flexible Framework of Deep Reinforcement Learning for Operation Research

$P^{3}O$: Transferring Visual Representations for Reinforcement Learning via Prompting

A Hierarchical Hybrid Learning Framework for Multi-agent Trajectory Prediction

Efficient Planning of Multi-Robot Collective Transport using Graph Reinforcement Learning with Higher Order Topological Abstraction

Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks

A Multiplicative Value Function for Safe and Efficient Reinforcement Learning