Proximal Policy Optimization

Proximal Policy Optimization (PPO) is a reinforcement learning algorithm used to train agents to make optimal decisions in complex environments, with a current research focus on improving its efficiency and robustness. Recent work explores enhancements such as refined credit assignment methods (e.g., VinePPO), incorporation of human feedback and safety mechanisms (e.g., HI-PPO, PRPO), and addressing challenges in high-dimensional spaces and sample efficiency through techniques like diffusion model integration. These advancements are significant for various applications, including robotics, autonomous systems, and large language model alignment, where PPO's ability to learn effective policies from interactions with the environment is crucial.

Papers

September 15, 2024

Mitigating Dimensionality in 2D Rectangle Packing Problem under Reinforcement Learning Schema
Waldemar Kołodziejczyk, Mariusz Kaleta
Reinforcement Learning Proximal Policy Optimization Dimension Reduction Heuristic Search Reinforcement Learning Framework Irregular Shape Packing

September 13, 2024

AIPO: Improving Training Objective for Iterative Preference Optimization
Yaojie Shen, Xinyao Wang, Yulei Niu, Ying Zhou, Lexin Tang, Libo Zhang, Fan Chen, Longyin Wen
Proximal Policy Optimization Preference Optimization Training Objective

September 11, 2024

Policy Filtration in RLHF to Fine-Tune LLM for Code Generation
Wei Shen, Chuheng Zhang
Reinforcement Learning Medical LLM Code Generation Proximal Policy Optimization Reward Model Policy Learning Reinforcement Learning From Human Feedback Policy Search

September 9, 2024

Developing Path Planning with Behavioral Cloning and Proximal Policy Optimization for Path-Tracking and Static Obstacle Nudging
Mingyan Zhou, Biao Wang, Tian Tan, Xiatao Sun
Reinforcement Learning Autonomous Driving Imitation Learning Proximal Policy Optimization Trajectory Planning Behavior Cloning Static Obstacle

September 7, 2024

Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn
Hongyao Tang, Glen Berseth
Reinforcement Learning Proximal Policy Optimization Learning Dynamic Net Present Value Policy Iteration Churn Prediction Customer Churn

September 2, 2024

August 28, 2024

Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System
Georg Schäfer, Jakob Rehrl, Stefan Huber, Simon Hirlaender
Deep Reinforcement Learning Predictive Control Consistent Comparison Proximal Policy Optimization Linear Quadratic Regulator

August 27, 2024

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data
Han Xia, Songyang Gao, Qiming Ge, Zhiheng Xi, Qi Zhang, Xuanjing Huang
Reinforcement Learning Optimal Policy Proximal Policy Optimization Direct Preference Optimization Token Level

August 9, 2024

Kolmogorov-Arnold Network for Online Reinforcement Learning
Victor Augusto Kich, Jair Augusto Bottega, Raul Steinmetz, Ricardo Bedin Grando, Ayano Yorozu, Akihisa Ohya
Proximal Policy Optimization Kolmogorov Arnold Network Online Reinforcement Learning Reinforcement Learning Model

August 6, 2024

Integrated Intention Prediction and Decision-Making with Spectrum Attention Net and Proximal Policy Optimization
Xiao Zhou, Chengzhen Meng, Wenru Liu, Zengqi Peng, Ming Liu, Jun Ma
Reinforcement Learning Autonomous Driving Decision Making Proximal Policy Optimization Intent Prediction

July 30, 2024

ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning
Hosung Lee, Sejin Kim, Seungpil Lee, Sanha Hwang, Jihwan Lee, Byung-Jun Lee, Sundong Kim
Reinforcement Learning Action Space Proximal Policy Optimization Cognitive Abstraction Reasoning Corpus Reinforcement Learning Research

July 29, 2024

July 25, 2024

Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimation
Jean Seong Bjorn Choe, Jong-Kook Kim
Proximal Policy Optimization Actor Critic Maximum Entropy Reinforcement Learning Trust Region Policy Optimization Efficient Policy Learning

July 21, 2024

Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms
Sheila Schoepp, Mehran Taghian, Shotaro Miwa, Yoshihiro Mitsuka, Shadan Golestan, Osmar Zaïane
New Machine Policy Gradient Proximal Policy Optimization Robot Learning State of the Art Reinforcement Adaptive Machine

July 19, 2024

A Comparative Study of Deep Reinforcement Learning Models: DQN vs PPO vs A2C
Neil De La Fuente, Daniel A. Vidal Guerra
Deep Reinforcement Learning Comparative Study Proximal Policy Optimization Deep Q Network Policy Optimization Dynamic Game Apical 2 Chamber

July 18, 2024

PG-Rainbow: Using Distributional Reinforcement Learning in Policy Gradient Methods
WooJae Jeon, KangJun Lee, Jeewoo Lee
Policy Gradient Proximal Policy Optimization Distributional Reinforcement Learning Reinforcement Learning Task Policy Gradient Algorithm

July 12, 2024

HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context
Federico Arangath Joseph, Kilian Konstantin Haefeli, Noah Liniger, Caglar Gulcehre
Dynamical System Context Information State Space Model Proximal Policy Optimization Sequence Model Explicit Weight

July 10, 2024

Structural Design Through Reinforcement Learning
Thomas Rochefort-Beaudoin, Aurelian Vadean, Niels Aage, Sofiane Achiche
Proximal Policy Optimization Topology Optimization Empirical Game Structural Optimization Open Source Reinforcement Learning

Proximal Policy Optimization

Papers

Mitigating Dimensionality in 2D Rectangle Packing Problem under Reinforcement Learning Schema

AIPO: Improving Training Objective for Iterative Preference Optimization

Policy Filtration in RLHF to Fine-Tune LLM for Code Generation

Developing Path Planning with Behavioral Cloning and Proximal Policy Optimization for Path-Tracking and Static Obstacle Nudging

Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn

Enhancing Sample Efficiency and Exploration in Reinforcement Learning through the Integration of Diffusion Models and Proximal Policy Optimization

Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning

Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data

Kolmogorov-Arnold Network for Online Reinforcement Learning

Integrated Intention Prediction and Decision-Making with Spectrum Attention Net and Proximal Policy Optimization

ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning

Appraisal-Guided Proximal Policy Optimization: Modeling Psychological Disorders in Dynamic Grid World

Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank

Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimation

Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms

A Comparative Study of Deep Reinforcement Learning Models: DQN vs PPO vs A2C

PG-Rainbow: Using Distributional Reinforcement Learning in Policy Gradient Methods

HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context

Structural Design Through Reinforcement Learning