Proximal Policy Optimization

Proximal Policy Optimization (PPO) is a reinforcement learning algorithm used to train agents to make optimal decisions in complex environments, with a current research focus on improving its efficiency and robustness. Recent work explores enhancements such as refined credit assignment methods (e.g., VinePPO), incorporation of human feedback and safety mechanisms (e.g., HI-PPO, PRPO), and addressing challenges in high-dimensional spaces and sample efficiency through techniques like diffusion model integration. These advancements are significant for various applications, including robotics, autonomous systems, and large language model alignment, where PPO's ability to learn effective policies from interactions with the environment is crucial.

Papers

August 10, 2023

Proximal Policy Optimization Actual Combat: Manipulating Output Tokenizer Length
Miao Fan, Chen Hu, Shuchang Zhou
Large Language Model Reinforcement Learning Proximal Policy Optimization Reward Model

July 26, 2023

LiDAR-based drone navigation with reinforcement learning
Pawel Miera, Hubert Szolc, Tomasz Kryjak
Reinforcement Learning Unmanned Aerial Vehicle Proximal Policy Optimization Single Drone Autonomous Drone Drone Navigation

July 24, 2023

SafeSteps: Learning Safer Footstep Planning Policies for Legged Robots via Model-Based Priors
Shafeef Omar, Lorenzo Amatucci, Victor Barasuol, Giulio Turrisi, Claudio Semini
Legged Robot Proximal Policy Optimization Quadrupedal Locomotion Model Based Prior Footstep Planning Policy

July 19, 2023

PyTAG: Challenges and Opportunities for Reinforcement Learning in Tabletop Games
Martin Balla, George E. M. Long, Dominik Jeurissen, James Goodman, Raluca D. Gaina, Diego Perez-Liebana
Reinforcement Learning Technical Challenge Emerging Opportunity Proximal Policy Optimization Tabletop Game

July 11, 2023

Secrets of RLHF in Large Language Models Part I: PPO
Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin, Qin Liu, Yuhao Zhou, Limao Xiong, Lu Chen, Zhiheng Xi, Nuo Xu, Wenbin Lai, Minghao Zhu, Cheng Chang, Zhangyue Yin, Rongxiang Weng, Wensen Cheng, Haoran Huang, Tianxiang Sun, Hang Yan, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang
Reinforcement Learning Proximal Policy Optimization Reward Model Policy Optimization Reinforcement Learning From Human Feedback Dark Secret

July 7, 2023

Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learning
Seungyong Moon, Junyoung Yeom, Bumsoo Park, Hyun Oh Song
Reinforcement Learning Contrastive Learning Long Term Proximal Policy Optimization Teacher Student Distillation Hierarchical Reward

July 6, 2023

A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations
Sergio F. Chevtchenko, Yeshwanth Bethi, Teresa B. Ludermir, Saeed Afshar
Reinforcement Learning Spiking Neural Network Proximal Policy Optimization Actor Critic Algorithm Efficient Reinforcement Learning Real Data Neuromorphic Architecture

July 5, 2023

Multi-objective Deep Reinforcement Learning for Mobile Edge Computing
Ning Yang, Junrui Wen, Meng Zhang, Ming Tang
Proximal Policy Optimization Mobile Edge Computing Multi Objective Deep Reinforcement Learning Next Generation Wireless Network

June 30, 2023

Navigation of micro-robot swarms for targeted delivery using reinforcement learning
Akshatha Jagadish, Manoj Varma
Reinforcement Learning Proximal Policy Optimization Local Navigation Robust Policy Delivery Problem Micro Robot Swarm

June 20, 2023

Learning to Generate Better Than Your LLM
Jonathan D. Chang, Kiante Brantley, Rajkumar Ramamurthy, Dipendra Misra, Wen Sun
Reinforcement Learning LeArning Abstract Medical LLM Text Generation Proximal Policy Optimization LLM Fine Tuning

June 14, 2023

A reinforcement learning strategy for p-adaptation in high order solvers
David Huergo, Gonzalo Rubio, Esteban Ferrer
Reinforcement Learning Proximal Policy Optimization Actor Critic Reinforcement Learning High Order Solver Mesh Adaptation

June 4, 2023

Fine-Tuning Language Models with Advantage-Induced Policy Alignment
Banghua Zhu, Hiteshi Sharma, Felipe Vieira Frujeri, Shi Dong, Chenguang Zhu, Michael I. Jordan, Jiantao Jiao
Large Language Model Reinforcement Learning Proximal Policy Optimization Reward Model Language Model Fine Tuning

June 2, 2023

Deep Q-Learning versus Proximal Policy Optimization: Performance Comparison in a Material Sorting Task
Reuf Kozlica, Stefan Wegenkittl, Simon Hirländer
Proximal Policy Optimization Deep Q Learning Performance Comparison Material Separation Policy Based Algorithm

May 30, 2023

Policy Optimization for Continuous Reinforcement Learning
Hanyang Zhao, Wenpin Tang, David D. Yao
Reinforcement Learning Policy Gradient Proximal Policy Optimization Policy OpTimization

May 24, 2023

Neural Lyapunov and Optimal Control
Daniel Layeghi, Steve Tonneau, Michael Mistry
Reinforcement Learning Proximal Policy Optimization Optimal Control Reinforcement Learning Algorithm Lyapunov Function Neural Lyapunov Optimal Value Function

May 23, 2023

May 19, 2023

Learning Diverse Risk Preferences in Population-based Self-play
Yuhua Jiang, Qihan Liu, Xiaoteng Ma, Chenghao Li, Yiqin Yang, Jun Yang, Bin Liang, Qianchuan Zhao
Reinforcement Learning Proximal Policy Optimization Self Play Risk Quadrangle

May 15, 2023

A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processes
Han Zhong, Tong Zhang
Proximal Policy Optimization Theoretical Analysis Adversarial Markov Decision Process Linear Markov Decision Process

May 9, 2023

Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization
Paul Seurin, Koroush Shirvan
Deep Reinforcement Learning Proximal Policy Optimization Direct Assessment Stochastic Optimization Design Optimization Reactor Optimization

Proximal Policy Optimization

Papers

Proximal Policy Optimization Actual Combat: Manipulating Output Tokenizer Length

LiDAR-based drone navigation with reinforcement learning

SafeSteps: Learning Safer Footstep Planning Policies for Legged Robots via Model-Based Priors

PyTAG: Challenges and Opportunities for Reinforcement Learning in Tabletop Games

Secrets of RLHF in Large Language Models Part I: PPO

Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learning

A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations

Multi-objective Deep Reinforcement Learning for Mobile Edge Computing

Navigation of micro-robot swarms for targeted delivery using reinforcement learning

Learning to Generate Better Than Your LLM

A reinforcement learning strategy for p-adaptation in high order solvers

Fine-Tuning Language Models with Advantage-Induced Policy Alignment

Deep Q-Learning versus Proximal Policy Optimization: Performance Comparison in a Material Sorting Task

Policy Optimization for Continuous Reinforcement Learning

Neural Lyapunov and Optimal Control

Constrained Proximal Policy Optimization

Research on Multi-Agent Communication and Collaborative Decision-Making Based on Deep Reinforcement Learning

Learning Diverse Risk Preferences in Population-based Self-play

A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processes

Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization