Proximal Policy Optimization

Also known as PPO

Overview

Proximal Policy Optimization (PPO) is an actor–critic method that learns actions from rewards while limiting how much the policy changes at each update, commonly through clipped probability ratios and generalized advantage estimation. Current approaches refine this update rule with divergence-based trust regions, adaptive clipping, better value critics, and selective reuse of past experience; safety constraints, demonstrations, and sequence models are also used to guide learning. PPO is applied to benchmarks such as Atari and MuJoCo, as well as robotics, multi-agent control, and language-model reasoning. A central challenge is improving sample efficiency without making updates unstable: clipping and on-policy data can help control risk, but may restrict useful learning, while critic errors and stale data can undermine performance.

Latest papers

CardsList

126 papers12 papers in the last four weeks, down 37% on the four weeks before. 0.1% of all new papers.

  1. Policy Improvement with Style-Specific Demonstrations

    Jun 20, 2025Lingfeng Li, Yunlong Lu, Yongyi Wang +1Reinforcement LearningGame-Playing Agents

  2. PPO in the Fisher-Rao geometry

    Jun 4, 2025Razvan-Andrei Lascu, David Šiška, Łukasz SzpruchProximal Policy OptimizationPolicy Optimization

  3. Learning Large-Scale Competitive Team Behaviors with Mean-Field Interactions

    Apr 29, 2025Bhavini Jeloka, Yue Guan, Panagiotis TsiotrasMean-Field MARLMulti-Agent Reinforcement Learning

  4. Atomic Proximal Policy Optimization for Electric Robo-Taxi Dispatch and Charger Allocation

    Feb 19, 2025Jim Dai, Manxi Wu, Zhanhao ZhangProximal Policy OptimizationElectric Vehicle Charging

  5. Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms

    Jul 21, 2024Sheila Schoepp, Mehran Taghian, Shotaro Miwa +3RL ControlReinforcement Learning

  6. Transductive Off-policy Proximal Policy Optimization

    Jun 6, 2024Yaozhong Gan, Xiaoyang Tan, Zhe Wu +1Policy OptimizationProximal Policy Optimization