Policy Optimization

Follow this topic Ask about this topic

Policy optimization studies how to update a decision-making rule so it earns higher rewards while meeting constraints such as safety, fairness, or limited data. Common approaches include policy gradients, actor-critic methods, trust regions and mirror descent, as well as policy iteration and newer generative-policy methods. Work spans continuous-control benchmarks such as MuJoCo, offline and multi-agent settings, and language-model training for mathematical reasoning, coding, and tool use, where token-, segment-, or trajectory-level rewards guide updates. A central challenge is balancing effective exploration and sample reuse with stable updates, useful credit assignment, and improvements that carry over from training to deployment.

Latest papers

221 papers32 papers in the last four weeks, up 52% on the four weeks before. 0.2% of all new papers.

SB-TRPO: Towards Safe Reinforcement Learning with Hard Constraints
Dec 29, 2025Dominik Wagner, Ankit Kanwar, Luke OngSafety-Critical Control Policy Optimization
NTU Singapore · Sony Corporation
Auto-exploration for online reinforcement learning
Dec 6, 2025Caleb Ju, Guanghui LanPolicy Optimization RL Exploration
Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
Oct 9, 2025Yuchen Zhu, Wei Guo, Jaemoo Choi +4RL for Language Model Reasoning Policy Optimization
Georgia Institute of Technology.
Sequential Cohort Selection under Uncertainty
Aug 22, 2025Hortence Yiepnou, Christos DimitrakakisSequential Decision Making Policy Optimization
University of Neuch atel
TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
Aug 4, 2025Lei Pang, Jun Luo, Ruinan JinImportance Sampling for RL RL for Language Models
Peking University Beijing, China · The Ohio State University Columbus, USA · Shugu Yuntai Technology Co., Ltd. +1
Risk-Aware Alignment Must Incorporate Reward Uncertainty
Oct 31, 2024Debangshu Banerjee, Kintan Saha, Aditya GopalanPolicy Optimization LLM Alignment
Department of Electrical and Communication Engineering, Indian Institute of Science, India
Transductive Off-policy Proximal Policy Optimization
Jun 6, 2024Yaozhong Gan, Xiaoyang Tan, Zhe Wu +1Policy Optimization Proximal Policy Optimization
Independent Researcher
Reflective Policy Optimization
Jun 6, 2024Yaozhong Gan, Renye Yan, Zhe Wu +1Policy Optimization Policy Gradient Methods
QiYuan Lab.
Leader Reward for POMO-Based Neural Combinatorial Optimization
May 22, 2024Chaoyang Wang, Pengzhi Cheng, Jingze Li +1Traveling Salesperson Problem Reward Shaping
Fudan University
Bridging the Gap between Newton-Raphson Method and Regularized Policy Iteration
Oct 11, 2023Zeyang Li, Chuxiong Hu, Yunan Wang +4Markov Decision Processes Reinforcement Learning
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA 02139, United States · Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China · College of Artificial Intelligence and the School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
Policy design in experiments with unknown interference
Nov 16, 2020Davide Viviano, Jess RudderCausal Effect Estimation Policy Optimization
Department of Economics, Harvard University, 1805 Cambridge St, Cambridge, MA 02138. · Department of Economics, Oregon State University, 1500 SW Jefferson Way, Corvallis, OR 97331.

Dec 29, 2025J/K move · Enter open · S save

SB-TRPO: Towards Safe Reinforcement Learning with Hard Constraints

Dominik Wagner, Ankit Kanwar, Luke Ong

NTU Singapore · Sony Corporation

Safety-Critical Control Policy Optimization Constrained RL Trust Region Policy Optimization Safe RL Policy Learning

Open paper AskPDF arXiv

In safety-critical domains, reinforcement learning (RL) systems must satisfy strict, zero-cost safety constraints while achieving meaningful task performance. Existing model-free methods can struggle to achieve high safety without substantially compromising task performance. We introduce Safety-Biased Trust Region Policy Optimisation (SB-TRPO), a principled approach to RL with zero-cost constraints, which requires only a fixed fraction of the maximal cost reduction achievable within the trust region, thus retaining flexibility for reward optimisation. We show that the idealised update nevertheless converges to zero cost and maximal reward amongst zero-cost policies in finite MDPs. A practical gradient-based approximation provides local improvements in both safety and reward under suitable gradient alignment. Experiments on Safety Gymnasium demonstrate high safety alongside strong task performance.