Policy Optimization
Overview
Policy optimization studies how to update a decision-making rule so it earns higher rewards while meeting constraints such as safety, fairness, or limited data. Common approaches include policy gradients, actor-critic methods, trust regions and mirror descent, as well as policy iteration and newer generative-policy methods. Work spans continuous-control benchmarks such as MuJoCo, offline and multi-agent settings, and language-model training for mathematical reasoning, coding, and tool use, where token-, segment-, or trajectory-level rewards guide updates. A central challenge is balancing effective exploration and sample reuse with stable updates, useful credit assignment, and improvements that carry over from training to deployment.
Latest papers
237 papers18 papers in the last four weeks, down 51% on the four weeks before. 0.1% of all new papers.
A Theory of Policy Optimization in Post-Training: Coverage Collapse and Reference Resets
Audited Gradients, Anchored Policies: Model-Based Reinforcement Learning for Efficient Flow Control
Ordered Policy Optimization
Foresee to Trade: Forecast-Augmented Policy Optimization for Portfolio Management
Learning Multi-Agent Orchestration:Higher-Order Interactions in Policy Improvement
MAPO: Mixture of Experts with Alternating Policy Optimization for Universal Humanoid Motion Tracking
Do Visual RL Gains Scale at Test Time? A Quantile View of Post-Training
Zeroth-Order Policy Optimization for Scheduling in Queueing Networks
Hi-CoLA: High Confidence Lower Bound Approximation Based Reinforcement Learning for Flex-Route Transit Operation Control
Conservative Policy Iteration For Zero-Sum POSGs
PolicyMediator: Multi-Regional Policy Learning through Mediation Agents with Evolving Skills in OPC Ecosystems
Multiscale Policy Optimization for Long-Horizon LLM Agents
When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning
Dual-Adaptive Z-Score PPO: Dynamic Clipping Using Action Confidence and Normalized Advantage Magnitude
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents
Doob-Based Generative Policy Optimization
Bellman Language Models: Solving the Recurring Decisions of LLM Agents
Lifted State Policy Optimization: Internalizing Self-Correction through Transition Geometry
DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation
Generalized Proximal Policy Optimization: Unifying LLM RL Algorithms from the Perspective of Divergence
Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target
avg@32 and pass@32 performance across benchmarks and data regimes. On Qwen3-1.7B, CBPO-Unified improves benchmark-averaged avg@32 and pass@32 over SFT DAPO by 2.33 and 4.44 percentage points, respectively. In the replay ablation, retaining historical rollouts improves these metrics by 1.87 and 3.72 percentage points over the no-history configuration.