Trust Region Policy Optimization

Follow this topic Ask about this topic

Momentum

0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 16

All topics

Group Adaptive Clipping Policy Optimization
Aug 31, 2026Sheng Jia, Xiao Wang, Shiva Prasad Kasiviswanathan +1Group Relative Policy Optimization Policy Optimization
University of Toronto · Amazon
Deconstructing Off-Policy Ratios: Entropy-Normalized Trust Regions for Asynchronous Reinforcement Learning
Jul 24, 2026Guanqun Zhao, Zijun Xie, Binbin Zheng +5Asynchronous RL RL for Language Model Reasoning
Beijing University of Posts and Telecommunications · Baidu Inc. · Peking University +1
Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning
Jul 21, 2026Junyao Yang, Yucheng Shi, Zongxia Li +6Asynchronous RL Proximal Policy Optimization
1Tencent Hy LLM Frontier · 2National University of Singapore · University of Maryland, College Park +3
Low Variance Trust Region Optimization with Independent Actors and Sequential Updates in Cooperative Multi-agent Reinforcement Learning
Jun 24, 2026Bang Giang Le, Viet Cuong TaSequential MARL Trust Region Policy Optimization
Human Machine Interaction Laboratory, VNU University of Engineering and Technology, 144 Xuan Thuy, Cau Giay, 100000, Hanoi, Vietnam.
TRIDENT: Breaking the Hybrid-Safety-Physics Coupling for Provably Safe Multi-Agent Reinforcement Learning
Jun 16, 2026Zijie Meng, Ziwei Li, Yufei Liu +5Cyber-Physical Systems Reinforcement Learning
Peking University · National Taiwan University · WHU +1
$α$ -fair heterogeneous agent reinforcement learning
Jun 11, 2026Yao-hua Franck Xu, Tayeb Lemlouma, Jean-Marie Bonnin +1Algorithmic Fairness Policy Optimization
Orange Innov Lannion, France · IRISA Université de Rennes Lannion, France · IRISA IMT Atlantique Rennes, France
Rethinking the Divergence Regularization in LLM RL
Jun 8, 2026Jiarui Yao, Xiangxin Zhou, Penghui Qi +3RL for Language Models Policy Optimization
1Tencent Hunyuan · 2UIUC · 3NUS
Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitions
Jun 2, 2026Bingxu Liu, Jiashun Liu, Johan Obando-Ceron +5Non-Stationary RL Reinforcement Learning
Hong Kong University of Science and Technology · Fudan University · Mila - Québec AI Institute +2
Trust Region Q Adjoint Matching
May 26, 2026Yonghoon Dong, Kyungmin Lee, Changyeon Kim +2Reinforcement Learning KL-Regularized RL
1KAIST AI · 2Seoul National University · 3RLWRLD
Trust Region Inverse Reinforcement Learning: Explicit Dual Ascent using Local Policy Updates
May 10, 2026Anish Diwan, Davide Tateo, Christopher E. Mower +3Reinforcement Learning Policy Optimization
Technical University of Darmstadt · Robotics Institute Germany (RIG) · Lund University +3
Rethinking Ratio-Based Trust Regions for Policy Optimization in Multi-Agent Reinforcement Learning
May 9, 2026Chulabhaya Wijesundara, Andrea Baisero, Zhongheng Li +3Multi-Agent System Optimization Policy Optimization
Northeastern University · STR · University of California, Irvine
ANO: Robust Policy Optimization via Bounded, Redescending Gain Fields
May 4, 2026Yiheng Zhang, Yiming Wang, Kaiyan Zhao +3Policy Gradient Methods Proximal Policy Optimization
University of Macau · Wuhan University · National University of Singapore +1
Bounded Ratio Reinforcement Learning
Apr 20, 2026Yunke Ao, Le Chen, Bruce D. Lee +5Reinforcement Learning Group Relative Policy Optimization
ETH Zurich · MPI for Intelligent Systems · University of Alberta +2
QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuning
Feb 4, 2026Doyeon Lee, Eunyi Lyou, Hyunsoo Cho +3RL for Language Model Reasoning Policy Optimization
Seoul National University · Department of Artificial Intelligence / Institute for Multiscale Matter and Systems, Ewha Woman University · Geogia Institute of Technology
SB-TRPO: Towards Safe Reinforcement Learning with Hard Constraints
Dec 29, 2025Dominik Wagner, Ankit Kanwar, Luke OngSafety-Critical Control Policy Optimization
NTU Singapore · Sony Corporation
Trust Region Masking for Long-Horizon LLM Reinforcement Learning
Dec 28, 2025Yingru Li, Jiacai Liu, Jiawei Xu +4RL for Language Models Reinforcement Learning
Fudan University · The Chinese University of Hong Kong, Shenzhen

Aug 31, 2026, cs.LGJ/K move · Enter open · S save

Group Adaptive Clipping Policy Optimization

Sheng Jia, Xiao Wang, Shiva Prasad Kasiviswanathan, Rein Houthooft

University of Toronto · Amazon

Group Relative Policy Optimization Policy Optimization Reinforcement Learning with Verifiable Rewards Trust Region Policy Optimization Policy Learning

Open paper AskPDF arXiv

Group relative policy optimization for reinforcement learning with verifiable rewards (RLVR) typically uses a fixed importance-sampling (IS) ratio clipping boundary across all rollouts. We identify a key limitation: rare correct rollouts on harder problems and abundant correct rollouts on easier problems are clipped at comparable rates, despite contributing very different learning signals. Rollouts with low group success exhibit larger IS ratios and carry stronger gradient signal for exploration and solving new problems, yet are disproportionately suppressed by fixed clipping. To address this, we propose Group Adaptive Clipping Policy Optimization (GAPO), a plug-in modification to GRPO methods that adapts the clipping boundary to the rollout advantage. GAPO is motivated by a reverse-KL trust-region perspective, which suggests that rollouts with larger learning signal should receive proportionally greater update headroom. GAPO requires no reward shaping and preserves the standard PPO/GSPO surrogate while adapting only the clipping threshold. Across Qwen and Llama models, GAPO consistently improves both Pass@1 and Pass@k over fixed clipping and advantage-shaping baselines on math reasoning and coding benchmarks where the pass rates by the base model are relatively low.

Momentum

Related topics

Latest papers 16