Group Relative Policy Optimization

Also known as GRPO

Follow this topic Ask about this topic

Latest papers 305

48 papers in the last four weeks, up 109% on the four weeks before. 0.3% of all new papers.

All topics

Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals
May 21, 2026Shuo Yang, Jinda Lu, Chiyu Ma +8RL for Language Model Reasoning Group Relative Policy Optimization
Qwen Pilot Team, Alibaba Group
F-TIS: Harnessing Diverse Models in Collaborative GRPO
May 21, 2026Nikolay Blagoev, Oğuzhan Ersoy, Wendelin Boehmer +1RL for Language Models Communication-Efficient Distributed Training
Gensyn · University of Neuchatel · TU Delft
Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents
May 20, 2026Sikuan Yan, Ahmed Bahloul, Ercong Nie +4Group Relative Policy Optimization Credit Assignment in RL
Ludwig Maximilian University of Munich · Munich Center for Machine Learning · Huawei Heisenberg Research Center (Munich) +1
LamPO: A Lambda Style Policy Optimization for Reasoning Language Models
May 20, 2026Redacted by arXivRL for Language Model Reasoning Group Relative Policy Optimization
Pinterest, San Francisco, CA 94107, USA
Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
May 20, 2026Xixiang He, Qiyao Sun, Ao Cheng +5RL for Language Model Reasoning Group Relative Policy Optimization
National University of Defense Technology, Changsha, Hunan, China · Intelligent Game and Decision Lab, Beijing, China · The Chinese University of Hong Kong, Shenzhen, Guangdong, China
AGPO: Adaptive Group Policy Optimization with Dual Statistical Feedback
May 20, 2026Miaobo Hu, Shuhao Hu, Bokun Wang +5RL for Language Model Reasoning Group Relative Policy Optimization
Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100092, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100049, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
Complementing reinforcement learning with SFT through logit averaging in the post training of LLMs
May 19, 2026Xingwei Gan, Ying ZhuRL for Language Model Reasoning Group Relative Policy Optimization
UC San Diego
ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning
May 19, 2026Zuhao Yang, Kaichen Zhang, Sudong Wang +7Tool Use in VLMs Agentic RL
2NTU · 6LMMs-Lab · Work largely done during Z.Y., K.Z., S.W., and K.W.’s internship at MiroMind. +4
LambdaPO: A Lambda Style Policy Optimization for Reasoning Language Models
May 19, 2026Redacted by arXivRL for Language Model Reasoning Group Relative Policy Optimization
Pairwise Preference Reward and Group-Based Diversity Enhancement for Superior Open-Ended Generation
May 18, 2026Guining Cao, Jiaxin Peng, Chu Zeng +3Pairwise Preference Learning Open-Ended Generation
Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · School of Software and Microelectronics, Peking University · Tsinghua University
GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents
May 18, 2026Xiongbin Wu, Zhihao Luo, Shanzhe Lei +7Reinforcement Learning Group Relative Policy Optimization
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · East China Normal University +2
Curriculum Group Policy Optimization: Adaptive Sampling for Unleashing the Potential of Text-to-Image Generation
May 18, 2026Baoteng Li, Xianghao Zang, Xinran Wang +8Group Relative Policy Optimization Curriculum RL
School of Artificial Intelligence, Beijing University of Posts and Telecommunications1 · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance3 · Institute of Artificial Intelligence (TeleAI), China Telecom2
Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models
May 18, 2026Junyao Yang, Chen Qian, Kun Wang +4LLM Interpretability RL for Language Model Reasoning
National University of Singapore · Renmin University of China · Nanyang Technological University +1
How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
May 17, 2026Minghao Tian, Yunfei Xie, Chen WeiRL for Language Model Reasoning Group Relative Policy Optimization
Rice University
Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification
May 17, 2026Yuting Yang, Haichao Jiang, Tianming Liang +2Image Segmentation Video Object Segmentation
School of Computer Science and Engineering, Sun Yat-sen University, China · Guangdong Province Key Laboratory of Information Security Technology, China · Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China
Edit-GRPO: A Locality-Preserving Policy Optimization Framework for Image Editing
May 16, 2026Shaodong Xu, Zexian Li, Zhendong Wang +5Group Relative Policy Optimization Policy Optimization
Alibaba Group
Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models
May 16, 2026Siqi Luo, Jianghan Shen, Yi Xin +9Hierarchical RL Group Relative Policy Optimization
1Shanghai Jiao Tong University · 2Shanghai Artificial Intelligence Laboratory · 3Nanjing University +2
Learn Where Outcomes Diverge: Efficient VLA RL via Probabilistic Chunk Masking
May 15, 2026Vaidehi Bagaria, Nikshep Grampurohit, Pulkit VermaReinforcement Learning Neural Network Optimization
Indian Institute of Technology Madras
Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
May 15, 2026Xiaoxuan He, Siming Fu, Zeyue Xue +9Diffusion Model Alignment Video Diffusion Models
Zhejiang University · Joy Future Academy · Independent Researcher +1
Reference-Free Reinforcement Learning Fine-Tuning for MT: A Seq2Seq Perspective
May 15, 2026Ernesto Garcia-Estrada, Carlos Escolano, José A. R. FonallosaGroup Relative Policy Optimization Low-Resource MT
Universitat Politècnica de Catalunya Barcelona, Spain
Embedding-perturbed Exploration Preference Optimization for Flow Models
May 15, 2026Sujie Hu, Chubin Chen, Jiashu Zhu +3AI Alignment Group Relative Policy Optimization
1Tsinghua University · ⋆Work done during the internship at AMAP · 2AMAP, Alibaba Group
VSPO: Vector-Steered Policy Optimization for Behavioral Control
May 15, 2026Xuechen Zhang, Zijian Huang, Kai Yang +3Language Model Steering RL for Language Model Reasoning
University of Michigan
KVPO: ODE-Native GRPO for Autoregressive Video Alignment via KV Semantic Exploration
May 14, 2026Ruicheng Zhang, Kaixi Cong, Jun Zhou +5RL for Video Generation Long Video Generation
Tsinghua University · Video Rebirth · HKUST
GAGPO: Generalized Advantage Grouped Policy Optimization
May 13, 2026Siyuan Zhu, Chao Yu, Rongxin Yang +4Reinforcement Learning Group Relative Policy Optimization
School of Computer Science and Engineering, Sun Yat-sen University · Meituan
F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking
May 13, 2026Rohan Surana, Gagan Mundada, Junda Wu +9Group Relative Policy Optimization LLM Reranking
UC San Diego · University of Illinois at Urbana-Champaign · Adobe Research
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
May 13, 2026Feng Zhang, Xinhong Ma, Ziqiang Dong +5RL for Language Model Reasoning Group Relative Policy Optimization
Beijing Institute of Technology · Qwen Business Unit of Alibaba · The Chinese University of Hong Kong, Shenzhen +1
Holder Policy Optimisation
May 12, 2026Yuxiang Chen, Dingli Liang, Yihang Chen +8RL for Language Model Reasoning Group Relative Policy Optimization
University College London, London, United Kingdom. · 3The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · 2Shanghai Jiao Tong University, Shanghai, China.
Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting
May 12, 2026Cheng Wang, Qin Liu, Wenxuan Zhou +1Reinforcement Learning RL for Language Model Reasoning
National University of Singapore · University of California, Davis · University of Southern California
Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning
May 12, 2026Swapnil Parekh, Naman GoyalCoT Faithfulness Faithfulness of Language Model Explanations
Intuit
expo: Exploration-prioritized policy optimization via adaptive kl regulation and gaussian curriculum sampling
May 11, 2026Mingxiong Lin, Zhangquan Gong, Maowen Tang +6Reinforcement Learning KL-Regularized RL
OPPO AI Center

May 21, 2026J/K move · Enter open · S save

Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals

Shuo Yang, Jinda Lu, Chiyu Ma, Kexin Huang+7

Qwen Pilot Team, Alibaba Group

RL for Language Model Reasoning Group Relative Policy Optimization Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a central paradigm for scaling LLM reasoning, yet its optimization often suffers from training instability and suboptimal convergence. Through a systematic dissection of clipping-based GRPO-style objectives, we identify the rigid clipping decision induced by hard clipping as a key practical bottleneck in the studied RLVR setups. Specifically, our analysis suggests that informative signals can lie in the near-boundary region just beyond the clipping threshold, and are therefore discarded by the standard hard-clipping rule. Notably, once this bottleneck is precisely identified, even simple stochastic perturbations at the boundary can recover meaningful performance gains. Building on this finding, we propose Near-boundary Stochastic Rescue (NSR), a minimal, plug-and-play modification that stochastically retains these slightly out-of-bound tokens to recover lost signals. While NSR, via stochastic sampling, can be interpreted as inducing an implicit gradient decay in expectation, our ablations reveal that its stochastic, boundary-local rescue mechanism is consistently more effective than deterministic gradient decay. Validated by extensive experiments across model sizes from 7B to 30B and both dense and MoE architectures, as a plug-and-play solution, NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.