Group Relative Policy Optimization

Also known as GRPO

Follow this topic Ask about this topic

Latest papers 305

48 papers in the last four weeks, up 109% on the four weeks before. 0.3% of all new papers.

All topics

Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data
Apr 20, 2026Zhenwen Liang, Yujun Zhou, Sidi Lu +3Reinforcement Learning RL for Language Model Reasoning
Tencent AI Lab · University of Notre Dame
Negative Advantages Is a Double-Edged Sword: Calibrating advantages in GRPO for Search Agents
Apr 20, 2026Jiayi Wu, Ruobing Xie, Zeqian Huang +6Reinforcement Learning RL for Language Model Reasoning
School of Data Science and Engineering, East China Normal University · Tencent · Tsinghua University
Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thought
Apr 20, 2026Muhammed Emrullah Ildiz, Halil Alperen Gozeten, Ege Onur Taga +1CoT Reasoning RL for Language Model Reasoning
University of Michigan, Ann Arbor · Google Research
MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Models
Apr 18, 2026Zhaokang Liao, Yingguo Gao, Yi Yang +2RL for Language Model Reasoning Group Relative Policy Optimization
Ant Group · Zhejiang University
S-GRPO: Unified Post-Training for Large Vision-Language Models
Apr 17, 2026Yuming Yan, Kai Tang, Sihong Chen +4VLM Adaptation Group Relative Policy Optimization
Tencent Shenzhen, Guangdong, China
Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO
Apr 17, 2026Arash Ahmadi, Parisa Masnadi Khiabani, Sarah Sharif +4RL for Language Models Healthcare
School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, USA · Intelligent Neuromorphic and Quantum Understanding for Innovative Research and Engineering (INQUIRE) Laboratory, University of Oklahoma, Norman, OK, USA · Data Science and Analytics Institute, University of Oklahoma, Norman, OK, USA +3
Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization
Apr 15, 2026Junzhe Wang, Zhiheng Xi, Yajie Yang +4LLM Agents RL for LLM Agents
Fudan University · Shanghai Artificial Intelligence Laboratory · Shanghai Key Laboratory of Intelligent Information Processing
Balanced Aggregation: Understanding and Fixing Aggregation Bias in GRPO
Apr 14, 2026Zhiyuan Zeng, Jiameng Huang, Zhangyue Yin +8Reinforcement Learning RL for Language Model Reasoning
Fudan University · M-A-P · Peking University +3
Where Hindsight Credit Can Reside: A Signed-Capacity View of Token Updates in RLVR
Apr 13, 2026Yuhang He, Haodong Wu, Siyi Liu +7RL for Language Model Reasoning Token-Level Credit Assignment
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Ltd.
Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing
Apr 10, 2026Zhuohan Ouyang, Zhe Qian, Wenhuo Cui +1Text-Guided Image Editing Group Relative Policy Optimization
1South China Normal University · 2South China Agricultural University · 3Monash University
Skip-Connected Policy Optimization for Implicit Advantage
Apr 9, 2026Fengwei Teng, Jinyi Bai, Xinhao Yao +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Mohamed bin Zayed University of Artificial Intelligence · Renmin University of China · Carnegie Mellon University +1
OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks
Apr 9, 2026Wenbo Hu, Xin Chen, Yan Gao-Tian +3Multimodal Large Language Models Multimodal Reasoning
University of California, Los Angeles (UCLA)
MAR-GRPO: Stabilized GRPO for AR-diffusion Hybrid Image Generation
Apr 8, 2026Xiaoxiao Ma, Jiachen Lei, Tianfei Ren +6RL for Image Generation Autoregressive Diffusion
University of Science and Technology of China, Hefei, China · AMAP, Alibaba Group, Hangzhou, China · Zhejiang University, Hangzhou, China
Limits of Difficulty Scaling: Hard Samples Yield Diminishing Returns in GRPO-Tuned SLMs
Apr 7, 2026Suraj Yadav, Siddharth Yadav, Parth GoyalRL for Language Model Reasoning Small Language Models
IIIT Delhi
Asymmetric Advantage Modulation Calibrates Entropy Dynamics in RLVR
Apr 6, 2026Hengrui Gu, Xiaotian Han, Yujing Bian +2RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
North Carolina State University · Case Western Reserve University · Oak Ridge National Laboratory
Multi-Turn Reinforcement Learning for Tool-Calling Agents with Iterative Reward Calibration
Apr 3, 2026Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai +1RL for Tool Use Reinforcement Learning
Amity Research and Application Center (ARAC)
GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning
Apr 3, 2026Ornith Team, Xiaoya Li, Guoyin Wang +3RL for Code Generation Agentic RL
DeepReinforce Team
Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
Apr 2, 2026Gengsheng Li, Tianyu Yang, Junfeng Fang +6RL for Language Model Reasoning On-Policy Self-Distillation
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · National University of Singapore +2
Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training
Apr 2, 2026William Hoy, Binxu Wang, Xu PanEvolution Strategies RL Post-Training
Department of Industrial and Systems Engineering, University of Miami · Kempner Institute, Harvard University · Harvard University
ShapE-GRPO: Shapley-Enhanced Reward Allocation for Multi-Candidate LLM Training
Mar 31, 2026Rui Ai, Yu Pan, David Simchi-Levi +1RL Post-Training Reward Design for RL
MIT · University of Sydney · University of Texas at Dallas
MedLoc-R1: Performance-Aware Curriculum Reward Scheduling for GRPO-Based Medical Visual Grounding
Mar 30, 2026Guangjing Yang, Ziyuan Qin, Chaoran Zhang +6Medical Image Grounding Reward Shaping
Beijing University of Posts and Telecommunications · Emory University · Peking University +1
PAPO: Stabilizing Rubric Integration Training via Decoupled Advantage Normalization
Mar 27, 2026Zelin Tan, Zhouliang Yu, Bohan Lin +9RL for Language Model Reasoning Process Reward Models
University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · The Chinese University of Hong Kong +2
UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation
Mar 24, 2026Jie Liu, Zilyu Ye, Linxiao Yuan +8Image Generation RL for Generative Models
The Chinese University of Hong Kong · ByteDance Seed
Manifold-Aware Exploration for Reinforcement Learning in Video Generation
Mar 23, 2026Mingzhe Zheng, Weijie Kong, Yue Wu +9RL for Video Generation RL Exploration
HKUST · Tencent Hunyuan
Prompt replay: speeding up grpo with on-policy reuse of high-signal prompts
Mar 22, 2026Andrei Baroian, Rutger BergerRL Post-Training Group Relative Policy Optimization
Leiden Institute of Advanced Computer Science Leiden University
Mitigating Selection Bias in Large Language Models via Permutation-Aware GRPO
Mar 22, 2026Jinquan Zheng, Jia Yuan, Jiacheng Yao +3Group Relative Policy Optimization Permutation-Invariant Learning
School of Economics and Management, East China Normal University
Are complicated loss functions necessary for teaching LLMs to reason?
Mar 19, 2026Gabriele Carrino, Andrea Sassella, Nicolo Brunello +2RL for Language Model Reasoning Group Relative Policy Optimization
DEIB, Politecnico di Milano Via Ponzio 34/5, 20133, Milano (MI), Italy
TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas
Mar 17, 2026Ai Jian, Xiaoyun Zhang, Eryu Guo +6Schema Linking Group Relative Policy Optimization
Beijing University of Posts and Telecommunications, Beijing, China · State Key Lab of Processors, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences +1
GRPO and Reflection Reward for Mathematical Reasoning in Large Language Models
Mar 14, 2026Zhijie WangLLM Reasoning RL for Language Model Reasoning
School of Computer Science and Engineering, Sun Yat-sen University(SYSU), Guangzhou, China
MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learning
Mar 14, 2026Hongjun Wang, Wei Liu, Weibo Gu +2Policy Gradient Methods RL for Language Model Reasoning
The University of Hong Kong · Tencent Youtu Lab

Apr 20, 2026J/K move · Enter open · S save

Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data

Zhenwen Liang, Yujun Zhou, Sidi Lu, Xiangliang Zhang+2

Tencent AI Lab · University of Notre Dame

Reinforcement Learning RL for Language Model Reasoning Group Relative Policy Optimization RL Post-Training Mode Collapse

Open paper AskPDF arXiv

Reinforcement Learning (RL) enhances LLM reasoning, yet a paradox emerges as models scale: strong base models saturate standard benchmarks (e.g., MATH), yielding correct but homogeneous solutions. In such environments, the lack of failure cases causes the advantage signal in group-relative algorithms (e.g., GRPO) to vanish, driving policies into mode collapse. To address this, we propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration. Unlike standard sampling that follows model biases, CUTS flattens the local optimization landscape by sampling uniformly from constrained high-confidence candidates. We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance. Experiments on Qwen3 models demonstrate that our approach prevents policy degeneration and significantly boosts out-of-domain generalization. Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO, validating that maintaining diversity within the semantic manifold is critical for rigorous reasoning.