Group Relative Policy Optimization

Also known as GRPO

Follow this topic Ask about this topic

Latest papers 305

48 papers in the last four weeks, up 109% on the four weeks before. 0.3% of all new papers.

All topics

When Right Meets Wrong: Bilateral Context Conditioning with Reward-Confidence Correction for GRPO
Mar 13, 2026Yu Li, Tian Lan, Zhengling QiRL for Language Model Reasoning Group Relative Policy Optimization
Department of Electrical and Computer Engineering, George Washington University · School of Business, George Washington University
From Sparse to Dense: Multi-View GRPO for Flow Models via Augmented Condition Space
Mar 13, 2026Jiazi Bu, Pengyang Ling, Yujie Zhou +8RL for Image Generation Group Relative Policy Optimization
Shanghai Jiao Tong University · S-Lab, Nanyang Technological University · Shanghai AI Laboratory +7
From Broad Exploration to Stable Synthesis: Entropy-Guided Optimization for Autoregressive Image Generation
Mar 12, 2026Han Song, Yucheng Zhou, Jianbing Shen +1Image Generation Autoregressive Image Generation
The Chinese University of Hong Kong · University of Macau
Resonate: Reinforcing Text-to-Audio Generation via Online Feedback from Large Audio Language Models
Mar 12, 2026Xiquan Li, Junxi Liu, Wenxi Chen +3Text-to-Audio Generation Group Relative Policy Optimization
X-LANCE Lab, Shanghai Jiao Tong University, China · SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University, China
Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization
Mar 10, 2026Ming Nie, Chunwei Wang, Jianhua Han +2Multimodal Generation Unified Multimodal Models
School of Data Science, Fudan University · Noah’s Ark Lab, Huawei · Yinwang Intelligent Technology Co., Ltd.
Improving Search Agent with One Line of Code
Mar 10, 2026Jian Li, Dongsheng Chen, Zhenhua Xu +5RL for Tool Use Group Relative Policy Optimization
Nanjing University · Tencent YoutuLab
Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
Mar 8, 2026Zongqian Li, Shaohan Huang, Zewen Chi +5RL for Code Generation LLM Code Generation
Microsoft Research · University of Cambridge · Princeton University
Stabilizing Reinforcement Learning for Diffusion Language Models
Mar 6, 2026Jianyuan Zhong, Kaibo Wang, Ding Ding +5RL for Diffusion Models Reinforcement Learning
Huawei Foundation Model Department · The Chinese University of Hong Kong · The Hong Kong University of Science and Technology
Unbiased Dynamic Pruning for Efficient Group-Based Policy Optimization
Mar 4, 2026Haodong Zhu, Yangyang Ren, Yanjing Li +6Reinforcement Learning RL for Language Model Reasoning
Beihang University · Zhongguancun Academy · Rakuten Singapore +3
From Shallow to Deep: Pinning Semantic Intent via Causal GRPO
Mar 3, 2026Shuyi Zhou, Zeen Song, Wenwen Qiang +4LLM Alignment Adversarial Robustness of Language Models
University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China
Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generation
Mar 2, 2026Yi Gu, Yanqing Liu, Chen Yang +1Text-to-Audio Generation RL for Diffusion Models
Tsinghua University · Microsoft
Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
Mar 1, 2026Hongyi Zhou, Kai Ye, Erhan Xu +4Group Relative Policy Optimization Policy Gradient Methods
Department of Mathematics, Tsinghua University · Department of Statistics, London School of Economics and Political Science · School of Mathematics, University of Birmingham +2
DIVA-GRPO: Enhancing Multimodal Reasoning through Difficulty-Adaptive Variant Advantage
Mar 1, 2026Haowen Gao, Zhenyu Zhang, Liang Pang +7Group Relative Policy Optimization RL for VLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Kuaishou Technology, Beijing, China
Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion
Feb 26, 2026Zhen Tan, Chengshuai Zhao, Song Wang +3Language Model Distillation Reasoning Distillation
Arizona State University · University of Virginia · University of North Carolina at Chapel Hill
Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks
Feb 26, 2026Shuo He, Lang Feng, Qi Wei +3Agentic RL RL for LLM Agents
Nanyang Technological University · Southeast University
Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization
Feb 25, 2026Jinghan Li, Junfeng Fang, Jinda Lu +5Multimodal Large Language Models Multimodal Reasoning
University of Science and Technology of China · National University of Singapore
Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design
Feb 25, 2026Haoxiang Sun, Tao Wang, Chenwei Tang +2Large Vision-Language Models RL for VLMs
College of Computer Science, Sichuan University · School of Electronic and Computer Engineering, Peking University
NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning
Feb 24, 2026Ishaan Rawal, Shubh Gupta, Yihan Hu +1Vision-Language-Action Models Autonomous Driving
Applied Intuition · Texas A&M University · UC Berkeley
Overton Pluralistic Reinforcement Learning for Large Language Models
Feb 24, 2026Yu Fu, Seongho Son, Ilija BogunovicRL for Language Models Multi-Objective Language Model Alignment
University College London · University of Basel
Smooth Gate Functions for Soft Advantage Policy Optimization
Feb 22, 2026Egor Denisov, Svetlana Glazyrina, Maksim Kryzhanovskiy +1Policy Optimization RL for Language Model Reasoning
Lomonosov Moscow State University, Moscow, Russia · Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia
Proximity-Based Multi-Turn Optimization: Practical Credit Assignment for LLM Agent Training
Feb 22, 2026Yangyi Fang, Jiaye Lin, Xiaoliang Fu +4LLM Agent Training RL for LLM Agents
Tsinghua University · Fudan University · Peking University +3
Advances in GRPO for Generation Models: A Survey
Feb 21, 2026Zexiang Liu, Xianglong He, Yangguang LiRL for Generative Models Group Relative Policy Optimization
SJTU · THU · CUHK
WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
Feb 19, 2026Gagan Mundada, Zihan Huang, Rohan Surana +8Efficient Language Model Reasoning RL for Language Model Reasoning
University of California, San Diego · University of Toronto · 3Adobe Research +2
Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
Feb 17, 2026Jialu Wang, Heinrich Peters, Asad A. Butt +6LLM Alignment Group Relative Policy Optimization
1Apple Inc., California, United States.
Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems
Feb 9, 2026Lang Feng, Longtao Zheng, Shuo He +2Multi-Agent LLM Systems Multi-Agent System Optimization
Nanyang Technological University, Singapore
F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare
Feb 6, 2026Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov +4Reinforcement Learning RL for Language Model Reasoning
T-Tech · Saint Petersburg Electrotechnical University “LETI”
Constrained Group Relative Policy Optimization
Feb 5, 2026Roger Girgis, Rodrigue de Schaetzen, Luke Rowe +3Group Relative Policy Optimization Constrained RL
Mila – Quebec AI Institute Université de Montréal
Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation
Feb 5, 2026Zhiqi Yu, Zhangquan Chen, Mengting Liu +2RL for Language Model Reasoning Group Relative Policy Optimization
University of Hong Kong · Tsinghua University · Sun Yat-sen University
MedAD-R1: Consistency-Reinforced Policy Optimization for Interpretable Medical Anomaly Detection
Feb 1, 2026Haitao Zhang, Yingying Wang, Jiaxiang Wang +5Healthcare Group Relative Policy Optimization
School of Informatics, Xiamen University · Institute of Artificial Intelligence, Xiamen University · Zhejiang Expressway Co., Ltd. +3
Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients
Jan 30, 2026Cheng Ge, Caitlyn Heqi Yin, Hao Liang +1RL for Language Model Reasoning Group Relative Policy Optimization
Department of Aeronautics and Astronautics, MIT · Department of Statistics, University of Wisconsin–Madison · Department of Informatics, King’s College London +1

Mar 13, 2026J/K move · Enter open · S save

When Right Meets Wrong: Bilateral Context Conditioning with Reward-Confidence Correction for GRPO

Yu Li, Tian Lan, Zhengling Qi

Department of Electrical and Computer Engineering, George Washington University · School of Business, George Washington University

RL for Language Model Reasoning Group Relative Policy Optimization LLM Reasoning Contrastive RL

Open paper AskPDF arXiv

Group Relative Policy Optimization (GRPO) has emerged as an effective method for training reasoning models. While it computes advantages based on group mean, GRPO treats each output as an independent sample during the optimization and overlooks a vital structural signal: the natural contrast between correct and incorrect solutions within the same group, thus ignoring the rich, comparative data that could be leveraged by explicitly pitting successful reasoning traces against failed ones. To capitalize on this, we present a contrastive reformulation of GRPO, showing that the GRPO objective implicitly maximizes the margin between the policy ratios of correct and incorrect samples. Building on this insight, we propose Bilateral Context Conditioning (BICC), a mechanism that allows the model to cross-reference successful and failed reasoning traces during the optimization, enabling a direct information flow across samples. We further introduce Reward-Confidence Correction (RCC) to stabilize training by dynamically adjusts the advantage baseline in GRPO using reward-confidence covariance derived from the first-order approximation of the variance-minimizing estimator. Both mechanisms require no additional sampling or auxiliary models and can be adapted to all GRPO variants. Experiments on mathematical reasoning benchmarks demonstrate consistent improvements across comprehensive models and algorithms. Code is available at https://github.com/Skylanding/BiCC.