Group Relative Policy Optimization

Also known as GRPO

Group Relative Policy Optimization studies how to improve a model or agent by sampling several responses to the same prompt and using their relative rewards to guide policy updates. Its critic-free, group-normalized approach is widely used for reinforcement learning with verifiable rewards in reasoning tasks such as GSM8K and MATH, and has been adapted to tool-use agents, vision-language models, and diffusion-based image and video generation. Current methods refine reward design and credit assignment, adjust sampling and curricula, and modify normalization, clipping, or entropy controls to improve learning efficiency and stability. A central challenge is that sparse or uniform group rewards can provide little or misleading signal, while choices such as response-length normalization and diversity control can trade off update reliability, exploration, and performance.

Latest papers

343 papers53 papers in the last four weeks, up 33% on the four weeks before. 0.3% of all new papers.

MedAD-R1: Consistency-Reinforced Policy Optimization for Interpretable Medical Anomaly Detection
Feb 1, 2026Haitao Zhang, Yingying Wang, Jiaxiang Wang +5Healthcare Group Relative Policy Optimization
School of Informatics, Xiamen University · Institute of Artificial Intelligence, Xiamen University · Zhejiang Expressway Co., Ltd. +3
Why GRPO Needs Normalization: A Local-Curvature Perspective on Adaptive Gradients
Jan 30, 2026Cheng Ge, Caitlyn Heqi Yin, Hao Liang +1RL for Language Model Reasoning Group Relative Policy Optimization
Department of Aeronautics and Astronautics, MIT · Department of Statistics, University of Wisconsin–Madison · Department of Informatics, King’s College London +1
Can David Beat Goliath? On Multi-Hop Reasoning with Resource-Constrained Agents
Jan 29, 2026Hojae Han, Heeyun Jung, Jongyoon Kim +1RL for Language Model Reasoning Multi-Hop QA
ETRI · The University of Hong Kong · Seoul National University
Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling
Jan 29, 2026Abhijeet Sinha, Sundari Elango, Dianbo LiuReinforcement Learning Behavioral Diversity in RL
National University of Singapore, Singapore · IIT Madras, India
Your Group-Relative Advantage Is Biased
Jan 13, 2026Fengkai Yang, Zherui Chen, Xiaohan Wang +10RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Beihang University · Peking University · Meituan +1
GDEPO: Group Dual-dynamic and Equal-right Advantage Policy Optimization for Sample-Efficient Reinforcement Learning
Jan 11, 2026Zhengqing Yan, Fan Guo, Xinyang Liu +5Group Relative Policy Optimization Reinforcement Learning with Verifiable Rewards
State Key Laboratory of Engines, Tianjin University, 300354 Tianjin, China. · Artificial Intelligence Center, Cylingo Group, 100080 Beijing, China.
Rethinking Reward Signals in Video GRPO: When Scores Become Targets
Nov 24, 2025Rui Li, Yuanzhi Liang, Ziqi Ni +3Reward Hacking RL for Video Generation
University of Science and Technology of China, Hefei, China · Institute of Artificial Intelligence (TeleAI), China Telecom, Shanghai, China · Southeast University, Nanjing, China
SCOPE-RL: Stable and Quantitative Control of Policy Entropy in RL Post-Training
Oct 9, 2025Chen Wang, Zhaochun Li, Jionghao Bai +4Entropy Collapse in RL Reinforcement Learning
College of Software, Nankai University · Zhongguancun Academy · Beijing Institute of Technology +2
Group Contrastive Policy Optimization: Learning Beyond the Self-Exploration Limit with Golden Answers
Oct 9, 2025Hao Wu, Wei Liu, Shizhe Xiang +3RL for Language Model Reasoning LLM Reasoning
Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse
Sep 30, 2025Yuheng Zhang, Wenlin Yao, Changlong Yu +5RL for Language Models Reinforcement Learning with Verifiable Rewards
University of Illinois Urbana-Champaign · Amazon
TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
Aug 4, 2025Lei Pang, Jun Luo, Ruinan JinImportance Sampling for RL RL for Language Models
Peking University Beijing, China · The Ohio State University Columbus, USA · Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, UAE
MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE
Jul 29, 2025Junzhe Li, Yutao Cui, Tao Huang +8Diffusion Model Alignment Flow Matching
Hunyuan, Tencent, China · School of Computer Science, Peking University, China · Computer Center, Peking University, China +2
HiMA-Ecom: A Long-Horizon Memory Benchmark for Hierarchical Multi-Agent E-commerce Assistants
Jun 24, 2025Junxing Hu, Ai Han, Haolan Zhan +8Hierarchical MARL Long-Horizon Agent Evaluation
JD Retail, JD.com, Inc., Beijing, 101111, China · Faculty of Engineering and Information Technology, The University of Melbourne, Parkville, Victoria, 3082, Australia · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China

Feb 1, 2026J/K move · Enter open · S save

MedAD-R1: Consistency-Reinforced Policy Optimization for Interpretable Medical Anomaly Detection

Haitao Zhang, Yingying Wang, Jiaxiang Wang, Haote Xu+4

School of Informatics, Xiamen University · Institute of Artificial Intelligence, Xiamen University · Zhejiang Expressway Co., Ltd.+3

Healthcare Group Relative Policy Optimization Medical Image Anomaly Detection Multimodal Large Language Models Medical Visual Question Answering Multimodal Anomaly Detection+2

Open paper AskPDF arXiv

Medical Anomaly Detection (MedAD) offers a promising direction for medical image analysis with Large Multimodal Models (LMMs). However, progress is limited by fragmented datasets and the tendency of Supervised Fine-Tuning (SFT) to learn superficial image-text correlations rather than verifiable diagnostic reasoning. Consequently, current models often generate fluent explanations that are either insufficiently grounded in the image or inconsistent with their final answers, limiting their reliability in high-stakes medical applications. To address these issues, we introduce MedAD-38K, a large-scale, multimodal, and multicenter benchmark containing structured Visual Question Answering pairs and quality-controlled diagnostic Chain-of-Thought annotations across five core MedAD tasks. Based on this benchmark, we propose a two-stage framework. Cognitive Injection first uses SFT to inject domain-specific medical knowledge and establish a structured think-then-answer format. Consistency Group Relative Policy Optimization (Con-GRPO) then employs an Evidence-Aware Consistency Reward to reinforce reasoning that remains grounded in the image and logically supports the final answer. The resulting MedAD-R1 achieves state-of-the-art performance on MedAD-38K and consistently outperforms all evaluated baselines across five source-disjoint external datasets spanning diverse imaging modalities. Beyond accuracy, it achieves higher reasoning-answer consistency and visual-grounding scores across multiple backbones. With only 0.8B parameters, MedAD-R1 offers practical potential for resource-constrained deployment. These results demonstrate the generality of evidence-aware consistency optimization for interpretable MedAD. Project resources are available at https://github.com/zhtstar/MedAD-R1.