Group Relative Policy Optimization

Also known as GRPO

Follow this topic Ask about this topic

Latest papers 288

40 papers in the last four weeks, up 82% on the four weeks before. 0.3% of all new papers.

All topics

Matched-Context Group Policy Optimization for LLM Agent Training
Oct 9, 2026Chenxu Zhang, Yaorui Shi, Wenlong Zhang +3RL for LLM Agents Preference-Based RL
Imperial College London · Meituan · University of Science and Technology of China +1
A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO
Oct 9, 2026Ronghan Li, Fei Chen, Mingyu LuCurriculum RL Curriculum Learning
College of Artificial Intelligence, Dalian Maritime University, Dalian, 116026, Liaoning, China. · Information Science and Technology College, Dalian Maritime University, Dalian, 116026, Liaoning, China.
AdaC-GRPO: Towards Mastering Long-Horizon Visual Sudoku Puzzles via Adaptive Curriculum Learning
Oct 9, 2026Sida Li, Yuanxin Liu, Linli Yao +4VLM Reasoning RL for VLMs
National Key Laboratory for Multimedia Information Processing, Peking University · Huawei Technologies Co., Ltd.
When KL Regularization Misfires in Group Policy Optimization
Oct 8, 2026Fei DingKL-Regularized RL Group Relative Policy Optimization
Alibaba Group
GRPODropout: Less is More for Online Reinforcement Learning Rollouts
Oct 8, 2026Hexuan Deng, Zihao Yan, Xuebo Liu +8Reinforcement Learning RL for Language Model Reasoning
Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen) · Beijing Zhongguancun Academy · XinzhuAI
Fed-GRPO: Reward-Signal-Driven Federated Group Relative Policy Optimization
Oct 8, 2026Pengxin Guo, Shuang Zeng, Zonggen Li +3Federated RL RL for Language Model Reasoning
The University of Hong Kong · Sun Yat-sen University
AdaptEvo: Adaptive Agent Learning with Evolving Supervision
Oct 8, 2026Shijun Wan, Jiancong Xie, Hang Xu +7Reinforcement Learning Self-Evolving Agents
Fudan University · Sun Yat-sen University · Xiaohongshu
SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages
Oct 7, 2026Raja Kumar, Rajat Koner, Ritwick Chaudhry +3VLM Reasoning RL for VLMs
University of Southern California · Amazon AGI
BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
Oct 7, 2026Yingxiang Yang, Weihang Xiao, Zhunxuan Wang +2RL for Language Model Reasoning Group Relative Policy Optimization
Amazon AGI · Virginia Tech
CERO: Where and When to Allocate Rollouts for RL Post-Training
Oct 7, 2026Yiming Zong, Yige Wang, Xing Hu +2Reinforcement Learning RL Post-Training
Department of Industrial Engineering & Decision Analytics, Hong Kong University of Science and Technology · Innovation and Information Management, The University of Hong Kong · Faculty of Engineering & Faculty of Business and Economics, The University of Hong Kong
RELACE: retrospective likelihood-based action credit estimation for long-horizon language agents
Oct 5, 2026Sayak Chakrabarti, Sathish Reddy IndurthiGroup Relative Policy Optimization Credit Assignment in RL
Columbia University · Zoom Communications, Inc.
Prompt Dominance and Asymmetric Verifier Costs: Empirical Ablations of GRPO at 1B Scale on GSM8K
Oct 4, 2026Yi HouRL for Language Model Reasoning Group Relative Policy Optimization
University of Chinese Academy of Sciences Beijing, China
DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation
Oct 3, 2026Karn Tiwari, Varnith Chordia, Prathosh A POn-Policy Distillation RL for Language Model Reasoning
Indian Institute of Science, Bengaluru, India · Snap Inc.
PB-GRPO: Learning Socially Adaptive LLM Agents from Persona-Driven Simulation with Preference-Batched GRPO
Oct 2, 2026Jingquan Wang, Jun Yin, Xu Han +3LLM Agent Training Group Relative Policy Optimization
Amazon
Exploration-Preserving Policy Optimization
Oct 2, 2026Hangzhan jin, Mohammad Hamdaqa, Doina PrecupRL for Language Model Reasoning RL Exploration
Mila - Quebec AI Institute · Polytechnique Montr´eal · McGill University +2
Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hacking
Oct 2, 2026Pawan Prakash, Philipp Höllmer, Addis Fuhr +4Crystal Structure Generation Reward Hacking
University of Florida · New York University · Oak Ridge National Laboratory
Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
Oct 2, 2026Yu Li, Guangfeng Cai, Long-Fei Li +5Credit Assignment Step-Level Credit Assignment in RL
School of Computer Science and Engineering, Southeast University, Nanjing, China · Huawei Noah’s Ark Lab, China
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Oct 1, 2026Qijia He, Ruinan Jin, Jun Luo +2Asynchronous RL RL for Language Model Reasoning
The Ohio State University · Arizona State University
Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals
Oct 1, 2026Ryunyi Lee, Kangjun Noh, Somin Kim +2Reward Modeling LLM-as-a-Judge
Yonsei University
SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning
Sep 30, 2026Xinchen Du, Zhengze Zhou, Wenhui Zhu +4Agentic RL Group Relative Policy Optimization
LinkedIn Corporation · Georgia Institute of Technology
Exploring More, Reasoning Better: Stepwise Risk-Sensitive GRPO for Diffusion Language Models
Sep 30, 2026Yue Yu, Bowen Zuo, David Crandall +2RL for Language Model Reasoning Group Relative Policy Optimization
Indiana University · University of California, Riverside
Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Construction
Sep 30, 2026Xiao Cui, Mo Zhu, Yulei Qin +3Group Relative Policy Optimization Language Model Distillation
University of Science and Technology of China · Zhejiang University · Independent Researcher
GRPO Training Dynamics for Small Language Models
Sep 30, 2026Rajat Ghosh, Vaishnavi Bhargava, Henry Wong +2RL for Language Model Reasoning Group Relative Policy Optimization
Nutanix
Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO
Sep 29, 2026Mei Okonkwo, Pixel Nomand, Julian Berg +5Group Relative Policy Optimization RL for Language Model Reasoning
University of Wisconsin–Madison · University of Washington
Cool the Sampler, Not the Learner: Sampling Temperature Moves the Staleness Cliff of Importance-Corrected GRPO
Sep 29, 2026Taiheng PanRL for Language Models Reinforcement Learning
The University of Melbourne
Dual-Channel Robust Group-Relative Policy Optimization via Advantage and Sequence-Weight Estimation
Sep 29, 2026Zhongyi Li, Wan Tian, Xiang Xu +4RL for Language Model Reasoning Group Relative Policy Optimization
Beihang University · Peking University · Nanjing University
Learn from the Gap: Differential-Aware Advantage Pruning with Adaptive Rollout Sampling for GRPO
Sep 29, 2026Jiahua Yang, Zhiwei Yang, Xianpeng Zhang +7Group Relative Policy Optimization RL for VLMs
Guangdong Institute of Smart Education, Jinan University, Guangzhou, China · OPPO AI Center, Shenzhen, China · Ragentile Intelligence Inc, Edmonton, Canada
STAR-GRPO: Canonical Anchoring and Reliability-First Advantages against Representation-Dependent Reward Hacking
Sep 29, 2026Wan Tian, Zhongyi Li, Xiang Xu +3Reward Hacking Group Relative Policy Optimization
Peking University · Beihang University · Nanjing University
CorrGRPO: Correlation-Normalized GRPO for Multi-Reward Learning
Sep 29, 2026Wenbin Hu, Huihao Jing, Haochen Shi +3Multi-Objective Reinforcement Learning RL for Language Models
Hong Kong University of Science and Technology
Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning
Sep 28, 2026Dongwon Jung, Hemanth Neelgund Ramesh, Yifan Wang +7Agentic RL Group Relative Policy Optimization
University of California, Davis · Microsoft · University of Washington

Oct 9, 2026J/K move · Enter open · S save

Matched-Context Group Policy Optimization for LLM Agent Training

Chenxu Zhang, Yaorui Shi, Wenlong Zhang, Jianchao Tan+2

Imperial College London · Meituan · University of Science and Technology of China+1

RL for LLM Agents Preference-Based RL Group Relative Policy Optimization Credit Assignment in RL

Open paper AskPDF arXiv

Training long-horizon language model agents requires assigning credit to intermediate decisions from sparse terminal feedback. Within a finite group of sibling rollouts, useful action comparisons depend on their context, evidence for both alternatives, and how that evidence enters learning. We propose Matched-Context Group Policy Optimization (MCGPO), a procedure for constructing weighted action preferences from already collected trajectories. MCGPO defines comparison contexts with pre-action state summaries and observed rubric progress, annotated by task rules. Two-action support selects among ordered exact and coarse matching rules. Fixed folds separate candidate decisions from the terminal outcomes used to orient their preferences, and support-dependent evidence weights scale an auxiliary pairwise objective alongside shared GraphGPO task learning. MCGPO reuses rollout data and each response's original history, preserving critic-free training. Across six environment-backbone settings with three training seeds each, MCGPO improves mean success over matched GraphGPO: by 0.47-1.46 percentage points on ALFWorld and WebShop with Qwen2.5-1.5B and 7B, and by 3.96 and 5.16 points on WebShop with Qwen3-4B and 8B. The ALFWorld gains build on baseline success above 92%, where remaining headroom is limited. Integrated success improves in three of four Qwen2.5 settings. Three-seed factorial controls support opposite-fold evidence and evidence weighting; joint state-rubric matching achieves the highest mean in all four representation-control settings.