Group Relative Policy Optimization

Also known as GRPO

Follow this topic Ask about this topic

Latest papers 305

48 papers in the last four weeks, up 109% on the four weeks before. 0.3% of all new papers.

All topics

Can David Beat Goliath? On Multi-Hop Reasoning with Resource-Constrained Agents
Jan 29, 2026Hojae Han, Heeyun Jung, Jongyoon Kim +1RL for Language Model Reasoning Multi-Hop QA
ETRI · The University of Hong Kong · Seoul National University
GDEPO: Group Dual-dynamic and Equal-right Advantage Policy Optimization for Sample-Efficient Reinforcement Learning
Jan 11, 2026Zhengqing Yan, Fan Guo, Xinyang Liu +5Group Relative Policy Optimization Reinforcement Learning with Verifiable Rewards
State Key Laboratory of Engines, Tianjin University, 300354 Tianjin, China. · Artificial Intelligence Center, Cylingo Group, 100080 Beijing, China.
Rethinking Reward Signals in Video GRPO: When Scores Become Targets
Nov 24, 2025Rui Li, Yuanzhi Liang, Ziqi Ni +3Reward Hacking RL for Video Generation
University of Science and Technology of China, Hefei, China · Institute of Artificial Intelligence (TeleAI), China Telecom, Shanghai, China · Southeast University, Nanjing, China
TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
Aug 4, 2025Lei Pang, Jun Luo, Ruinan JinRL for Language Models Group Relative Policy Optimization
Peking University Beijing, China · The Ohio State University Columbus, USA · Shugu Yuntai Technology Co., Ltd. +1
MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE
Jul 29, 2025Junzhe Li, Yutao Cui, Tao Huang +8Diffusion Model Alignment Flow Matching
Hunyuan, Tencent, China · School of Computer Science, Peking University, China · Computer Center, Peking University, China +2

Jan 29, 2026J/K move · Enter open · S save

Can David Beat Goliath? On Multi-Hop Reasoning with Resource-Constrained Agents

Hojae Han, Heeyun Jung, Jongyoon Kim, Seung-won Hwang

ETRI · The University of Hong Kong · Seoul National University

RL for Language Model Reasoning Multi-Hop QA Sample-Efficient RL RL for IR Group Relative Policy Optimization

Open paper AskPDF arXiv

Reinforcement learning (RL) trains small language model agents to answer multi-hop questions by retrieving evidence over multiple turns, but reported gains typically rely on thousands of on-policy rollouts per update. We study RL for such agents on commodity GPUs, where each update samples only a few rollouts per question. Most sampled trajectories then retrieve none of the required evidence, so the outcome reward gives the policy little to learn from and small agents settle for answering without retrieval, a failure we call retrieval collapse. DAVID-GRPO addresses this with two mechanisms: (1) expert trajectory seeding places a handful of off-policy expert trajectories into the GRPO groups of the early updates, and (2) evidence-guided continuation rewards evidence coverage and resumes the most promising partial trajectory. The evidence for each training question is constructed from the corpus link graph, without annotated evidence. On six multi-hop QA benchmarks, DAVID-GRPO trained on four RTX 3090 GPUs with 144 rollouts per step brings Qwen2.5-1.5B to 22.6 average EM against 11.9 for the best baseline under the same budget, is comparable to Tree-GRPO trained with 20 times more rollouts, and, unlike the baselines that stop after at most one search, learns to retrieve across turns. The implementation is available at: https://github.com/AsadalJung/David-GRPO

Latest papers 305

Can David Beat Goliath? On Multi-Hop Reasoning with Resource-Constrained Agents

GDEPO: Group Dual-dynamic and Equal-right Advantage Policy Optimization for Sample-Efficient Reinforcement Learning

Rethinking Reward Signals in Video GRPO: When Scores Become Targets

TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback

MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE