Reward Design for RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 167

All topics

Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?
May 27, 2026Yibo Zhao, Zichen Ding, Jiayi Wu +2LLM Agents Information Retrieval
School of Data Science and Engineering, East China Normal University · Shanghai AI Laboratory
Knowing When to Ask: Segment-Level Credit Assignment for LLM Tool Use
May 27, 2026Abhijit Kumar, Zoey Wu, Mohit SuleyTool-Augmented Language Model Agents Learning to Defer
Microsoft AI Redmond, WA
Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generation
May 26, 2026Zixuan Yang, Yiqun Chen, Wei Yang +7Open-Ended Generation Reward Modeling
Renmin University of China · University of Southern California · Zhejiang University +1
Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards
May 26, 2026Yu Huang, Zihua Zhao, Zhaoxin Huan +9Multi-Objective Reinforcement Learning RL for Language Models
Shanghai Jiao Tong University · Ant Group
RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents
May 25, 2026Mingchen Li, Hansi Zeng, Zhuo Qian +3RL for Language Model Reasoning Credit Assignment in RL
University of Massachusetts, Amherst · Texas Tech University · University of Connecticut
The Behavioral Credibility Trilemma: When Calibrated Autonomy Becomes Impossible
May 25, 2026Lauri Lovén, Nam Do, Hassan Mehmood +2AI Agent Reliability AI Agent Governance
Future Computing Group University of Oulu Oulu, Finland · Department of Computer Science University of Helsinki Helsinki, Finland
From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
May 22, 2026Christian Gumbsch, Leonardo Barcellona, Lennard Schünemann +7RL for Robotics Multimodal Reward Modeling
University of Amsterdam · 2Catholic University of Leuven · 3Toyota Research Institute +1
ARMS: Automatic Reward Shaping for Sparse-Reward Multi-Agent Reinforcement Learning
May 22, 2026Elie Abboud, Oren GalMulti-Agent System Optimization Sparse-Reward RL
Department of Marine Technologies University of Haifa
ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning
May 22, 2026Xiaoyuan Li, Keqin Bao, Moxin Li +5RL for Language Models Reinforcement Learning
University of Science and Technology of China · Alibaba Group · National University of Singapore
From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning
May 22, 2026Ranxu zhang, zeyang li, Jiacheng Huang +5Agentic RL Reinforcement Learning
University of Science and Technology of China · Alibaba Group
Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework
May 21, 2026Shourov Joarder, Diganta Sikdar, Ahsan Habib Akash +2Multi-Objective Reinforcement Learning Intrinsic Reward Methods
Bangladesh University of Engineering and Technology, Bangladesh · West Virginia University, USA · University of Aberdeen, UK +2
CIG: Exploration via Conditional Information Gain
May 20, 2026Tim Joseph, Marcus Fechner, Philipp Stegmaier +2Intrinsic Reward Methods Intrinsic Motivation
FZI · KIT
Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR
May 19, 2026Utkarsh Tyagi, Xingang Guo, MohammadHossein Rezaei +5Reward Modeling Rubric-Based RL
Scale AI · Persona
ARC-RL: A Reinforcement Learning Playground Inspired by ARC Raiders
May 19, 2026Carlo Romeo, Andrew D. BagdanovRL Benchmarks Robotics Simulation
Media Integration and Communication Center – University of Florence
AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning
May 18, 2026Peilin Wu, Xinlu Zhang, Kun Wan +4RL for Language Models Reinforcement Learning
The University of Texas at Dallas · Department of Computer Science, University of California, Santa Barbara · Adobe Inc.
Pairwise Preference Reward and Group-Based Diversity Enhancement for Superior Open-Ended Generation
May 18, 2026Guining Cao, Jiaxin Peng, Chu Zeng +3Pairwise Preference Learning Open-Ended Generation
Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · School of Software and Microelectronics, Peking University · Tsinghua University
Step-wise Rubric Rewards for LLM Reasoning
May 17, 2026Weichu Xie, Haozhe Zhao, Wenpu Liu +15Reward Modeling Credit Assignment in RL
Peking University · JD Explore Academy · Shanghai Jiao Tong University +2
TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition
May 16, 2026Anay Kulkarni, ChiaEn Lu, Dheeraj Mekala +3Reward Modeling Reinforcement Learning with Verifiable Rewards
UC San Diego · Cisco Research
Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
May 13, 2026Yang Bai, Kaiyuan Liu, Ziyuan Zhuang +5Multi-Objective Reinforcement Learning RL for Language Models
Meituan, China
ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization
May 12, 2026Nirmal Patel, Fei Wang, Inderjit S. DhillonRL for Language Models Reinforcement Learning
University of Texas at Austin · Google
Reward Hacking in Rubric-Based Reinforcement Learning
May 12, 2026Anas Mahmoud, MohammadHossein Rezaei, Zihao Wang +3Reward Hacking Reinforcement Learning
Scale AI
EvoNav: Evolutionary Reward Function Design for Robot Navigation with Large Language Models
May 12, 2026Zhikai Zhao, Chuanbo Hua, Federico Berto +4Evolutionary Optimization Robot Navigation
1KAIST · 2Radical Numerics · 3UC Riverside +1
GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation
May 12, 2026Sijia Li, Yuchen Huang, Zifan Liu +7RL for Language Model Reasoning Credit Assignment in RL
Hong Kong University of Science and Technology · Microsoft Research Asia
Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning
May 8, 2026Manish Bhattarai, Ismael Boureima, Nishath Rajiv Ranasinghe +2Reinforcement Learning RL for Language Model Reasoning
Los Alamos National Laboratory, Los Alamos, NM, USA.
ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression
May 8, 2026Tingcheng Bian, Yuzhe Zhang, Jing Jin +5Reward Shaping RL for Language Model Reasoning
1Baidu Inc. · 2Shenzhen University · 3Peking University +2
MARBLE: Multi-Aspect Reward Balance for Diffusion RL
May 7, 2026Canyu Zhao, Hao Chen, Yunze Tong +3Multi-Objective Reinforcement Learning Diffusion Model Alignment
Zhejiang University · Work done during an internship at HiThink. · HiThink +1
Reward Shaping and Action Masking for Compositional Tasks using Behavior Trees and LLMs
May 7, 2026Nicholas Potteiger, Ankita Samaddar, Taylor T. Johnson +1Reinforcement Learning Reward Shaping
Vanderbilt University, Nashville, TN
RVPO: Risk-Sensitive Alignment via Variance Regularization
May 7, 2026Ivan Montero, Tomasz Jurczyk, Bhuwan DhingraRL for Language Models Risk-Sensitive RL
Apple
Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards
May 5, 2026Tianyang Han, Hengyu Shi, Junjie Hu +3RL for Code Generation Process Reward Models
1D4 Lab · 2Independent Researcher
Discovering Reinforcement Learning Interfaces with Large Language Models
May 5, 2026Akshat Singh Jaswal, Ashish Baghel, Paras ChopraReinforcement Learning LLM-Guided RL
Lossfunk

May 27, 2026, cs.CLJ/K move · Enter open · S save

Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?

Yibo Zhao, Zichen Ding, Jiayi Wu, Zun Wang+1

School of Data Science and Engineering, East China Normal University · Shanghai AI Laboratory

LLM Agents Information Retrieval LLM Agent Training Reward Design for RL RL for LLM Agents

Open paper AskPDF arXiv

Search agents powered by large language models can autonomously decompose queries, retrieve information, and synthesize answers through multi-step reasoning. However, the rapid growth of training methods has outpaced controlled comparison: existing works differ in retrieval corpora, reward designs, and training protocols, making it unclear what actually drives improvements. We present a controlled empirical study that isolates three under-explored dimensions of search agent training. First, we identify a critical data-coverage issue in the widely used Wikipedia 2018 corpus and show that correcting it alone yields larger gains than the differences between training algorithms. Second, we systematically compare outcome-based and process-based reward methods across three base models, finding that the simplest outcome-based approach achieves competitive or superior performance in most settings, and that process-level credit assignment can over-correct agent behavior. Third, we analyze training data diversity, off-policy data utilization, and search budget scaling, distilling practical guidelines for training effective search agents. Our code is available at https://github.com/YiboZhao624/SearchAgentReview.

Momentum

Related topics

Latest papers 167