RL for IR

RL: Reinforcement Learning · IR: Information Retrieval

Reinforcement learning for information retrieval studies how systems can learn what to retrieve, how to rank it, and when to stop using feedback from downstream tasks or user interactions. Approaches include policy-gradient methods such as PPO, GRPO, and REINFORCE, offline methods such as CQL and IQL, and joint training of search agents, retrievers, and rankers. Work spans multi-hop question answering and agentic search, retrieval benchmarks such as BEIR and BRIGHT, and industrial settings including e-commerce search and recommendation. A central challenge is assigning useful credit when rewards are delayed or sparse: intermediate evidence and process rewards can guide learning, but may be noisy, invite reward hacking, or trade off against retrieval cost and generalization.

Latest papers

53 papers21 papers in the last four weeks, up 320% on the four weeks before. 0.1% of all new papers.

GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning
Aug 23, 2026Jun Chen, Yongchao Liu, Pengyu Qiu +6Multi-Hop QA Agentic RAG
1Shenzhen University, Shenzhen, China · 2Great Bay University, Dongguan, China · 3Ant Group, Hangzhou, China +1
Harness-G: A Graph-Structured Harness for Search Agents
Jul 30, 2026Yanning Hou, Haoyuan Chen, Sihang Zhou +7RL for LLM Agents Credit Assignment in RL
National University of Defense Technology · Changsha, China
ToE: A Hierarchical and Explainable Claim Verification Framework with Dynamic Multi-source Evidence Retrieval and Aggregation
Jun 26, 2026Zhaoqi Wang, Zijian Zhang, Kun Zheng +4Evidence-Grounded Reasoning Automated Fact-Checking
School of Cyberspace Science and Technology, Beijing Institute of Technology · TravelSky Technology Limited · School of Computer Science, University of Auckland
Understanding the Behaviors of Environment-aware Information Retrieval
Jun 15, 2026Ruifeng Yuan, Chaohao Yuan, David Dai +4Retrieval-Augmented Generation Reinforcement Learning
1Fudan University · 2Alibaba DAMO Academy · 3Chinese University of Hong Kong +2
RL-Index: Reinforcement Learning for Retrieval Index Reasoning
Jun 15, 2026Yongjia Lei, Nedim Lipka, Zhisheng Qi +5Reinforcement Learning Agentic Retrieval
♡University of Oregon · ♣Adobe Research
Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses
Jun 1, 2026Pengcheng Jiang, Zhiyi Shi, Kelly Hong +5Reinforcement Learning Agentic Retrieval
University of Illinois at Urbana-Champaign · †UC Berkeley · ✧Chroma
RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents
May 25, 2026Mingchen Li, Hansi Zeng, Zhuo Qian +3RL for Language Model Reasoning Credit Assignment in RL
University of Massachusetts, Amherst · Texas Tech University · University of Connecticut
CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search
Apr 19, 2026Hansi Zeng, Liam Collins, Bhuvesh Kumar +2Reinforcement Learning Learning to Rank
Center for Intelligent Information Retrieval, University of Massachusetts Amherst · Snap Inc.
Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization
Apr 15, 2026Junzhe Wang, Zhiheng Xi, Yajie Yang +4LLM Agents RL for LLM Agents
Fudan University · Shanghai Artificial Intelligence Laboratory · Shanghai Key Laboratory of Intelligent Information Processing
Cycle-Consistent Search: Question Reconstructability as a Proxy Reward for Search Agent Training
Apr 14, 2026Sohyun An, Shuibenyang Yuan, Hayeon Lee +2Unsupervised RL Cycle-Consistent Learning
Meta Superintelligence Labs · UCLA
Task-Adaptive Retrieval over Agentic Multi-Modal Web Histories via Learned Graph Memory
Apr 9, 2026Saman Forouzandeh, Kamal Berahmand, Mahdi JaliliAdaptive Retrieval Multimodal IR
School of Engineering, Royal Melbourne Institute of Technology University, Melbourne, Australia
Dual-Rerank: Fusing Causality and Utility for Industrial Generative Reranking
Apr 8, 2026Chao Zhang, Shuai Lin, ChengLei Dai +5Learning to Rank RL for IR
Kuaishou Technology, Hangzhou, China · Kuaishou Technology, Beijing, China · Unaffiliated, Beijing, China
SubSearch: Intermediate Rewards for Unsupervised Guided Reasoning in Complex Retrieval
Apr 8, 2026Roxana Petcu, Evangelos Kanoulas, Maarten de RijkeRL for Language Model Reasoning Process Reward Models
IRLab, University of Amsterdam
Region-R1: Reinforcing Query-Side Region Cropping for Multi-Modal Re-Ranking
Apr 7, 2026Chan-Wei Hu, Zhengzhong TuMultimodal Reranking Multimodal RAG
Texas A&M University
Offline RL for Adaptive Policy Retrieval in Prior Authorization
Apr 6, 2026Ruslan Sharifullin, Maxim Gorshkov, Hannah ClayAdaptive Retrieval RL for IR
Stanford University.
Document Optimization for Black-Box Retrieval via Reinforcement Learning
Apr 6, 2026Omri Uzan, Ron Polonsky, Douwe Kiela +1RL for IR Large Language Model-Guided Optimization
Stanford University · ContextualAI
OASES: Outcome-Aligned Search-Evaluation Co-Training for Agentic Search
Apr 4, 2026Erhan Zhang, Yiqun Chen, Zechun Niu +6Agentic Search Process Reward Models
Renmin University of China · University of Southern California · Xiaohongshu Inc.
OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models
Mar 27, 2026Haijian Liang, Zenghao Niu, Junjie Wu +3RL for RAG Agentic RAG
Shenzhen University · OPPO Research Institute
Quality Over Clicks: Iterative Reinforcement Learning for Early-Stage E-Commerce Query Suggestion
Mar 24, 2026Qi Sun, Kejun Xiao, Huaipeng Zhao +2RL for IR Information Retrieval
Alibaba International Digital Commercial Group
APEX-Searcher: Refining Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation
Mar 14, 2026Kun Chen, Qingchao Kong, Zhao Feifei +1Agentic RAG RL for RAG
University of Chinese Academy of Sciences · MAIS, Institute of Automation, Chinese Academy of Sciences · Wenge Technology Co., Ltd
How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1
Feb 23, 2026Yinuo Xu, Shuo Lu, Jianjie Cheng +5RL for LLM Agents Autonomous Research Agents
NLPR & MAIS, CASIA · School of AI, UCAS · Meituan Inc.
Can David Beat Goliath? On Multi-Hop Reasoning with Resource-Constrained Agents
Jan 29, 2026Hojae Han, Heeyun Jung, Jongyoon Kim +1RL for Language Model Reasoning Multi-Hop QA
ETRI · The University of Hong Kong · Seoul National University
Proof-of-Use: Mitigating Tool-Call Hacking in Deep Research Agents
Oct 13, 2025Shengjie Ma, Chenlong Deng, Jiaxin Mao +5LLM Agent Reliability RL for Tool Use
Gaoling School of Artificial Intelligence, Renmin University of China China · OPPO China

Aug 23, 2026J/K move · Enter open · S save

GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning

Jun Chen, Yongchao Liu, Pengyu Qiu, Jiajun Zheng+5

1Shenzhen University, Shenzhen, China · 2Great Bay University, Dongguan, China · 3Ant Group, Hangzhou, China+1

Multi-Hop QA Agentic RAG RL for RAG RL for IR Retrieval-Augmented Reasoning

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) enables LLMs to access external knowledge for answering knowledge-intensive questions. For complex multi-hop questions, multi-turn retrieval-augmented reasoning extends RAG into an iterative process that repeatedly searches for and integrates evidence across documents. However, existing reinforcement-learning (RL) approaches for agentic RAG are typically optimized with final-answer rewards, which provide sparse supervision and overlook whether the model actually retrieves the required evidence chain. We present GTA-RAG, a graph-trajectory-augmented RL framework for multi-turn retrieval-augmented reasoning. From an entity--document graph, we sample connected document paths, synthesize multi-hop QA trajectories, and validate them with the deployed retriever to obtain executable trajectory-level supervision. We then optimize the retrieval policy with Group Relative Policy Optimization (GRPO) and a trajectory-guided reward that encourages both accurate answers and acquisition of target evidence documents, followed by answer-reward training on natural QA instances. Experiments on three multi-hop and two simple QA benchmarks show that \method{} consistently outperforms RL-based RAG baselines with both Qwen2.5-3B and Qwen2.5-7B backbones, while substantially improving evidence-chain coverage. Our code is available at https://github.com/cjcj46262/GTA-RAG.