RL for IR

RL: Reinforcement Learning · IR: Information Retrieval

Reinforcement learning for information retrieval studies how systems can learn what to retrieve, how to rank it, and when to stop using feedback from downstream tasks or user interactions. Approaches include policy-gradient methods such as PPO, GRPO, and REINFORCE, offline methods such as CQL and IQL, and joint training of search agents, retrievers, and rankers. Work spans multi-hop question answering and agentic search, retrieval benchmarks such as BEIR and BRIGHT, and industrial settings including e-commerce search and recommendation. A central challenge is assigning useful credit when rewards are delayed or sparse: intermediate evidence and process rewards can guide learning, but may be noisy, invite reward hacking, or trade off against retrieval cost and generalization.

Latest papers

51 papers21 papers in the last four weeks, up 320% on the four weeks before. 0.1% of all new papers.

RANGER: Retrieval Agent for Navigating Graphs with Semantic Evaluation Rewards
Oct 20, 2026Artem Mikhaylov, Pavel Konoplev, Nikolay Efanov +2Knowledge Graph Question Answering LLM-Guided RL
Skolkovo Institute of Science and Technology (Skoltech), Moscow, Russia · Moscow Institute of Physics and Technology (MIPT), Moscow, Russia · AIRI, Moscow, Russia
EG-GRPO: Evidence-Guided Step-level Credit Assignment for Deep Search Agents
Oct 18, 2026Sijia Luo, Mengjie Gao, Yuxiao He +4RL for LLM Agents Step-Level Credit Assignment in RL
Renmin University of China, Beijing 100872, China · Key Laboratory of Data Engineering and Knowledge Engineering, MOE, China · Ant Group, Beijing, China +2
Learning to Program Retrievals: Composing Search Primitives via Reinforcement Learning
Oct 17, 2026Kyuyoung Kim, Zilin Xiao, Zhiyuan Wang +4Retrieval-Augmented Reasoning RL for RAG
KAIST AI · Meta Superintelligence Labs
Expert-Guided Rubrics as Dense Reward Signals for Reinforcement Learning in Non-Verifiable Domains
Oct 17, 2026Ashish Bansal, Hossam Amer, Keerat Kaur Guliani +3RL for LLM Agents Reward Design for RL
The Vanguard Group Malvern, Pennsylvania, USA · The Vanguard Group Toronto, Ontario, Canada
RL-Guided Temporal Localization for Dual-Channel Retrieval in Long-Horizon Agent Memory
Oct 16, 2026Dabin Sheng, Zhe Wu, Jinming Zhao +2Temporal IR Agentic Retrieval
Key Laboratory of Pervasive Computing, Ministry of Education, Department of Computer Science and Technology, Tsinghua University · Qiyuan Lab
Grounded Credit Assignment in Reinforcement Learning for Multi-Turn Search Agents
Oct 16, 2026Qirui Liu, Hao Chen, Yi Wen +7Credit Assignment in RL Reinforcement Learning with Verifiable Rewards
South China University of Technology · FiT, Tencent · City University of Hong Kong +5
Reasoning Reranking with Semantic IDs as the Sole Item Representation
Oct 16, 2026Zhenwei Tang, Jintao Liu, Di Bai +2LLM Reranking Semantic ID Recommendation
Google · University of Toronto
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
Oct 15, 2026Siqi Chen, Zhengxi Lu, Yuchen Yan +10RL for LLM Agents Reinforcement Learning
Zhejiang University · Tencent
QREF: Improving DeepSearch Agents by Aligning Search with Reasoning via Query Refinement and Evidence Filtering
Oct 15, 2026Yujia Chen, Yuhui Wang, Junzhe Wang +15Agentic Search Web Search Agents
Fudan University · Tencent
CASI: Credit Assignment via Set Intervention for Agentic Retrieval
Oct 14, 2026Xinyi Cheng, Tianfu Zhang, Jiwen Li +5Credit Assignment Fine-Grained Credit Assignment
Xiaohongshu Inc.
Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation
Oct 14, 2026Shijun Li, Yang Liu, Najmeh Sadoughi +5Machine Translation Retrieval-Augmented Generation
The University of Texas at Austin · Amazon
WeGR: Towards Unified Generative Retrieval via Behavior-aware Tokenization and Preference Alignment
Oct 12, 2026Jiajun Han, Guoxin Ma, Felix +9Generative Retrieval RL for IR
Tencent Penguin Island
Co-adaptive Rule Routing and Constraining for Structured Knowledge Retrieval
Oct 10, 2026Zhisheng Qi, Haoyu Han, Jiaxi Li +9Information Retrieval Adaptive Retrieval
University of Georgia · Michigan State University · Georgia Institute of Technology +4
Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval
Oct 8, 2026Jianfei Zhao, Yifan Wang, Feng Zhang +7Information Retrieval Cross-Modal Retrieval
School of Computer Science and Technology, Beijing Institute of Technology · Zhongguancun Academy · Xiaohongshu +2
Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents
Oct 7, 2026Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis +2Reward Design for RL RL for IR
University of Edinburgh, UK · Huawei Technologies Research & Development (UK) Limited
Learning to Retrieve via Reinforcement Learning in Embedding Space
Oct 6, 2026Qi Liu, Fengming Liang, Yiqun Chen +2RL for IR Text Embedding Models
Renmin University of China
T-Search: An Open Agentic Retriever and Playground for Hard Multi-Step Search
Oct 5, 2026Olga Tsymboi, Ramil Latypov, Aleksandr Medvedev +5Multi-Hop QA Agentic Search
T-Tech
LexiHorizon: Stabilizing Reinforcement Learning for Long-Horizon Deep Search
Oct 4, 2026Zhiqing Nong, Liang Wen, Chao-Hsuan LiuLong-Horizon Agent Evaluation RL for IR
Learning to Route in Visual Space via Multi-Step Embedding Retrieval
Sep 30, 2026Tianyu Chen, Mingyuan Zhou, Jiaxing WuMultimodal Search Agents Agentic Retrieval
The University of Texas at Austin · Google DeepMind
Learning to Retrieve Missing Evidence for Long-Term Memory QA
Sep 29, 2026Yi-Xuan Deng, Yi Zhang, Wei Liu +2Long-Term Conversational Memory Evidence Retrieval
Tsinghua University
Efficient Offline Learning of Ranking Policies via Top- $k$ Policy Decomposition
Sep 29, 2026Ren Kishimoto, Koichi Tanaka, Haruka Kiyohara +4Learning to Rank Off-Policy RL
Hanjuku-kaso, Co., Ltd. Tokyo, Japan
IGSD: Environment-Verified Hindsight Self-Distillation for Search Agents
Sep 26, 2026Angqing Jiang, Gaoming Zhang, Chaoqun Zhang +5Self-Distillation Agentic Search
University of Science and Technology of China · Tsinghua University · Meituan
Search-Aware Reinforcement Learning for Multi-Component Query Understanding in Roblox Game Search
Sep 24, 2026Nayoung Choi, Shengjian Chen, Xiaokai Wei +6Reinforcement Learning RL for IR
Emory University, Atlanta, Georgia, USA · Roblox Corporation, San Mateo, California, USA
What Should We Ask Next? Retrieval-Aware Question Learning for Interactive ReID
Sep 18, 2026Lyucheng Qian, John Yuehan Zhang, Pingyu WangPerson Re-Identification RL for IR
Sichuan University · University of California, Berkeley
It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning
Sep 1, 2026Runpeng Dai, Kaili Huang, Changsung Kang +1Reinforcement Learning Generative Retrieval
University of North Carolina at Chapel Hill · Apple
Learning from What You Retrieve: Online RL Fine-Tuning for Semantic Retrieval
Aug 31, 2026Shaowei Wei, Chong Huang, Songtao Fang +3Semantic Search RL for IR
Alibaba Group Hangzhou, China
SearchWiki: Learning to Build and Navigate Knowledge Wikis for Active Information Seeking
Aug 30, 2026Guransh Singh, Vishwajeet Kumar, Arkadeep Acharya +3Retrieval-Augmented Generation Agentic Retrieval
Birla Institute of Technology, Mesra · IBM
MetaRAG: Belief-Action Aligned Policy Optimization for Agentic RAG
Aug 25, 2026Qiuyi Qi, Tian Liang, Jiamu Wang +7Agentic RAG RL for LLM Agents
Zhejiang University · Ant Group · City University of Hong Kong
GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning
Aug 23, 2026Jun Chen, Yongchao Liu, Pengyu Qiu +6Multi-Hop QA Agentic RAG
1Shenzhen University, Shenzhen, China · 2Great Bay University, Dongguan, China · 3Ant Group, Hangzhou, China +1
Harness-G: A Graph-Structured Harness for Search Agents
Jul 30, 2026Yanning Hou, Haoyuan Chen, Sihang Zhou +7RL for LLM Agents Credit Assignment in RL
National University of Defense Technology · Changsha, China

Oct 20, 2026J/K move · Enter open · S save

RANGER: Retrieval Agent for Navigating Graphs with Semantic Evaluation Rewards

Artem Mikhaylov, Pavel Konoplev, Nikolay Efanov, Artyom Sorokin+1

Skolkovo Institute of Science and Technology (Skoltech), Moscow, Russia · Moscow Institute of Physics and Technology (MIPT), Moscow, Russia · AIRI, Moscow, Russia

Knowledge Graph Question Answering LLM-Guided RL RL for IR Knowledge Graphs

Open paper AskPDF arXiv

The integration of Large Language Models and Reinforcement Learning offers a promising direction for complex sequential decision-making tasks, such as multi-hop question answering over knowledge graphs. While LLMs provide strong semantic understanding and reasoning capabilities, they often struggle with precise, multi-step navigation in structured environments. Conversely, RL agents excel at learning navigation policies but are hindered by the difficulty of designing semantically meaningful reward functions. This paper introduces a novel hybrid approach in which an LLM serves as a mentor to an RL agent during training by providing dense, semantically informed reward signals. The RL agent, built on a Transformer-based policy and trained with Group Relative Policy Optimization, learns to navigate a knowledge graph to retrieve relevant information. The LLM evaluates the semantic relevance of the agent's traversed paths, generating reward signals that align the agent's behavior with the information needs expressed in a query. We integrate this Retrieval Agent for Navigating Graphs with Semantic Evaluation Rewards (RANGER) into a complete question-answering pipeline and evaluate it on the open question answering datasets. Experimental results show that the proposed approach outperforms both a baseline retrieval-augmented system with classical graph search and one using an RL agent trained with heuristic rewards, achieving a significant increase in NLP metrics and LLM-as-a-Judge scores. Our findings demonstrate that deep integration of LLMs into the RL training loop yields more effective and semantically aligned retrieval policies for complex reasoning tasks.