RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

This field studies how to train large language model agents to act over multiple steps in environments such as web browsers, coding tools, games, and simulated workplaces. Common approaches use policy-gradient methods such as PPO and GRPO, verifiable task rewards, and process-level credit assignment to connect outcomes to particular actions or turns. Other methods teach agents to plan hierarchically, manage memory and reusable skills, or coordinate multiple agents, often using simulators and environment-generated feedback. A central challenge is making learning stable and efficient when rewards are sparse and tasks are long, while ensuring gains transfer to new environments and do not come from exploiting flawed reward signals.

Latest papers

616 papers134 papers in the last four weeks, up 139% on the four weeks before. 0.8% of all new papers.

AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning
Apr 19, 2026Jingbo Sun, Wenyue Chong, Songjun Tu +7Agentic Retrieval Agentic RAG
SKL-MAIS, Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences +1
Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
Apr 18, 2026Weiyu Ma, Yongcheng Zeng, Yan Song +4Reinforcement Learning RL Post-Training
King Abdullah University of Science and Technology (KAUST) · Chinese Academy of Sciences, Institute of Automation (CASIA) · AI Centre, Department of Computer Science, University College London +1
CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution
Apr 17, 2026Shidong Yang, Ziyu Ma, Tongwen Huang +3LLM Agent Training RL for LLM Agents
AMAP, Alibaba Group
"Excuse me, may I say something..." CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborations
Apr 16, 2026Yang Wu, Jinhong Yu, Jingwei Xiong +2AI-Assisted Scientific Research LLM Agents
Worcester Polytechnic Institute, Worcester, MA, USA · University of California, Davis, CA, USA · Jiangsu University, Zhenjiang, Jiangsu, China
Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis
Apr 16, 2026Zhiyuan Zhai, Wenjing Yan, Xiaodan Shao +1LLM Agent Training RL for LLM Agents
Fudan University · Chinese University of Hong Kong · University of Waterloo
Mind DeepResearch Technical Report
Apr 16, 2026MindDR Team, Li Auto IncMulti-Agent LLM Systems Deep Research Agents
MindDR Team, Li Auto Inc
Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization
Apr 15, 2026Junzhe Wang, Zhiheng Xi, Yajie Yang +4LLM Agents RL for LLM Agents
Fudan University · Shanghai Artificial Intelligence Laboratory · Shanghai Key Laboratory of Intelligent Information Processing
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
Apr 15, 2026Jiahang Lin, Kai Hu, Binghai Wang +12Long-Document VQA Multi-Turn RL
Fudan University · Shanghai Qiji Zhifeng Co., Ltd
Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation
Apr 13, 2026Jiashu Yao, Heyan Huang, Zeming Liu +1Reinforcement Learning RL for LLM Agents
Beijing Institute of Technology · Beihang University
OOM-RL: Out-of-Money Reinforcement Learning Market-Driven Alignment for LLM-Based Multi-Agent Systems
Apr 13, 2026Kun Liu, Liqun ChenAI Alignment RL for LLM Agents
Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning
Apr 13, 2026Xiaozhe Li, Tianyi Lyu, Yizhao Yang +6Lost-in-the-Middle Effect LLM Context Management
Tongji University · CurrentsAI Research · Stanford University +1
From Topology to Trajectory: LLM-Driven World Models For Supply Chain Resilience
Apr 13, 2026Jia LuoLLM Agent Planning Supply Chain Management
Huazhong University and Science and Technology Wuhan, China
Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents
Apr 12, 2026Hao Wang, Guozhi Wang, Han Xiao +8RL for LLM Agents LLM Agent Skill Learning
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences · vivo AI Lab · The Chinese University of Hong Kong +2
Trust Your Memory: Verifiable Control of Smart Homes through Reinforcement Learning with Multi-dimensional Rewards
Apr 11, 2026Kai-Yuan Guo, Jiang Wang, Renjie Zhao +5Memory-Augmented Language Model Agents Internet of Things
AI Research Center, Midea Group, Shanghai, China
Instructing LLMs to Negotiate using Reinforcement Learning with Verifiable Rewards
Apr 10, 2026Shuze Daniel Liu, Claire Chen, Jiabao Sean Xiao +4Automated Negotiation Multi-Agent Negotiation
Purdue University · California Institute of Technology · University of Illinois Urbana-Champaign +1
From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models
Apr 10, 2026Chenchen ZhangCredit Assignment in RL RL for Language Models
Independent Researcher
Training Language Models for Bilateral Trade with Private Information
Apr 10, 2026Dirk Bergemann, Soheil Ghili, Xinyang Hu +2Multi-Agent Negotiation LLM Agent Training
Department of Economics, Yale University · Yale School of Management, Yale University · Department of Statistics and Data Science, Yale University +1
HiRO-Nav: Hybrid ReasOning Enables Efficient Embodied Navigation
Apr 9, 2026He Zhao, Yijun Yang, Zichuan Lin +2Adaptive Reasoning in Language Models Embodied Navigation
Nanyang Technological University · Tencent · Independent Researcher
Beyond Stochastic Exploration: What Makes Training Data Valuable for Agentic Search
Apr 9, 2026Chuzhan Hao, Wenfeng Feng, Guochao Jiang +3Agentic Search RL for Language Model Reasoning
Alibaba Cloud Computing
Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent
Apr 8, 2026Bingxuan Li, Simo Du, Yue GuoClinical Reasoning Medical Diagnosis
University of Illinois Urbana-Champaign · Albert Einstein College of Medicine
Reason in Chains, Learn in Trees: Self-Rectification and Grafting for Multi-turn Agent Policy Optimization
Apr 8, 2026Yu Li, Sizhe Tang, Tian LanRL for Language Model Reasoning RL for LLM Agents
Department of Electrical and Computer Engineering, George Washington University
Multi-Turn Reasoning LLMs for Task Offloading in Mobile Edge Computing
Apr 8, 2026Ning Yang, Chuangxin Cheng, Haijun ZhangEdge Computing LLM Reasoning
Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China · School of Mechano-Electronic Engineering, Xidian University, Xi’an, 710071, China · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, China
AgentGL: Towards Agentic Graph Learning with LLMs via Reinforcement Learning
Apr 7, 2026Yuanfu Sun, Kang Li, Dongzhe Fan +2LLMs for Graph Learning Text-Attributed Graph Learning
New York University Shanghai · New York University · Tsinghua University
Hierarchical Reinforcement Learning with Augmented Step-Level Transitions for LLM Agents
Apr 7, 2026Shuai Zhen, Yanhua Yu, Ruopei Guo +2RL for LLM Agents Reinforcement Learning
Beijing University of Posts and Telecommunications · China Mobile Group Design Institute Co., Ltd · Singapore Management University
COSMO-Agent: Tool-Augmented Agent for Closed-loop Optimization,Simulation,and Modeling Orchestration
Apr 7, 2026Liyuan Deng, Shujian Deng, Yongkang Chen +6LLM Agent Orchestration Parametric CAD Modeling
Northwestern Polytechnical University · Shanghai Artificial Intelligence Laboratory
RAGEN-2: Reasoning Collapse in Agentic RL
Apr 7, 2026Zihan Wang, Chi Gui, Xing Jin +13LLM Agents RL for LLM Agents
Northwestern University · UIUC · Independent +5
TRACE: Capability-Targeted Agentic Training
Apr 7, 2026Hangoo Kang, Tarun Suresh, Jon Saad-Falcon +1LLM Agent Self-Improvement Agent Skill Learning
Stanford University
Synthetic Sandbox for Training Machine Learning Engineering Agents
Apr 6, 2026Yuhang Zhou, Lizhu Zhang, Yifan Wu +4LLM Agent Training RL for LLM Agents
Meta AI
Marginal-Contribution Policy Gradients under Filtered Feedback for Multi-Agent LLMs
Apr 3, 2026Elai Ben-Gal, Stela TongMulti-Agent LLM Systems Multi-Agent System Optimization
Department of Mathematics Stanford University · Graduate School of Business Stanford University
Co-Evolution of Policy and Internal Reward for Language Agents
Apr 3, 2026Xinyu Wang, Hanwei Wu, Jingwei Song +8RL for LLM Agents Intrinsic Reward Methods
McGill University · McMaster University · The University of Hong Kong +6

Apr 19, 2026J/K move · Enter open · S save

AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning

Jingbo Sun, Wenyue Chong, Songjun Tu, Qichao Zhang+6

SKL-MAIS, Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences+1

Agentic Retrieval Agentic RAG RL for LLM Agents RL for RAG

Open paper AskPDF arXiv

Agentic retrieval-augmented generation (RAG) systems enable large language models (LLMs) to solve complex tasks through multi-step interaction with external retrieval tools. However, such multi-step interaction often involves redundant search steps, incurring substantial computational cost and latency. Prior work limits search depth (i.e., the number of search steps) to reduce cost, but this often leads to underexploration of complex questions. To address this, we first investigate how search depth affects accuracy and find a minimal sufficient search depth that defines an accuracy-efficiency trade-off, jointly determined by question complexity and the agent's capability. Furthermore, we propose AutoSearch, a reinforcement learning (RL) framework that evaluates each search step via self-generated intermediate answers. By a self-answering mechanism, AutoSearch identifies the minimal sufficient search depth and promotes efficient search by rewarding its attainment while penalizing over-searching. In addition, reward mechanisms are introduced to stabilize search behavior and improve answer quality on complex questions. Extensive experiments on multiple benchmarks show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality.