RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

This field studies how to train large language model agents to act over multiple steps in environments such as web browsers, coding tools, games, and simulated workplaces. Common approaches use policy-gradient methods such as PPO and GRPO, verifiable task rewards, and process-level credit assignment to connect outcomes to particular actions or turns. Other methods teach agents to plan hierarchically, manage memory and reusable skills, or coordinate multiple agents, often using simulators and environment-generated feedback. A central challenge is making learning stable and efficient when rewards are sparse and tasks are long, while ensuring gains transfer to new environments and do not come from exploiting flawed reward signals.

Latest papers

616 papers134 papers in the last four weeks, up 139% on the four weeks before. 0.8% of all new papers.

MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments
May 8, 2026Yicheng Gao, Xiaolin Zhou, Yahan Li +2Healthcare Medical Diagnosis
University of Southern California · Arizona State University
PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
May 7, 2026Minghao Yan, Bo Peng, Benjamin Coleman +11Test-Time Adaptation Reinforcement Learning
Google · University of Wisconsin–Madison · Google DeepMind
SkillOS: Learning Skill Curation for Self-Evolving Agents
May 7, 2026Siru Ouyang, Jun Yan, Yanfei Chen +13Continual Learning for LLM Agents LLM Agent Skill Learning
University of Illinois Urbana-Champaign · Google Cloud AI Research · Massachusetts Institute of Technology
Weblica: Scalable and Reproducible Training Environments for Visual Web Agents
May 7, 2026Oğuzhan Fatih Kar, Roman Bachmann, Yuanzheng Gong +2LLM Agent Training RL for LLM Agents
Apple
A $^2$ TGPO: Agentic Turn-Group Policy Optimization with Adaptive Turn-level Clipping
May 7, 2026Dingwei Chen, Zefang Zong, Zhipeng Ma +5Credit Assignment in RL Policy Gradient Methods
Tencent Inc · The Chinese University of Hong Kong · Shenzhen MSU-BIT University
Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning
May 7, 2026Yaorui Shi, Yuxin Chen, Zhengxi Lu +6LLM Agent Skill Learning LLM Agent Skill Retrieval
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2
Milestone-Guided Policy Learning for Long-Horizon Language Agents
May 7, 2026Zixuan Wang, Yuchen Yan, Hongxing Li +7Credit Assignment in RL Long-Horizon Agent Tasks
Zhejiang University · Baidu Inc.
Selective Rollout: Mid-Trajectory Termination for Multi-Sample Agent RL
May 7, 2026Zhiyuan Zhai, Xin WangReinforcement Learning Group Relative Policy Optimization
Fudan University
From History to State: Constant-Context Skill Learning for LLM Agents
May 6, 2026Haoyang Xie, Xinyuan Wang, Yancheng Wang +2LLM Agent Skill Learning LLM Agent Training
School of Computing and Augmented Intelligence, Arizona State University
OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents
May 6, 2026Shuang Chen, Kaituo Feng, Hangting Chen +7Multimodal IR Multimodal Search Agents
Tencent Hunyuan · University of California, Los Angeles · The Chinese University of Hong Kong +1
Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation
May 6, 2026Zhiqing Cui, Haotong Xie, Jiahao Yuan +11Multi-Agent System Optimization Multi-Agent Orchestration
Project lead.
Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Traces
May 4, 2026Chenchen ZhangMulti-Agent LLM Systems LLM Agent Orchestration
Independent Researcher
On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
May 4, 2026Sunghwan Kim, Junhee Cho, Beong-woo Kwak +6Long-Horizon Planning Length Generalization
Department of Artificial Intelligence, Yonsei University · †Work done during internship at Microsoft Research. · 2Microsoft Research.
T $^2$ PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
May 4, 2026Haixin Wang, Hejie Cui, Chenwei Zhang +7RL for LLM Agents Reinforcement Learning
University of California, Los Angeles · 2Amazon.com Inc.
Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning
May 4, 2026Wenyi Wu, Sibo Zhu, Kun Zhou +1Reinforcement Learning Multi-Agent Collaboration
University of California, San Diego
Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory
May 1, 2026Derong Xu, Shuochen Liu, Pengfei Luo +8Memorization in Language Models Memorization in Generative Models
University of Science and Technology of China & State Key Laboratory of Cognitive Intelligence · City University of Hong Kong · Dalian University of Technology +1
Healthcare AI GYM for Medical Agents
May 1, 2026Minbyul JeongMedical LLMs RL for Tool Use
Upstage AI
AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning
May 1, 2026Haotian Zhao, Songlin Zhou, Yuxin Zhang +9Agentic RL Credit Assignment in RL
Baidu · Tsinghua University · Fudan University
Exploration Hacking: Can LLMs Learn to Resist RL Training?
Apr 30, 2026Eyon Jang, Damon Falck, Joschka Braun +6RL Exploration LLM Safety
MATS · UC San Diego · Anthropic +1
Rethinking Agentic Reinforcement Learning In Large Language Models
Apr 30, 2026Fangming Cui, Ruixiao Zhu, Cheng Fang +2RL for Language Models Agentic RL
Beijing, China · Shanghai, China
FutureWorld: A Live Reinforcement Learning Environment for Predictive Agents with Real-World Outcome Rewards
Apr 29, 2026Zhixin Han, Yanzhi Zhang, Chuyang Wei +11Agentic RL Reinforcement Learning
College of Software, Nankai University · Zhongguancun Academy, Beijing, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences +3
DPEPO: Diverse Parallel Exploration Policy Optimization for LLM-based Agents
Apr 27, 2026Junshuo Zhang, Chengrui Huang, Feng Guo +6Reinforcement Learning RL Exploration
University of Electronic Science and Technology of China · DiDi
Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis
Apr 27, 2026Zhisong Qiu, Shuofei Qiao, Kewei Xu +4Process Reward Models RL for LLM Agents
Zhejiang University Ant Group Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph Hangzhou, China · Zhejiang University Hangzhou, China · Ant Group Hangzhou, China +2
JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Training
Apr 26, 2026Zhengding Hu, Hehua Ouyang, Chang Chen +6RL Post-Training Pipeline Parallelism
University of California, San Diego
Ask Only When Needed: Proactive Retrieval from Memory and Skills for Experience-Driven Lifelong Agents
Apr 22, 2026Yuxuan Cai, Wei Li, Jie Zhou +4Lifelong Learning Agents LLM Agent Skill Learning
School of Computer Science and Technology, East China Normal University, Shanghai · Shanghai AI Laboratory
DR-MMSearchAgent: Deepening Reasoning in Multimodal Search Agents
Apr 21, 2026Shengqin Wang, Wentao Yan, Huichi Zhou +4Reward Shaping RL for Language Model Reasoning
East China Normal University · University College London · Huawei Noah’s Ark Lab +2
SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution
Apr 21, 2026Xiachong Feng, Yi Jiang, Xiaocheng Feng +9Credit Assignment in RL Step-Level Credit Assignment in RL
The University of Hong Kong · Harbin Institute of Technology · Harbin Institute of Technology, Shenzhen
StepPO: Step-Aligned Policy Optimization for Agentic Reinforcement Learning
Apr 20, 2026Daoyu Wang, Qingchuan Li, Mingyue Cheng +4Agentic RL Credit Assignment in RL
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China
Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
Apr 20, 2026Guanting Dong, Junting Lu, Junjie Huang +17Lifelong Learning Agents AI Agent Benchmarks
Renmin University of China · ByteDance Seed
CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search
Apr 19, 2026Hansi Zeng, Liam Collins, Bhuvesh Kumar +2Reinforcement Learning Learning to Rank
Center for Intelligent Information Retrieval, University of Massachusetts Amherst · Snap Inc.

May 8, 2026J/K move · Enter open · S save

MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments

Yicheng Gao, Xiaolin Zhou, Yahan Li, Yue Zhao+1

University of Southern California · Arizona State University

Healthcare Medical Diagnosis LLM Agent Training RL for LLM Agents LLM Agent Reliability Medical AI Agents

Open paper AskPDF arXiv

Real-world clinical diagnosis is a complex process in which the doctor is required to obtain information from both interaction with the patient and conducting medical exams. Additionally, the doctor needs to adapt to different patient personas, as well as noisy and incomplete information that can happen at any time during the process. However, existing benchmarks for medical LLMs and methods for automatic diagnosis largely simplify this process by reducing it to single-turn question answering, noise-free conversations, or sequential exam making, etc., ignoring the interactive and uncertain nature of clinical diagnosis. In this paper, we aim to address this gap by formalizing clinical diagnosis as a Partially Observable Markov Decision Process (POMDP) with three action types: questioning the patient, ordering medical exams as tool calls, and issuing a diagnosis. We also introduce a systematic noise model comprising seven patient noise types and three exam noise types. Using our proposed environment, we train an effective diagnosis agent, MedExAgent, through a two-stage pipeline that first performs supervised finetuning on synthetic conversations structured after the Calgary-Cambridge model for clinical interviews, and then applies DAPO to optimize a composite reward capturing diagnostic accuracy, tool call quality, and exam cost including financial cost and patient discomfort. Through extensive experiments and ablation studies, we demonstrate that MedExAgent achieves diagnostic performance comparable to larger models while maintaining cost-efficient examination strategies.