RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

This field studies how to train large language model agents to act over multiple steps in environments such as web browsers, coding tools, games, and simulated workplaces. Common approaches use policy-gradient methods such as PPO and GRPO, verifiable task rewards, and process-level credit assignment to connect outcomes to particular actions or turns. Other methods teach agents to plan hierarchically, manage memory and reusable skills, or coordinate multiple agents, often using simulators and environment-generated feedback. A central challenge is making learning stable and efficient when rewards are sparse and tasks are long, while ensuring gains transfer to new environments and do not come from exploiting flawed reward signals.

Latest papers

616 papers134 papers in the last four weeks, up 139% on the four weeks before. 0.8% of all new papers.

Training language models in simulation for psychiatric intake
Oct 29, 2026Samuel Schmidgall, Tim Strother, Alex Bijamov +53Mental Health Conversational Agents LLM Agent Training
Don't Lose the Thread: Empowering Long-Horizon LLM Agents with Cognitive Resource Self-Allocation
Oct 29, 2026Zheng Zhu, Xiaozhuan Liang, Xi Chen +2Long-Horizon LLM Agents Memory Management for LLM Agents
The Chinese University of Hong Kong · Tencent · The Hong Kong University of Science and Technology
Aligning Autoresearch: Co-Improvement between Agents and Humans
Oct 28, 2026Tianjian Li, Chuanyang Jin, Seungone Kim +10Autonomous Research Agents Human-AI Collaboration
FAIR at Meta · Johns Hopkins University · Carnegie Mellon University +1
EnvSched: Environment-Aware Scheduling for Rollout-Efficient Agent Reinforcement Learning
Oct 27, 2026Sihan Yang, Weichen Bai, Ruichuan An +9Agentic RL Reinforcement Learning
PKU · XJTU · THU +3
Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces
Oct 27, 2026Jian Mu, Ruibo Shen, Yao ShuLLM Agent Harnesses Agent Harness Optimization
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
Action-Value Calibration via Continuation Sharing for Long-Horizon Agentic Reinforcement Learning
Oct 27, 2026Yingjian Chen, Lang Zhou, Zixin Zhao +3Agentic RL Step-Level Credit Assignment in RL
Sun Yat-sen University · Shenzhen Loop Area Institute
CoHARL: Co-Evolving Agent Policies and Executable Harnesses with Reinforcement Learning
Oct 27, 2026Jianwen Chen, Peng Xia, Kaide Zeng +5Agent Harness Optimization RL for LLM Agents
University of North Carolina at Chapel Hill · RIKEN · Cisco
FOCUS: Active Critic Learning for Long-Horizon Language Model Agents
Oct 26, 2026Zhengping Jiang, William Jurayj, Liaoyaqi Wang +3RL for LLM Agents Long-Horizon RL
Johns Hopkins University
MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors
Oct 26, 2026Runchu Tian, Zaid Khan, Zun Wang +2RL for LLM Agents Reinforcement Learning
Department of Computer Science, University of North Carolina at Chapel Hill
InterQ-RL: Interaction-Aware Correction of Execution Mismatch in Quantized Agentic RL
Oct 26, 2026Yingbo Hao, Song Liu, Guanghua Yu +3Reinforcement Learning Agentic RL
South China University of Technology · Tencent
Multiscale Policy Optimization for Long-Horizon LLM Agents
Oct 26, 2026Tao Ren, Hui Yang, Jia Deng +16Policy Optimization RL for LLM Agents
Peking University · IQuest Research · Tsinghua University +3
EnvLunch: The Environment as a Free Lunch for Learning to Interact
Oct 24, 2026Wei Liu, Min Gao, Yali Du +1LLM Agent Training RL for LLM Agents
King’s College London · Chongqing University · The Alan Turing Institute
ProtAgent: Post-Training Compact Protein Tool Agents with Interaction-Level Credit Assignment
Oct 24, 2026Yuxiang Wang, Baojian Zhou, Qifang Zhao +3Tool-Using Agents LLM Agent Training
SWE-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self-Distillation
Oct 24, 2026Xudong Cai, Zhengxi Lu, Yuchen Yan +4Software Engineering Agents Repository-Level Program Repair
Zhejiang University
Towards User Simulators That Stay On Task And Still Sound Natural
Oct 22, 2026Mustafa Omer Gul, Philippe Laban, Tanya Goyal +1User Simulation Human Behavior Simulation
Cornell University · Microsoft Research
RIPPLE: Recovery Internalization through Process-Level Policy Learning and Effect-Aware Credit for Long-Horizon Agentic Coding
Oct 22, 2026Jingxuan Xu, Zihang Xu, Yuechen Luo +9Code Generation RL for LLM Agents
National University of Singapore, Singapore · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Tsinghua University, Beijing, China +1
ReflectPO: Reflection-Guided Policy Optimization for Long-Horizon LLM Agents
Oct 22, 2026Rumei Li, Xunliang CaiLong-Horizon LLM Agents RL for LLM Agents
Meituan, China
RoleFlow: Self-Evolving Multi-Role Agentic Collaboration via Structural Adaptation
Oct 21, 2026Yi Ge, Pan Wang, Xuehao Cui +4Multi-Agent LLM Systems LLM Agent Workflow Optimization
University of Maryland, College Park · University of Pittsburgh · The University of Texas at Dallas
Uni-Mem: Learning a Long-Horizon Agentic Memory Manager with Unified Online Policy Optimization
Oct 21, 2026Zhiqian Xia, Jielong Tang, Ling-An Zeng +8Memory Management for LLM Agents RL for LLM Agents
Southeast University · Huawei Technologies, Noah’s Ark Lab · Sun Yat-sen University +1
One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents
Oct 21, 2026Shaojin Ma, Ruitao Wang, Pengyi Li +6Policy Optimization RL for LLM Agents
Tianjin University · Fudan University · Renmin University of China
BIRD-RL: Scaling Agentic Reinforcement Learning over Stateful Data-Centric Environments
Oct 21, 2026Ge Qu, Jinyang Li, Xiaohan Xu +6Agentic RL RL for LLM Agents
The University of Hong Kong · National University of Singapore · The Chinese University of Hong Kong, Shenzhen
Steering Black-Box Large Language Model Agents with Pitfall Anticipation World Model
Oct 21, 2026Jiongxiao Wang, Han Ding, Youzhi Luo +4LLM World Models LLM Agents
University of Wisconsin–Madison · AWS AI
FlashNext-RL: A System Optimization Study for Long-Horizon Agentic RL of Qwen3.8-Flash-Next
Oct 20, 2026Zhewei Yao, Yuxiong HeReinforcement Learning RL for LLM Agents
Snowflake AI Research
Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets
Oct 20, 2026Hieu Tran, Hong YuAlgorithmic Collusion RL for LLM Agents
University of Massachusetts Amherst · University of Massachusetts Lowell
Measure First, Then Allocate: Calibrated Rollout Budgets for MCP Agents in Reinforcement Learning
Oct 20, 2026Hieu Tran, Zonghai Yao, Sunjae Kwon +1RL for LLM Agents RL for Language Models
University of Massachusetts Amherst · University of Massachusetts Lowell
Capable Teacher, Myopic Scorer: Reward-Free Teacher Repair for Agentic On-Policy Distillation
Oct 20, 2026Shu Zhao, Tan YuOn-Policy Distillation RL for LLM Agents
NVIDIA
Rewards for Budget-Aware Multi-Turn Agents That Adapt to Arbitrary Test-Time Constraints
Oct 20, 2026Tom A. Lamb, Adam Golinski, Eleonora Gualdoni +2LLM Agent Efficiency Reward Design for RL
Apple · University of Oxford
FALCA: Falsification-Aware Reinforcement Learning with Hierarchical Credit Assignment for Long-Horizon Deep Search Agents
Oct 20, 2026Zijun Dou, Kun Luo, Chaofan Li +2Long-Horizon LLM Agents RL for LLM Agents
BAAI · THU · CASIA
SAPD: Learning from Agent Experience without Environment Interaction
Oct 20, 2026Juncheng Diao, Haibin Wang, Wei Shi +4LLM Agent Training Language Model Distillation
Meituan, LongCat Team · University of Chinese Academy of Sciences
From Outcomes to Steps: Reliable Success-Aware Critic Learning for Long-Horizon Language Agents
Oct 20, 2026Jiaxi Liu, Shixiong Kai, Zhentao Tang +4Reinforcement Learning RL for LLM Agents
Shanghai Jiao Tong University · Huawei Noah’s Ark Lab

Oct 29, 2026J/K move · Enter open · S save

Training language models in simulation for psychiatric intake

Samuel Schmidgall, Tim Strother, Alex Bijamov, Min Woo Sun+52

Mental Health Conversational Agents LLM Agent Training Large Language Model-Based Role-Play Simulation Clinical Language Model Evaluation RL for LLM Agents

Open paper AskPDF arXiv

In psychiatric practice, the clinical intake interview remains a primary diagnostic instrument, requiring the clinician to establish rapport while disentangling complex, overlapping presentations. Here, we show that multi-turn reinforcement learning (RL) in simulation environments improves language model performance on structured text-based psychiatric intake, spanning history-taking, differential diagnosis, and initial management planning. In a randomized, double-blind crossover study (

N = 60

matched scenario pairs, 120 encounters) comparing the RL-trained agent against its identically prompted base model, 32 board-certified psychiatrists role-played standardized patients while a separate panel of 15 independent, board-certified psychiatrists scored the de-identified transcripts. The trained agent outperformed the base model in 76.7% of matched scenario pairs (

p < 0.0001

), with gains on an adapted clinical-examination rubric (

d_z = 0.77

p < 0.0001

), patient-centered communication (82.0% vs. 65.2%,

p < 0.001

), and differential diagnosis appropriateness (68.3% vs. 33.3%,

p < 0.001

). Across 127 psychiatrist-validated simulated intake scenarios, the trained agent nearly doubled its base model's clinical rubric score (0.706 vs. 0.379; composite reward 0.564 vs. 0.294,

p < 0.0001

). To systematically probe conversational safety, we established two Delphi-consensus taxonomies, of clinical safety risks and relational competencies, from which 256 challenging adversarial presentations were derived; we find that training in simulation environments more than halved under-triage of acute safety risks (12.9% vs. 30.5%,

p < 0.0001

). Simulation environments thus provide a controlled testbed for training and stress-testing psychiatric conversational agents, with gains that transferred to psychiatrist-enacted encounters, though residual gaps require safeguards and clinician oversight.