RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

Overview

This field studies how to train large language model agents to act over multiple steps in environments such as web browsers, coding tools, games, and simulated workplaces. Common approaches use policy-gradient methods such as PPO and GRPO, verifiable task rewards, and process-level credit assignment to connect outcomes to particular actions or turns. Other methods teach agents to plan hierarchically, manage memory and reusable skills, or coordinate multiple agents, often using simulators and environment-generated feedback. A central challenge is making learning stable and efficient when rewards are sparse and tasks are long, while ensuring gains transfer to new environments and do not come from exploiting flawed reward signals.

Latest papers

CardsList

660 papers68 papers in the last four weeks, down 51% on the four weeks before. 0.5% of all new papers.

  1. SELAUR: Self Evolving LLM Agent via Uncertainty-aware Rewards

    Feb 24, 2026Dengjia Zhang, Xiaoou Liu, Lu Cheng +3RL for LLM AgentsReward Shaping

  2. How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1

    Feb 23, 2026Yinuo Xu, Shuo Lu, Jianjie Cheng +5RL for LLM AgentsAutonomous Research Agents

  3. Proximity-Based Multi-Turn Optimization: Practical Credit Assignment for LLM Agent Training

    Feb 22, 2026Yangyi Fang, Jiaye Lin, Xiaoliang Fu +4LLM Agent TrainingRL for LLM Agents

  4. MagicAgent: Towards Generalized Agent Planning

    Feb 22, 2026Xuhui Ren, Shaokang Dong, Chen Yang +21LLM Agent PlanningLLM Agent Training

  5. AgentConductor: Topology Evolution for Multi-Agent Competition-Level Code Generation

    Feb 19, 2026Siyu Wang, Ruotian Lu, Zhihao Yang +7LLM Agent Workflow OptimizationMulti-Agent Coding

  6. Phase-Aware Mixture of Experts for Agentic Reinforcement Learning

    Feb 19, 2026Shengtian Yang, Yu Li, Shuo He +4Reinforcement LearningRL for LLM Agents

  7. HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents

    Feb 18, 2026Jiangweizhi Peng, Yuanxin Liu, Ruida Zhou +4RL for LLM AgentsHierarchical RL

  8. Beyond Static Pipelines: Learning Dynamic Workflows for Text-to-SQL

    Feb 17, 2026Yihan Wang, Peiyu Liu, Runyu Chen +1Text-to-SQLLLM Agent Workflow Optimization

  9. Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation

    Feb 17, 2026Kaito Baba, Risa Kishikawa, Satoshi KoderaRadiology Report GenerationMultimodal Agents

  10. Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning

    Feb 13, 2026Kehao Zhang, Shangtong Gui, Sheng Yang +2Persistent Memory for Language ModelsLong-Context Inference

  11. TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents

    Feb 12, 2026Aladin Djuhera, Swanand Kadhe, Farhan Ahmed +3Tree SearchLLM Agent Training

  12. FlexMARL: Graph-Driven Rollout-Training Co-Design for LLM-Based Multi-Agent Reinforcement Learning

    Feb 10, 2026Zhida Jiang, Zhaolong Xing, Jiawei Lu +13Multi-Agent Reinforcement LearningDistributed RL

  13. Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems

    Feb 9, 2026Lang Feng, Longtao Zheng, Shuo He +2Multi-Agent LLM SystemsMulti-Agent System Optimization

  14. SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

    Feb 9, 2026Peng Xia, Jianwen Chen, Hanyang Wang +10LLM Agent Skill LearningRL for LLM Agents

  15. Reinforcement World Model Learning for LLM-based Agents

    Feb 5, 2026Xiao Yu, Baolin Peng, Ruize Xu +6World Model LearningAction-Conditioned World Models

  16. Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning

    Feb 1, 2026Yu Li, Mingyang Yi, Xiuyu Li +6RL for Language Model ReasoningGradient Interference

  17. CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning

    Jan 21, 2026Tianshi Xu, Yuteng Chen, Meng LiAgentic RLReinforcement Learning

  18. PEARL: Self-Evolving Assistant for Time Management with Reinforcement Learning

    Jan 17, 2026Bingxuan Li, Jeonghwan Kim, Cheng Qian +4Long-Horizon Agent TasksRL for LLM Agents

  19. MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards

    Jan 9, 2026Zhiyu Shen, Ziming Wu, Fuming Lai +2Reinforcement LearningMemory-Augmented Language Models

  20. AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning

    Dec 18, 2025Tzu-Han Lin, Wei-Lin Chen, Chen-An Li +3Retrieval-Augmented GenerationKnowledge Augmentation for Language Models

  21. Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space

    Dec 4, 2025Joey Hong, Kang Liu, Zhan Ling +2RL for LLM AgentsActor-Critic Methods

  22. Iterative Policy Optimization: Batch-Online Multi-Turn RL via Single-Turn RL

    Nov 26, 2025Daniel R. Jiang, Ankur Samanta, Yukai Yang +3Reinforcement LearningPolicy Iteration

  23. Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization

    Nov 25, 2025Chenliang Li, Adel Elmahdy, Alex Boyd +7RL for Language Model ReasoningImportance Sampling

  24. Learning Robust Social Strategies with Large Language Models

    Nov 24, 2025Dereck Piche, Mohammed Muqeeth, Milad Aghajohari +3Cooperative MARLRL for LLM Agents

  25. Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective

    Oct 16, 2025Shiqi Chen, Tongyao Zhu, Zian Wang +8RL ExplorationPolicy Learning

  26. There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning

    Aug 6, 2025Roman Belaire, Arunesh Sinha, Pradeep VarakanthamLLM Red TeamingAdversarial Attacks on LLMs

  27. Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agents

    Feb 6, 2025Selim Furkan Tekin, Gaowen Liu, Ramana Rao Kompella +1Multi-Agent LLM SystemsLanguage Model Ensembles