RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

Ask about this topic

Overview

This field studies how to train large language model agents to act over multiple steps in environments such as web browsers, coding tools, games, and simulated workplaces. Common approaches use policy-gradient methods such as PPO and GRPO, verifiable task rewards, and process-level credit assignment to connect outcomes to particular actions or turns. Other methods teach agents to plan hierarchically, manage memory and reusable skills, or coordinate multiple agents, often using simulators and environment-generated feedback. A central challenge is making learning stable and efficient when rewards are sparse and tasks are long, while ensuring gains transfer to new environments and do not come from exploiting flawed reward signals.

Latest papers

CardsList

616 papers134 papers in the last four weeks, up 139% on the four weeks before. 0.8% of all new papers.

  1. ProCeedRL: Process Critic with Exploratory Demonstration Reinforcement Learning for LLM Agentic Reasoning

    Apr 2, 2026Jingyue Gao, Yanjiang Guo, Xiaoshuai Chen +1Agentic RLRL for Language Model Reasoning

    Institute for Interdisciplinary Information Sciences, Tsinghua University · Shanghai Qizhi Institute · Independent Researcher

  2. ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents

    Apr 2, 2026Yong Wu, YanZhao Zheng, TianZe Xu +9LLM Context ManagementLong-Horizon LLM Agents

    Zhejiang University, Hangzhou, China · Alibaba Group, Hangzhou, China

  3. DeltaMem: Towards Agentic Memory Management via Reinforcement Learning

    Apr 2, 2026Qi Zhang, Shen Huang, Chu Liu +4Agent Memory ManagementAgent Memory

    Alibaba Tongyi Lab, Hangzhou, China · Zhejiang University, Hangzhou, China

  4. LangMARL: Natural Language Multi-Agent Reinforcement Learning

    Apr 1, 2026Huaiyuan Yao, Longchao Da, Xiaoou Liu +3RL for LLM AgentsMulti-Agent Reinforcement Learning

    Arizona State University · Cisco Research · University of North Carolina at Chapel Hill

  5. Agent Q-Mix: Selecting the Right Action for LLM Multi-Agent Systems through Reinforcement Learning

    Apr 1, 2026Eric Hanchen Jiang, Levina Li, Rui Sun +9Multi-Agent Communication TopologyMulti-Agent Reasoning

    University of California Los Angeles · University of Washington · Northwestern University

  6. MemFactory: Unified Inference & Training Framework for Agent Memory

    Mar 31, 2026Ziliang Guo, Ziheng Li, Bo Tang +2LLM Agent MemoryAgent Memory Management

    MemTensor

  7. MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination

    Mar 25, 2026Zhuo Li, Yupeng Zhang, Pengyu Cheng +8LLM Hallucination MitigationLLM Hallucination Detection

    Qwen Large Model Application Team, Alibaba · The Chinese University of Hong Kong, Shenzhen

  8. From Pixels to Digital Agents: An Empirical Study on the Taxonomy and Technological Trends of Reinforcement Learning Environments

    Mar 25, 2026Lijing Luo, Yiben Luo, Alexey Gorbatovski +2RL BenchmarksReinforcement Learning

    Sun Yat-sen University Shenzhen, China · Yancheng Institute of Technology Yancheng, China · Central University Moscow, Russia +1

  9. EVA: Efficient Reinforcement Learning for End-to-End Video Agent

    Mar 24, 2026Yaolun Zhang, Ruohui Wang, Jiahao Wang +6Video UnderstandingTemporal Video Understanding

    SenseTime Research

  10. A Context Engineering Framework for Improving Enterprise AI Agents based on Digital-Twin MDP

    Mar 23, 2026Xi Yang, Aurelie Lozano, Naoki Abe +6LLM AgentsLLM Agent Training

    IBM Software Innovation Lab, Yorktown Heights, New York, USA

  11. SOLAR: A Self-Optimizing Open-Ended Autonomous Agent for Lifelong Learning and Continual Adaptation

    Mar 23, 2026Nitin Vetcha, Dianbo LiuContinual Learning for LLM AgentsContinual Learning for LLMs

    Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore · Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, Karnataka, India

  12. PivotRL: High Accuracy Agentic Post-Training at Low Compute Cost

    Mar 22, 2026Junkeun Yi, Damon Mosk-Aoyama, Baihe Huang +9RL Post-TrainingRL for LLM Agents

    NVIDIA · UC Berkeley

  13. A Subgoal-driven Framework for Improving Long-Horizon LLM Agents

    Mar 20, 2026Taiyi Wang, Sian Gooding, Florian Hartmann +2Long-Horizon LLM AgentsWeb Agents

    Google DeepMind

  14. Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation

    Mar 19, 2026Zhuolin Yang, Zihan Liu, Yang Chen +14RL for Language Model ReasoningRL for LLM Agents

  15. RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models

    Mar 19, 2026Xiao Feng, Bo Han, Zhanke Zhou +5RL for LLM AgentsReward Design for RL

    TMLR Group, Hong Kong Baptist University · TCL Corporate Research (HK) Co., Ltd · Cooperative Medianet Innovation Center, Shanghai Jiao Tong University +1

  16. ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agents

    Mar 19, 2026Hao Zhang, Mingjie Liu, Shaokun Zhang +10LLM Agent TrainingLLM Agent Harnesses

  17. Memento-Skills: Let Agents Design Agents

    Mar 19, 2026Huichi Zhou, Siyuan Guo, Anjie Liu +14Agentic AILLM Agent Skill Learning

  18. HISR: Hindsight Information Modulated Segmental Process Rewards For Multi-turn Agentic Reinforcement Learning

    Mar 19, 2026Zhicong Lu, Zichuan Lin, Wei Jia +7RL for LLM AgentsAgentic RL

    Aerospace Information Research Institute, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tencent Hunyuan +1

  19. CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents

    Mar 18, 2026Lintang Sutawika, Aditya Bharat Soni, Bharath Sriraam R R +8Code LocalizationCode Search

    Carnegie Mellon University · Independent · OpenHands

  20. Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalation

    Mar 18, 2026Philipp Normann, Andreas Happe, Jürgen Cito +1LLM Agent SecurityLLM Post-Training

    TU Wien, Vienna, Austria

  21. Complementary RL: Towards Efficient Experience-Driven Agent Learning

    Mar 18, 2026Dilxat Muhtar, Jiashun Liu, Wei Gao +8Reinforcement LearningSample-Efficient RL

    Alibaba Group · HKUST

  22. SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Training

    Mar 18, 2026Prince Zizhuang Wang, Shuli JiangRL for LLM AgentsAgentic RL

    Carnegie Mellon University

  23. Learning to Present: Inverse Specification Rewards for Agentic Slide Generation

    Mar 17, 2026Karthik Ragunath Ananda Kumar, Subrahmanyam ArunachalamLLM Agent TrainingTool-Augmented Language Model Agents

    Tavus Inc. · University of Texas at Dallas · Texas A&M University

  24. When Should a Robot Think? Resource-Aware Reasoning via Reinforcement Learning for Embodied Robotic Decision-Making

    Mar 17, 2026Jun Liu, Pu Zhao, Zhenglun Kong +12Embodied AIEmbodied Reasoning

    Robotics Institute, Carnegie Mellon University · Northeastern University · Harvard University +9

  25. SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregation

    Mar 17, 2026Long Li, Zhijian Zhou, Jiangxuan Long +5Text-to-SQLStep-Level Credit Assignment in RL

    Griffith University, Brisbane, Australia · Fudan University, Shanghai, China · Shanghai Innovation Institute, China +4

  26. SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding

    Mar 17, 2026Songcheng Cai, Zhiheng Lyu, Yuansheng Ni +13AI Agent BenchmarksSoftware Engineering Benchmarks

    University of Waterloo · University of Toronto · The Hong Kong University of Science and Technology +2

  27. Joint Optimization of Multi-agent Memory System

    Mar 13, 2026Wenyu Mao, Haoyang Liu, Haosong Tan +4Memory-Augmented Language Model AgentsMulti-Agent System Optimization

    University of Science and Technology of China · Institute of Dataspace, Hefei Comprehensive National Science Center

  28. On Information Self-Locking in Reinforcement Learning for Active Reasoning of LLM agents

    Mar 12, 2026Deyu Zou, Yongqiang Chen, Fan Feng +4RL for LLM AgentsActive Information Gathering

    The Chinese University of Hong Kong · University of California San Diego · Georgia Institute of Technology +1

  29. Can RL Improve Generalization of LLM Agents? An Empirical Study

    Mar 12, 2026Zhiheng Xi, Xin Guo, Jiaqi Liu +11LLM AgentsRL for LLM Agents

    Fudan University · Shanghai Artificial Intelligence Laboratory · Meituan

  30. Evaluate-as-Action: Self-Evaluated Process Rewards for Retrieval-Augmented Agents

    Mar 10, 2026Jiangming Shu, Yuxiang Zhang, Ye Ma +2Retrieval-Augmented ReasoningAgentic RAG

    School of Computer Science and Technology, Beijing Jiaotong University · Hithink Research