RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

Overview

This field studies how to train large language model agents to act over multiple steps in environments such as web browsers, coding tools, games, and simulated workplaces. Common approaches use policy-gradient methods such as PPO and GRPO, verifiable task rewards, and process-level credit assignment to connect outcomes to particular actions or turns. Other methods teach agents to plan hierarchically, manage memory and reusable skills, or coordinate multiple agents, often using simulators and environment-generated feedback. A central challenge is making learning stable and efficient when rewards are sparse and tasks are long, while ensuring gains transfer to new environments and do not come from exploiting flawed reward signals.

Latest papers

CardsList

660 papers68 papers in the last four weeks, down 51% on the four weeks before. 0.5% of all new papers.

  1. Depth Before Breadth: Reverse-to-Forward Curriculum with Advantage Decomposition for LLM Agent

    Nov 18, 2026Siyuan Du, Yuxin Liu, Meng Tang +4RL for LLM AgentsLong-Horizon RL

  2. Communicative On-Policy Distillation: Distilling Black-Box Teacher Capabilities through Communication for Long-Horizon Tasks

    Nov 18, 2026Lirui Luo, Kelong Mao, Xinwei Yang +9On-Policy DistillationKnowledge Distillation

  3. EvoSkill-S1: Evolving Reusable Skills with Genetic-Inspired Operators

    Nov 17, 2026Yujun Wang, Jiqiang Li, Yinuo Lu +10LLM Agent Skill LearningSelf-Evolving Agents

  4. Interactive Structured Environment Scaling for Large Language Model Agentic Reasoning

    Nov 15, 2026Peng Yu, Luoyi Fu, Xinbing Wang +1RL for Language Model ReasoningRL for LLM Agents

  5. ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL

    Nov 13, 2026Zelin He, Siyu Zhu, Jacob Helwig +5LLM Agent HarnessesAgent Harness Optimization

  6. Skill Adapter: Personalized Modification and Online Routing of General-Purpose Skills

    Nov 12, 2026Qijia Zhuang, Yanyin Gou, Wenbin Ma +1LLM Agent Skill RetrievalLLM Agent Routing

  7. Outcome-Mass Alignment: Controlling Length-Dependent Trajectory Influence in Multi-Turn LLM Agent Reinforcement Learning

    Nov 12, 2026Dawei Zhang, Yubin Huangfu, Jessica Fan +5RL for LLM AgentsReinforcement Learning

  8. Reward-Driven Collider Physics Challenges for Self-Improving Agentic AI via Reinforcement Learning

    Nov 11, 2026Alexander Yue, Qibin Liu, Julia GonskiAI Agents for Scientific DiscoverySelf-Improving Agents

  9. Agentic-OPD: Recovering Forgotten Agent Capabilities via On-Policy Distillation for Agentic Tasks

    Nov 9, 2026Tingcheng Bian, Kangye Gong, Jinchang Luo +6On-Policy DistillationRL Post-Training

  10. Agent4Crypto: A Dual-Reflective and Market-Feedback-Aligned LLM-Based Multi-Agent System for Cryptocurrency Trading

    Nov 9, 2026Guangyan Gan, Jiahao Li, Jiashun Wang +1Cryptocurrency TradingAlgorithmic Trading

  11. ECOS: Environment-Calibrated On-Policy Self-Distillation for Language Agents

    Nov 9, 2026Chenhao Zhang, Zhongan Wang, Mingxuan Li +3Self-DistillationOn-Policy Distillation

  12. CARE-PO: Controllable Tool-Call Efficiency for LLM Agents

    Nov 9, 2026Xiaoyang Ye, Xiao Yao, Hairui Zhu +6LLM Agent TrainingLLM Agent Efficiency

  13. SkillDuet: Co-Evolving Models and Skills through Response Feedback

    Nov 9, 2026Xingguang Ji, Ziqiang Dong, Yubo Zhang +4Agent Skill LearningExperience Learning for LLM Agents

  14. JoMoHa: Joint Model--Harness Optimization with Reinforcement Learning

    Nov 8, 2026Xudong Wu, Guangchen Lan, Mingqi Yuan +1Agent Harness OptimizationRL for LLM Agents

  15. BlenderFORGE: Framework for Optimizing Reactive 3D-Graphics Editing Ability of MLLMs

    Nov 7, 2026Zilin Guo, Tianrui Zhang, Yi Rong +63D Scene EditingMultimodal Agents

  16. Represented but Not Used: Value-Policy Dissociation in LLM Agents

    Nov 7, 2026Yuzhang Luo, Mohamed Aghzal, Liangming Pan +1LLM AgentsMechanistic Interpretability

  17. What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?

    Nov 4, 2026Kejian Zhu, Zhuoran Jin, Shangqing Tu +5RL for LLM AgentsReinforcement Learning

  18. LegalEvolver: Agentic Legal Judgment Prediction via Rubric-Guided Self-Evolution

    Nov 4, 2026Xinyu Yang, Jiejun Tan, Chenlong Deng +3Legal Judgment PredictionAgentic RAG

  19. Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning

    Nov 3, 2026Tianming Sha, Junnan Li, Siqi ZhaoMulti-Agent Reinforcement LearningRL for LLM Agents

  20. Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning

    Nov 3, 2026Kevin Frans, Alexander M Rush, Sergey Levine +2RL for LLM AgentsAgentic RL

  21. Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training

    Nov 3, 2026Kangning Zhang, Wenxiang Jiao, Bowen Fang +8Agent Harness OptimizationAgent Harness Self-Evolution

  22. BLPO: Beyond-Local Policy Optimization for Long-Horizon LLM Agents

    Nov 2, 2026Xuanren Chen, Chongyang Tao, Wenpeng Hu +4RL for LLM AgentsLong-Horizon LLM Agents