RL for LLM Agents
RL: Reinforcement Learning · LLM: Large Language Model
Overview
This field studies how to train large language model agents to act over multiple steps in environments such as web browsers, coding tools, games, and simulated workplaces. Common approaches use policy-gradient methods such as PPO and GRPO, verifiable task rewards, and process-level credit assignment to connect outcomes to particular actions or turns. Other methods teach agents to plan hierarchically, manage memory and reusable skills, or coordinate multiple agents, often using simulators and environment-generated feedback. A central challenge is making learning stable and efficient when rewards are sparse and tasks are long, while ensuring gains transfer to new environments and do not come from exploiting flawed reward signals.
Latest papers
660 papers68 papers in the last four weeks, down 51% on the four weeks before. 0.5% of all new papers.
SELAUR: Self Evolving LLM Agent via Uncertainty-aware Rewards
How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1
Proximity-Based Multi-Turn Optimization: Practical Credit Assignment for LLM Agent Training
MagicAgent: Towards Generalized Agent Planning
AgentConductor: Topology Evolution for Multi-Agent Competition-Level Code Generation
Phase-Aware Mixture of Experts for Agentic Reinforcement Learning
EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments
HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents
Beyond Static Pipelines: Learning Dynamic Workflows for Text-to-SQL
Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation
Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning
TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents
FlexMARL: Graph-Driven Rollout-Training Co-Design for LLM-Based Multi-Agent Reinforcement Learning
Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Reinforcement World Model Learning for LLM-based Agents
Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning
PEARL: Self-Evolving Assistant for Time Management with Reinforcement Learning
MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards
AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning
Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space
Iterative Policy Optimization: Batch-Online Multi-Turn RL via Single-Turn RL
Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
Learning Robust Social Strategies with Large Language Models
Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective
There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning
Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agents
RL-Focal, a two-stage RL agent framework that routes and ensembles LLMs. First, we develop the Decider RL-agent, which learns to dynamically select an ensemble of small size () among LLMs () for incoming queries from a user-defined downstream task , by maximizing both error-diversity and reasoning-performance of the selected ensemble through iterative updates of task-adaptive rewards and policy. Second, to enable effective fusion of dynamically selected LLMs, we develop the stage-2 Fusion RL-agent, which learns to resolve reasoning conflicts from different LLMs and dynamically adapt to different ensemble teams composed by the Decider Agent for different downstream tasks. {\em Third}, we introduce the focal diversity metric to better model the error correlations among multiple LLMs further improving the generalization performance of the Decider Agent, which actively prunes the ensemble combinations. By focal diversity, we enhance performance across tasks by effectively promoting reward-aware and policy-adaptive ensemble selection and inference fusion. Extensive evaluations on five benchmarks show that RL-Focal achieves the performance improvement of 8.48% with an ensemble of small size compared to the best individual LLM in a pool and offers stronger robustness. Code is available here.