Agentic RL

RL: Reinforcement Learning

Agentic reinforcement learning studies how language and multimodal models can learn to act over multiple steps in environments, using tools, memory, and feedback to complete tasks. Common approaches extend policy-gradient methods such as PPO and GRPO with process rewards, step-level credit assignment, exploration strategies, and skill or experience reuse; some also use teacher guidance or learned world models. Training and evaluation span simulated and executable tool environments, web and coding tasks, embodied control, and real-world workflows, with systems work addressing the cost of generating long interaction trajectories. A central challenge is balancing sparse or imperfect outcome signals against fine-grained supervision while keeping learning stable, efficient, and robust beyond the training environment.

Latest papers

232 papers63 papers in the last four weeks, up 294% on the four weeks before. 0.4% of all new papers.

AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning
May 1, 2026Haotian Zhao, Songlin Zhou, Yuxin Zhang +9Agentic RL Credit Assignment in RL
Baidu · Tsinghua University · Fudan University
Rethinking Agentic Reinforcement Learning In Large Language Models
Apr 30, 2026Fangming Cui, Ruixiao Zhu, Cheng Fang +2RL for Language Models Agentic RL
Beijing, China · Shanghai, China
FutureWorld: A Live Reinforcement Learning Environment for Predictive Agents with Real-World Outcome Rewards
Apr 29, 2026Zhixin Han, Yanzhi Zhang, Chuyang Wei +11Agentic RL Reinforcement Learning
College of Software, Nankai University · Zhongguancun Academy, Beijing, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences +3
AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
Apr 23, 2026Yuanjie Lyu, Chengyu Wang, Haonan Zheng +4Tool-Augmented Language Model Agents Agentic RL
Alibaba Group, Hangzhou, China
DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data
Apr 21, 2026Venus Team, Sunhao Dai, Yong Deng +10Agentic RL Deep Research Agents
Ant Group
StepPO: Step-Aligned Policy Optimization for Agentic Reinforcement Learning
Apr 20, 2026Daoyu Wang, Qingchuan Li, Mingyue Cheng +4Agentic RL Credit Assignment in RL
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China
LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent
Apr 20, 2026Wanli Li, Bince Qu, Bo Pan +5Agentic RL Deep Research Agents
Zhejiang University · Simplex AI · The Hong Kong Polytechnic University
TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control
Apr 19, 2026Siqi Lai, Pan Zhang, Yuping Zhou +3Agentic RL Language Model-Based Control
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China · Shandong University, Jinan, Shandong, China
MolMem: Memory-Augmented Agentic Reinforcement Learning for Sample-Efficient Molecular Optimization
Apr 14, 2026Ziqing Wang, Yibo Wen, Abhishek Pandy +2Molecular Optimization Sample-Efficient RL
Northwestern University · AbbVie
RTMC: Step-Level Credit Assignment via Rollout Trees
Apr 13, 2026Tao Wang, Suhang Zheng, Xiaoxiao XuStep-Level Credit Assignment in RL Agentic RL
Alibaba Group
Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?
Apr 12, 2026Wanyi Chen, Xiao Yang, Xu Yang +7Agentic RL AI Agent Benchmarks
Soochow University · Microsoft Research Asia · Stony Brook University +2
Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning
Apr 10, 2026Siyuan Xu, Shiyang Li, Xin Liu +9Tool Use RL for Tool Use
The Pennsylvania State University · Amazon · Georgia Institute of Technology
ActFER: Agentic Facial Expression Recognition via Active Tool-Augmented Visual Reasoning
Apr 10, 2026Shifeng Liu, Zhengye Zhang, Sirui Zhao +7Facial Expression Recognition Affective Computing
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, China · Tsinghua University, Beijing, China · University of Science and Technology of China, Hefei, China +2
Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions
Apr 8, 2026Guo Gan, Yuxuan Ding, Cong Chen +3RL for GUI Agents Agentic RL
Zhejiang University · Qualcomm AI Research
Coupled Control, Structured Memory, and Verifiable Action in Agentic AI (SCRAT -- Stochastic Control with Retrieval and Auditable Trajectories): A Comparative Perspective from Squirrel Locomotion and Scatter-Hoarding
Apr 3, 2026Maximiliano Armesto, Christophe KolbAI Agent Reliability Memory-Augmented Agents
Taller Technologies
GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning
Apr 3, 2026Ornith Team, Xiaoya Li, Guoyin Wang +3RL for Code Generation Agentic RL
DeepReinforce Team
ProCeedRL: Process Critic with Exploratory Demonstration Reinforcement Learning for LLM Agentic Reasoning
Apr 2, 2026Jingyue Gao, Yanjiang Guo, Xiaoshuai Chen +1Agentic RL RL for Language Model Reasoning
Institute for Interdisciplinary Information Sciences, Tsinghua University · Shanghai Qizhi Institute · Independent Researcher
6GAgentGym: Tool Use, Data Synthesis, and Agentic Learning for Network Management
Mar 31, 2026Jiao Chen, Jianhua Tang, Xiaotong Yang +1AI Agent Benchmarks 6G Networks
Shenzhen Smart City Technology Development Group Company, Ltd. · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology · China Unicom Group Co., Ltd.
Dynamic Dual-Granularity Skill Bank for Agentic RL
Mar 30, 2026Songjun Tu, Chengdong Xu, Qichao Zhang +5LLM Agent Skill Learning Agent Skill Learning
Institute of Automation, Chinese Academy of Sciences · Pengcheng Laboratory · Sun Yat-Sen University +1
Heddle: A Distributed Orchestration System for Agentic RL Rollout
Mar 30, 2026Zili Zhang, Yinmin Zhong, Chengxu Yang +5LLM Agent Orchestration Agentic RL
School of Computer Science, Peking University · Independent Researcher
PivotRL: High Accuracy Agentic Post-Training at Low Compute Cost
Mar 22, 2026Junkeun Yi, Damon Mosk-Aoyama, Baihe Huang +9RL Post-Training RL for LLM Agents
NVIDIA · UC Berkeley
LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning
Mar 22, 2026Jianing Wang, Jianfei Zhang, Qi Guo +24Large Language Model-Guided Theorem Proving Automated Theorem Proving
Meituan LongCat Team
HISR: Hindsight Information Modulated Segmental Process Rewards For Multi-turn Agentic Reinforcement Learning
Mar 19, 2026Zhicong Lu, Zichuan Lin, Wei Jia +7RL for LLM Agents Agentic RL
Aerospace Information Research Institute, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tencent Hunyuan +1
SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Training
Mar 18, 2026Prince Zizhuang Wang, Shuli JiangRL for LLM Agents Agentic RL
Carnegie Mellon University
What if Pinocchio Were a Reinforcement Learning Agent: A Normative End-to-End Pipeline
Mar 17, 2026Benoît AlcarazAI Agent Safety Agentic RL
Faculty of Sciences, Technology, and Medicine · Doctoral School in Science and Engineering
SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregation
Mar 17, 2026Long Li, Zhijian Zhou, Jiangxuan Long +5Text-to-SQL Step-Level Credit Assignment in RL
Griffith University, Brisbane, Australia · Fudan University, Shanghai, China · Shanghai Innovation Institute, China +4
Gym-V: A Unified Vision Environment System for Agentic Vision Research
Mar 16, 2026Fanqing Meng, Lingxiao Du, Jiawei Gu +9AI Agent Benchmarks RL for VLMs
National University of Singapore. · University of Washington. · The Hong Kong Polytechnic University. +1
ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning
Mar 13, 2026Bangjun Xiao, Yihao Zhao, Xiangwei Deng +9Agentic RL Heterogeneous Computing
School of Computer Science, Peking University · LLM-Core Xiaomi
Meta-Reinforcement Learning with Self-Reflection for Agentic Search
Mar 11, 2026Teng Xiao, Yige Yuan, Hamish Ivison +6Reinforcement Learning Agentic Search
Allen Institute for AI · University of Washington · Independent
OpenClaw-RL: Train Any Agent Simply by Talking
Mar 10, 2026Yinjie Wang, Xuyang Chen, Xiaolong Jin +2Agentic RL Reinforcement Learning

May 1, 2026J/K move · Enter open · S save

AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning

Haotian Zhao, Songlin Zhou, Yuxin Zhang, Stephen S. -T. Yau+8

Baidu · Tsinghua University · Fudan University

Agentic RL Credit Assignment in RL Sparse-Reward RL RL for LLM Agents

Open paper AskPDF arXiv

Reinforcement learning (RL) has substantially improved the ability of large language model (LLM) agents to interact with environments and solve multi-turn tasks. However, effective agentic RL remains challenging: sparse outcome-only rewards provide limited guidance for assigning credit to individual steps within long interaction trajectories. Existing approaches often introduce dense intermediate supervision, such as process reward models or auxiliary self-supervised signals, which increases supervision and tuning complexity and may limit generalization across tasks and domains. We present AEM, a supervision-free credit assignment method that adaptively modulates entropy dynamics during RL training to improve the exploration-exploitation trade-off. Since in agentic RL the environment is typically affected by a complete response, rather than an individual token, our analysis lifts entropy dynamics from the token level to the response level, aligning uncertainty estimation with the effective action granularity of LLM agents and reducing sensitivity to token-level sampling noise. We further show that entropy drift under natural-gradient updates is governed by the interaction between the sampled-response advantage and its relative surprisal. Motivated by this result, AEM derives a practical response-level uncertainty proxy and uses it to rescale advantages, leveraging the evolving balance between positive and negative samples to naturally transition from exploration to exploitation. Extensive experiments on ALFWorld, WebShop, and SWE-bench-Verified with models ranging from 1.5B to 32B demonstrate that AEM consistently improves strong RL baselines, including a +1.4% gain when integrated into a state-of-the-art software-engineering RL training framework.