RL for Tool Use

RL: Reinforcement Learning

This field studies how language and multimodal agents can use tools such as search, code execution, databases, APIs, and graphical interfaces to solve multi-step tasks. Common approaches combine supervised demonstrations with policy optimization, often using executable environments and rewards for task success, valid calls, and efficient execution. A major focus is improving credit assignment across long interactions through turn-level rewards, process supervision, and experience or trajectory reuse. Open challenges include preventing tool overuse and training collapse while ensuring that skills learned in simulated or cached environments transfer reliably to changing real-world tools.

Latest papers

192 papers27 papers in the last four weeks, up 35% on the four weeks before. 0.2% of all new papers.

Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Model
Apr 20, 2026Chenming Tang, Hsiu-Yuan Huang, Weijie Liu +3LLM Agent Training RL for Tool Use
National Key Laboratory for Multimedia Information Processing, Peking University · School of Computer Science, Peking University · LLM Department, Tencent
Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis
Apr 16, 2026Zhiyuan Zhai, Wenjing Yan, Xiaodan Shao +1LLM Agent Training RL for LLM Agents
Fudan University · Chinese University of Hong Kong · University of Waterloo
UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization
Apr 15, 2026Zhengxi Lu, Fei Tang, Guangyi Liu +8Long-Horizon LLM Agents Mobile GUI Automation
Zhejiang University · Apple · Tencent
ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Execution
Apr 15, 2026Shouzheng Huang, Meishan Zhang, Baotian Hu +1LLM Tool Use LLM Agent Training
Harbin Institute of Technology (Shenzhen)
Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning
Apr 10, 2026Siyuan Xu, Shiyang Li, Xin Liu +9Tool Use RL for Tool Use
The Pennsylvania State University · Amazon · Georgia Institute of Technology
E3-TIR: Enhanced Experience Exploitation for Tool-Integrated Reasoning
Apr 10, 2026Weiyang Guo, Zesheng Shi, Liye Zhao +5Tool-Integrated Reasoning LLM Agent Training
Harbin Institute of Technology, Shenzhen, China · Huawei Technologies Co., Ltd. · Hong Kong University of Science and Technology
Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models
Apr 9, 2026Shilin Yan, Jintao Tong, Hongwei Xue +6Tool Use Multimodal Agents
Accio Team, Alibaba Group · Huazhong University of Science and Technology
TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning
Apr 9, 2026Yifei Gong, Xing Wu, Wenda Liu +1Text-to-CAD Generation LLM Tool Use
School of Computer Engineering & Science, Shanghai University
SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents
Apr 9, 2026Xinshun Feng, Xinhao Song, Lijun Li +2Self-Evolving Agents Memory-Augmented Agents
Shanghai Artificial Intelligence Laboratory · Shanghai Jiaotong University
Multi-Turn Reinforcement Learning for Tool-Calling Agents with Iterative Reward Calibration
Apr 3, 2026Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai +1Multi-Turn RL RL for Tool Use
Amity Research and Application Center (ARAC)
SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
Apr 2, 2026Zhengxi Lu, Zhiyuan Yao, Jinyang Wu +7LLM Agent Skill Learning RL for Tool Use
Zhejiang University · Meituan · Tsinghua University
Tool-Augmented Agent for Closed-loop Optimization,Simulation,and Modeling Orchestration
Apr 1, 2026Liyuan Deng, Shujian Deng, Yongkang Chen +6LLM Agent Orchestration Constrained Optimization
Northwestern Polytechnical University · Shanghai Artificial Intelligence Laboratory
Agentic Tool Use in Large Language Models
Apr 1, 2026Jinchao Hu, Meizhi Zhong, Kehai Chen +2LLM Agent Evaluation Tool-Using Agents
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China · Independent Researcher, China
6GAgentGym: Tool Use, Data Synthesis, and Agentic Learning for Network Management
Mar 31, 2026Jiao Chen, Jianhua Tang, Xiaotong Yang +1AI Agent Benchmarks 6G Networks
Shenzhen Smart City Technology Development Group Company, Ltd. · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology · China Unicom Group Co., Ltd.
Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards
Mar 25, 2026Cheng Jiayang, Xin Liu, Zhihan Zhang +8LLM Agent Orchestration RL for Tool Use
Amazon Services Inc., Palo Alto, CA, USA. · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China. · School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA, USA.
Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe
Mar 23, 2026Xixi Wu, Qianguo Sun, Ruiyang Zhang +4Travel Planning RL for Tool Use
The Chinese University of Hong Kong · IDEA Research · University of Macau
SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Training
Mar 18, 2026Prince Zizhuang Wang, Shuli JiangRL for LLM Agents Agentic RL
Carnegie Mellon University
TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas
Mar 17, 2026Ai Jian, Xiaoyun Zhang, Eryu Guo +6Schema Linking Group Relative Policy Optimization
Beijing University of Posts and Telecommunications, Beijing, China · State Key Lab of Processors, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences +1
TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMs
Mar 11, 2026Yutao Xie, Nathaniel Thomas, Nicklas Hansen +3RL for Language Model Reasoning Reward Shaping
UC San Diego · AWS AI
OpenClaw-RL: Train Any Agent Simply by Talking
Mar 10, 2026Yinjie Wang, Xuyang Chen, Xiaolong Jin +2Agentic RL Reinforcement Learning
Improving Search Agent with One Line of Code
Mar 10, 2026Jian Li, Dongsheng Chen, Zhenhua Xu +5RL for Tool Use Group Relative Policy Optimization
Nanjing University · Tencent YoutuLab
In-Context Reinforcement Learning for Tool Use in Large Language Models
Mar 9, 2026Yaoqi Ye, Yiran Zhao, Keyu Duan +4RL for Tool Use In-Context RL
National University of Singapore · Salesforce AI Research · University of California Berkeley +1
SynPlanResearch-R1: Encouraging Tool Exploration for Deep Research with Synthetic Plans
Mar 9, 2026Hansi Zeng, Zoey Li, Yifan Gao +7Synthetic Data Generation for LLM Agents LLM Agent Training
University of Massachusetts Amherst · Amazon · University of Montreal +2
AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints
Mar 7, 2026Yirong Zeng, Xiao Ding, Yufei Liu +9RL for Tool Use Efficient Language Model Reasoning
Harbin Institute of Technology, SCIR Lib · Peking University · Huawei Technologies Ltd.
Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery
Mar 6, 2026Lin Fan, Pengyu Dai, Zhipeng Deng +4Self-Evolving Agents LLM Agent Skill Learning
Southwest Jiaotong University, China · The University of Tokyo, Japan · RIKEN, Japan +2
Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
Mar 5, 2026Karan Gupta, Pranav Vajreshwari, Yash Pandya +3RL for Tool Use Agentic RL
Microsoft Research
EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection
Mar 5, 2026Shuo Yang, Soyeon Caren Han, Xueqi Ma +3Policy Optimization LLM Agent Self-Improvement
School of Computing and Information Systems, The University of Melbourne
The Tool-Overuse Illusion: Why Does LLM Prefer External Tools over Internal Knowledge?
Mar 3, 2026Yirong Zeng, Shen You, Yufei Liu +9LLM Tool Use RL for Tool Use
Harbin Institute of Technology, SCIR · Huawei Technologies Co., Ltd · Peking University
CoVe: Training Interactive Tool-Use Agents via Constraint-Guided Verification
Mar 2, 2026Jinpeng Chen, Cheng Gong, Hanbo Li +9Tool-Using Agents LLM Agent Training
Huawei Research · Independent Researcher
TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training
Mar 2, 2026Jinluan Yang, Yuxin Liu, Zhengyu Chen +7LLM Agent Training RL for Tool Use
Zhejiang University, Hangzhou, China · Meituan, Beijing, China.

Apr 20, 2026J/K move · Enter open · S save

Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Model

Chenming Tang, Hsiu-Yuan Huang, Weijie Liu, Junqiang Zheng+2

National Key Laboratory for Multimedia Information Processing, Peking University · School of Computer Science, Peking University · LLM Department, Tencent

LLM Agent Training RL for Tool Use

Open paper AskPDF arXiv

Reinforcement learning has become a prevalent paradigm for training tool calling agents, which typically requires online interactive environments. Existing approaches either rely on training data with ground truth annotations or require advanced proprietary language models (LMs) to synthesize environments that remain fixed once created. In this work, we propose TRUSTEE, a cost-friendly method for training tool calling agents with dynamic environments fully simulated by open-source LMs that can be as small as 8B, including task generation, user simulation, tool simulation, and trajectory evaluation, paired with an adaptive curriculum learning mechanism that controls task difficulty dynamically. Our empirical results show that TRUSTEE outperforms baselines which require extra external resources in most cases. These confirm that, with a sufficiently sophisticated design, even simulated environments with a local 8B LM as the backbone could set a strong baseline for tool learning. We hope our proposed paradigm could democratize tool learning and inspire future research on environment scaling with limited resources.