RL for Tool Use

RL: Reinforcement Learning

This field studies how language and multimodal agents can use tools such as search, code execution, databases, APIs, and graphical interfaces to solve multi-step tasks. Common approaches combine supervised demonstrations with policy optimization, often using executable environments and rewards for task success, valid calls, and efficient execution. A major focus is improving credit assignment across long interactions through turn-level rewards, process supervision, and experience or trajectory reuse. Open challenges include preventing tool overuse and training collapse while ensuring that skills learned in simulated or cached environments transfer reliably to changing real-world tools.

Latest papers

192 papers27 papers in the last four weeks, up 35% on the four weeks before. 0.2% of all new papers.

MRC-MMAgent: Reinforcement Learning for Multi-Role Collaboration in Multimodal Tool-Using Agents
Oct 29, 2026Shengqin Wang, Jie Jin, Xiaofan Li +8Multimodal Agents RL for Tool Use
When Fine-Grained Credit Assignment Helps and Hurts: Multimodal Agentic Reinforcement Learning through Video Editing
Oct 26, 2026Lei Zhao, Zongheng Cao, Xinyu HuReinforcement Learning Fine-Grained Credit Assignment
University of Pennsylvania · Philo Labs
Training Small Language Models for Transferable Tool Selection
Oct 22, 2026Zhaoyi Zhou, Akshayaa Magesh, Jeongyeol Kwon +4Tool Selection for Language Model Agents Small Language Models
Meta · Carnegie Mellon University
ReflectPO: Reflection-Guided Policy Optimization for Long-Horizon LLM Agents
Oct 22, 2026Rumei Li, Xunliang CaiLong-Horizon LLM Agents RL for LLM Agents
Meituan, China
ToolOPD: Credit-Aware On-Policy Distillation for Tool-Use Agents
Oct 19, 2026Chu Zhao, Lei Tang, Yanming Wang +5On-Policy Distillation Step-Level Credit Assignment in RL
Northeastern University · Honor Device Co., Ltd
CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents
Oct 18, 2026Zhongwei Xie, Yiming Du, Jiaxin Bai +10LLM Agent Training RL for Tool Use
The Hong Kong University of Science and Technology · The Chinese University of Hong Kong · Hong Kong Baptist University +1
Not Every User Says It All: Training Tool-Using Agents Beyond Ideal Simulated Users
Oct 16, 2026Han Bao, Zheyuan Zhang, Kaiwen Shi +1Tool-Using Agents RL for Tool Use
University of Notre Dame
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
Oct 15, 2026Siqi Chen, Zhengxi Lu, Yuchen Yan +10RL for LLM Agents Reinforcement Learning
Zhejiang University · Tencent
Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine
Oct 15, 2026Junyao Yang, Zongxia Li, Yucheng Shi +6Terminal Agents Recursive Self-Improvement
Lean4AgenticRL: Advancing Agentic RL with Verifiable Multi-level Process Rewards
Oct 15, 2026Ruida Wang, Gerald Friedland, Tong Zhang +1Agentic RL RL for Tool Use
Amazon Web Services · University of Illinois Urbana-Champaign
DeciCAD: Harness-Grounded Learning of Action Policies for CAD Agents
Oct 15, 2026Yufeng Peng, Yixin Zhang, Ziyi Liu +5LLM Agent Harnesses Language Model-Based Control
University of Science and Technology of China · Nanyang Technological University
Long-Horizon RL Needs Muon
Oct 14, 2026Kai Ruan, Jinghao Lin, Tao Chen +3Long-Horizon RL Long-Horizon LLM Agents
Gaoling School of Artificial Intelligence, Renmin University of China · Duke University · Institute of Automation, Chinese Academy of Sciences +1
Where to Start, How to Move: Analyzing Curricula under GRPO for Long-Horizon LLM Agents
Oct 14, 2026Minchan Kim, Sungho Bae, Youngjin Kim +1Long-Horizon LLM Agents Curriculum RL
Graduate School of Data Science, Seoul National University · Department of Computer Science and Engineering, Konkuk University
Teacher-Guided Tool Learning for Small Language Models via Bidirectional KL Constraints
Oct 13, 2026Haoxuan Gao, Qin-Wen Luo, Xiang Chen +5RL for Tool Use Reinforcement Learning
Nanjing University of Aeronautics and Astronautics · Security Capability Center · Nanjing University +2
Agent-Reflex: Teaching Language Agents to Act Reflectively
Oct 13, 2026Guanting Dong, Xinyu Yang, Jiajie Jin +9LLM Agent Training Self-Reflection in Language Model Agents
Gaoling School of Artificial Intelligence, Renmin University of China · National University of Singapore
X-Ragent: Tool-Augmented Reinforcement Learning for Evidence-Grounded Chest X-Ray Reasoning
Oct 12, 2026Yi Wang, Zhenlong Yuan, Yuchan Guo +9Medical VLMs Medical Imaging
Institute of Computing Technology · University of Chinese Academy of Sciences · Carnegie Mellon University +6
Of Errors and Echoes: Responsibility-Aware Credit Assignment for Tool-Use Agents
Oct 12, 2026Xingjian Diao, Tianyu Yang, Wenjun Huang +8Credit Assignment Fine-Grained Credit Assignment
Dartmouth College · University of Notre Dame · University of California, Irvine +3
ObsSpec: Accelerating Agentic Reinforcement Learning with Observation Speculation
Oct 11, 2026Kyle Montgomery, Wang Zhang, Chenguang WangAgentic RL Tool-Using Agents
UC Santa Cruz · ByteDance Inc.
Self-Evolving Visual Agent via Margin Reward
Oct 10, 2026Wenxi Yang, Yuzhong Zhao, Baochang Zhang +2Tool-Using Vision-Language Agents RL for Tool Use
UCAS · Microsoft Research Asia · Beihang University
MCPDisCo: Seesaw-Mitigated Tuning for MCP-Native Agents
Oct 10, 2026Kounianhua Du, Jianxing Liu, Shuai Shao +3RL for LLM Agents Agentic RL
Shanghai Jiao Tong University, Shanghai, China · Tencent. Shanghai, China
Self-Evolve With a Reference:Anchored Training of Tool-Integrated Agents
Oct 7, 2026Wenjie Liao, Liangjie Zhao, Zehong CaoLLM Agent Self-Improvement Self-Evolving Agents
Waseda University · Adelaide University
FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents
Oct 6, 2026Jia Liufu, Bin Hu, Linglin Jing +5RL for Code Generation Software Engineering Agents
NVIDIA · Tsinghua University
CIPO: Counterfactual Imagination Policy Optimization for Adaptive Tool Granularity Selection
Oct 4, 2026Yu Li, Yunlu Wan, Zijian Zhu +4LLM Agent Skill Learning RL for Tool Use
School of Computer Science and Engineering, Southeast University, Nanjing, China · KTH Royal Institute of Technology · Huawei Noah’s Ark Lab, China
HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents
Sep 29, 2026Tongbo Chen, Junbo Niu, Zhengxi Lu +8Computer-Use Agents GUI Agents
Zhejiang University · Peking University · Tsinghua University
RLTLDR: Self-improvement by Internalizing Self-generated Feedback
Sep 29, 2026Michael Kirchhof, Eleonora Gualdoni, Andrew Szot +6RL for Code Generation Reinforcement Learning with Verifiable Rewards
Apple
ContractRL: Shielded Group-Relative Policy Optimization for Auditable Tool-Call Repair
Sep 29, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +4LLM Self-Correction RL for Tool Use
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
Neuro-Symbolic Computer Use: Learning Reusable Policies for Reliable and Efficient Execution
Sep 29, 2026Hyewon Suh, Thanh Minh Nguyen, Chih-Lun Lee +5Computer-Use Agents Runtime Enforcement for AI Agents
Georgia Institute of Technology · Simular · Stanford University
WEFT: Scaling Tool-Use Post-Training for General-Purpose Agents
Sep 29, 2026Bo Mao, Hang He, Linting Wang +17LLM Agent Self-Improvement LLM Post-Training
East China Normal University · Shanghai Innovation Institute · Fudan University +2
MLToolBench: Learning Tool-Augmented Agents for Machine Learning Development
Sep 29, 2026Xin Yu, Lizhu Zhang, Jiamu Bai +7LLM Agent Skill Learning AI Agent Benchmarks
ParaAgent: Reinforcing Parallel Acting in Open-World Tool Environments
Sep 27, 2026Shengbin Yue, Hongru Wang, Siyuan Wang +3LLM Agent Training Tool Use
Fudan University · University of Edinburgh · Chinese University of Hong Kong +3

Oct 29, 2026J/K move · Enter open · S save

MRC-MMAgent: Reinforcement Learning for Multi-Role Collaboration in Multimodal Tool-Using Agents

Shengqin Wang, Jie Jin, Xiaofan Li, Wentao Yan+7

Multimodal Agents RL for Tool Use Multimodal Tool Use Multi-Agent Collaboration

Open paper AskPDF arXiv

Multimodal tool-using agents augment vision-language models with access to external information and capabilities, enabling them to solve knowledge-intensive and visually complex tasks that cannot be addressed using visual context alone. However, existing reinforcement learning pipelines largely train single-role agents to search and answer using terminal rewards, overlooking how humans inspect retrieved evidence, reformulate queries, and revise conclusions during search. Consequently, they fail to optimize this self-corrective process over complete interaction trajectories. We propose MRC-MMAgent, a multi-role collaborative reinforcement learning framework that incorporates trajectory assessment and continuation directly into the executable agent loop. First, the framework establishes a multi-role collaborative loop that integrates solving, inspection, and continuation, enabling the agent to acquire evidence through tools, evaluate the preceding trajectory, and resume problem solving when the current answer is insufficiently supported. Second, we introduce a role-coupled learning objective that combines task success, interaction validity, and successful correction with full-trajectory policy optimization across the decisions generated by all roles. Extensive experiments show that, using only the original baseline training set without constructing additional datasets, our method consistently outperforms the corresponding base models and achieves leading or highly competitive performance across benchmarks.