LLM Agent Training

LLM: Large Language Model

LLM agent training studies how to make language models act reliably over multiple steps, using tools, software, websites, or simulated worlds to complete tasks. Common approaches combine supervised fine-tuning on interaction traces with reinforcement learning from execution-verified outcomes, alongside distillation, preference learning, and feedback on individual steps. Researchers build executable environments and benchmarks such as SWE-bench, WebArena, Terminal-Bench, and τ²-Bench to provide repeatable practice and test tool use, coding, search, and long-horizon workflows. A central challenge is transferring gains beyond training environments while balancing task success with efficiency, safety, and robustness to changing tools and users.

Latest papers

590 papers130 papers in the last four weeks, up 132% on the four weeks before. 0.8% of all new papers.

GrepSeek: Training Search Agents for Direct Corpus Interaction
May 28, 2026Alireza Salemi, Chang Zeng, Atharva Nijasure +4Tool-Augmented Language Model Agents Agentic Search
University of Massachusetts Amherst · Princeton University · Carnegie Mellon University
DynSess: Dynamic Session-Level Evaluation and Optimization Framework for Role-Playing Agents
May 28, 2026Rongsheng Zhang, Jiji Tang, Junnan Ren +6Large Language Model-Based Role-Play Simulation Long-Horizon Agent Evaluation
1Zhejiang University · 2Fuxi AI Lab, NetEase Inc. · 3Xiamen University
unix-ctf: Procedural Environments for Unix-Competence Reinforcement Learning
May 27, 2026Geoffrey Bradway, Roger Creus Castanyer, Lorenz Wolf +3Terminal-Agent Benchmarks AI Agent Benchmarks
Vmax
DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agents
May 27, 2026Jiamian Wang, Ruiyi Zhang, Tong Yu +5Multimodal IR Multimodal QA
Rochester Institute of Technology · ⋆Work done during an internship at Adobe Research. · Adobe Research
Learn from Weaknesses: Automated Domain Specialization for Small Computer-Use Agents
May 27, 2026Suji Kim, Kangsan Kim, Sung Ju HwangComputer-Use Agents Synthetic Data Generation
KAIST · Samsung Electronics
Plan Before Search: Search Agents Need Plan
May 27, 2026Zhipeng Qian, Zihan Liang, Yufei Ma +7Multi-Hop QA Tool-Augmented Language Model Agents
1Kuaishou Technology · 2Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · 3VCIP, CS, Nankai University
Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?
May 27, 2026Yibo Zhao, Zichen Ding, Jiayi Wu +2LLM Agents Information Retrieval
School of Data Science and Engineering, East China Normal University · Shanghai AI Laboratory
Hierarchical Prompt-Domain Control and Learning for Resource-Constrained Agentic Language Models
May 26, 2026Joan Vendrell Gallart, Russell Bent, Michael GrosskopfContinual Learning for LLM Agents Language Model-Based Control
Department of Mechanical and Aerospace University of California Irvine Irvine, CA 92617-4322, USA · T-5 Los Alamos National Laboratory Los Alamos, NM 88220, USA · CAI-4 Los Alamos National Laboratory Los Alamos, NM 88220, USA
You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agents
May 26, 2026Nicole Hsing, Asuka Yuxi Zheng, Yi Zhao +2Social Dilemmas AI Alignment
1Arcarae · University of California, Santa Cruz · 3Northwestern University +1
Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments
May 26, 2026Yuxin Chen, Xiaodong Cai, Junfeng Fang +9LLM Agents LLM Agent Training
National University of Singapore · Meituan · Tsinghua University +2
Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement
May 26, 2026Dingwei Chen, Zefang Zong, Zhipeng Ma +5Agentic RL LLM Agent Training
Tencent Inc · The Chinese University of Hong Kong · Shenzhen MSU-BIT University
UnityMAS-O: A General RL Optimization Framework for LLM-Based Multi-Agent Systems
May 26, 2026Yiqun Chen, Wei Yang, Erhan Zhang +14Multi-Agent LLM Systems Multi-Agent System Optimization
Renmin University of China · Xiaohongshu Inc.
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
May 26, 2026MiniMax, :, Aili Chen +204Mixture-of-Experts Language Models Agentic RL
CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents
May 25, 2026Bowen Wang, Dunjie Lu, Junli Wang +11Computer-Use Agents Synthetic Data Generation
The University of Hong Kong · Qwen Team, Alibaba Inc. · University of California, San Diego +1
HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol
May 25, 2026Xinyi Mou, Siyuan Wang, Zejun Li +2Agent Communication Protocols LLM Agent Training
Fudan University · The Chinese University of Hong Kong · King’s College London +2
LipoAgent: Coordinating Fine-Tuned LLM Agents for Safer Lipid Design
May 24, 2026Leshu Li, An Lu, Haiyu Wang +5Multi-Agent LLM Systems Materials Science
New York University, USA · University of Illinois Chicago, USA
Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents
May 23, 2026Yuxin Zhang, Mengxue Hu, Zheng Lin +8LLM Agent Training LLM Routing
1Fudan University · 2Alibaba Group · 3The University of Hong Kong +4
ECHO: Terminal Agents Learn World Models for Free
May 23, 2026Vaishnavi Shrivastava, Piero Kauffmann, Ahmed Awadallah +1World Model Learning Computer-Use Agents
Microsoft Research
SEAL: Synergistic Co-Evolution of Agents and Learning Environments
May 23, 2026Yihao Hu, Zhihao Wen, Xiujin Liu +3LLM Agent Training Self-Improving Agents
Ant Group · Westlake University · University of Michigan–Ann Arbor +1
QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks
May 22, 2026Jian Xie, Tianhe Lin, Zilu Wang +16Deep Research Agents LLM Agent Training
The Ohio State University · Amazon AGI SF Lab
EVE-Agent: Evidence-Verifiable Self-Evolving Agents
May 21, 2026Yamato Arai, Yuma IchikawaLLM Agent Self-Improvement LLM Agent Training
Fujitsu Limited Department of Basic Science · The University of Tokyo · Fujitsu Limited +1
Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning
May 21, 2026Banghao Chi, Yining Xie, Mingyuan Wu +9Tool-Using Agents LLM Agent Training
University of Illinois Urbana-Champaign · Meta
Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost
May 21, 2026Simon Dennis, Rivaan Patil, Kevin Shabahang +1LLM Agent Orchestration Agentic Workflows
i14, University of Melbourne · i14
Efficient Agentic Reasoning Through Self-Regulated Simulative Planning
May 21, 2026Mingkai Deng, Jinyu Hou, Lara Sá Neves +4LLM Planning World Model-Based Planning
Carnegie Mellon University · Institute of Foundation Models (IFM)
Terminal-World: Scaling Terminal-Agent Environments via Agent Skills
May 20, 2026Zihao Cheng, Hongru Wang, Zeming Liu +6Tool-Augmented Language Model Agents Terminal Agents
School of Computer Science and Engineering, Beihang University, Beijing, China · Independent Researcher · 3Beijing Institute of Technology +2
ClinSeekAgent: Automating Multimodal Evidence Seeking for Agentic Clinical Reasoning
May 19, 2026Juncheng Wu, Letian Zhang, Yuhan Wang +9Healthcare Clinical Decision Support
UC Santa Cruz
Weasel: Out-of-Domain Generalization for Web Agents via Importance-Diversity Data Selection
May 19, 2026Fatemeh Pesaran Zadeh, Seyeon Choi, Xing Han Lù +2Diversity-Aware Sampling OOD Generalization
Seoul National University · McGill University · Mila – Quebec AI Institute +1
EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
May 18, 2026Minrui Xu, Zilin Wang, Mengyi DENG +12Tool-Augmented Language Model Agents Agentic RL
LARK, HKUST (GZ) · University of Cambridge · UCL +2
DuIVRS-2: An LLM-based Interactive Voice Response System for Large-scale POI Attribute Acquisition
May 18, 2026Le Zhang, Shengming Zhang, Rui Zha +3Spoken Dialogue Systems LLM Agent Training
Baidu Inc., Beijing, China
HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents
May 18, 2026Woongyeng Yeo, Yumin Choi, Taekyung Ki +1Credit Assignment in RL LLM Agent Training
KAIST

May 28, 2026J/K move · Enter open · S save

GrepSeek: Training Search Agents for Direct Corpus Interaction

Alireza Salemi, Chang Zeng, Atharva Nijasure, Jui-Hui Chung+3

University of Massachusetts Amherst · Princeton University · Carnegie Mellon University

Tool-Augmented Language Model Agents Agentic Search LLM Agent Training Knowledge-Intensive QA RL for LLM Agents

Open paper AskPDF arXiv

Large Language Model (LLM) search agents have shown strong promise on knowledge-intensive tasks through iterative reasoning and retrieval. Most existing systems rely on retrievers that return ranked documents from a pre-built index. We explore a complementary paradigm in which the agent treats the corpus as the search environment and finds evidence through executable shell commands. We introduce GrepSeek, an optimized direct corpus interaction (DCI) agent that learns to find, filter, and compose evidence over large text corpora. To stabilize reinforcement learning (RL) over large corpora, we train in two stages: first, we initialize the policy using verified, causally grounded search trajectories generated by an answer-aware Tutor and an answer-blind Planner; then, we refine the policy using Group Relative Policy Optimization (GRPO). To make DCI practical at scale, we introduce two semantics-preserving execution optimizations: Pruned Adaptive Command Execution, which reduces shell-based search latency by up to

77\times

on a 14GB corpus with 21 million documents using a compact auxiliary structure, and Sharded-Parallel Corpus Search, which achieves up to

7.6\times

speedup without additional preprocessing; both preserve equivalence with sequential execution. Across eight open-domain QA benchmarks, GrepSeek achieves the strongest overall performance, with a statistically significant relative improvement of

5.7\%

over the best baseline. Our analysis shows how DCI-optimized agents conduct flexible and effective compositional search through direct corpus interaction.