RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

70 papers in the last four weeks, up 141% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 402

All topics

Headroom-Drift Replay: A Primitive for Principled Replay Control in GRPO
Sep 3, 2026Hyun Bin Park, Du-Seong ChangRL for Language Model Reasoning Group Relative Policy Optimization
Department of Artificial Intelligence, Sogang University, Seoul, Republic of Korea
StrixAE: An Intelligent Agent for Audio Enhancement under Complex Distortion Coupling in Real-World Scenarios
Sep 3, 2026Chenglin Wu, Junjie Wu, Jinhang Chen +5Speech Enhancement RL for LLM Agents
1Xiamen University · 2Fuyao University of Science and Technology
TIGPO: Temporal Instance-Graph Policy Optimization for Long-Horizon LLM Agents
Sep 3, 2026Jinwei GanGroup Relative Policy Optimization Credit Assignment in RL
Department of Computer Science Nanjing University
Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents
Sep 2, 2026Yanting Yang, Can Jin, Jinman Zhao +6LLM Agent Workflow Optimization LLM Agent Skill Learning
Rutgers University · University of Toronto · The Hong Kong Polytechnic University +2
The Rise of Verbal Reinforcement Learning
Sep 1, 2026Kshitij Tayal, Arun Sharma, Genta Indra Winata +2RL for Language Models Reinforcement Learning
1AI Foundations, Capital One · University of Minnesota, USA
ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning
Sep 1, 2026Fanrui Zhang, Ruixue Ding, Qiang Zhang +13AI Agent Benchmarks Rubric-Based RL
Alibaba ATH Token Foundry · Hema, Alibaba Group
VerNav: Verifier-First Low-Latency Vision-and-Language Navigation
Sep 1, 2026Zhixin Wang, Chengzheyi Yao, Leyuan Liu +2Efficient VLM Inference Vision-Language Navigation
University of Electronic Science and Technology of China
One Policy, Any Budget: Internalizing Budget-Aware Search via Reinforcement Learning
Sep 1, 2026Xiaowei Sun, Jin Li, Yili Hong +2Cost-Aware Inference Constrained RL
College of Computer Science and Artiﬁcial Intelligence, Fudan University · College of Software Engineering, Southeast University · School of Artiﬁcial Intelligence, Shanghai Jiao Tong University
How do World Models and Policies Compose in LLM Agents? A Joint Spectral and Behavioral Account
Aug 30, 2026Ruize Xu, Xiao Yu, Yujin Tang +2World Model Learning World Models
Dartmouth College · Columbia University
SearchWiki: Learning to Build and Navigate Knowledge Wikis for Active Information Seeking
Aug 30, 2026Guransh Singh, Vishwajeet Kumar, Arkadeep Acharya +3Retrieval-Augmented Generation Agentic Retrieval
Birla Institute of Technology, Mesra · IBM
A^2Agent: Action-Aware Reinforcement Learning for Repository-Level Code Localization Agents
Aug 30, 2026Doyeon Kim, Suyoung Bae, Yumin Lee +1Software Engineering Agents Credit Assignment in RL
College of Computing and Informatics Sungkyunkwan University, South Korea
Harness-RL: Black-Box Reinforcement Learning with Action-Args Decoupling for Central-Agent Multi-Agent Harnesses
Aug 30, 2026Xinke Jiang, Zhixin Zhang, Zhibang Yang +6Multi-Agent LLM Systems Agent Harness Optimization
National Engineering Research Center of Software Engineering, Peking University, Beijing, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +3
AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing
Aug 30, 2026Xinke Jiang, Yue Fang, Zhibang Yang +12Retrieval-Augmented Generation Agentic RAG
National Engineering Research Center of Software Engineering, Peking University, Beijing, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +3
Learning Generalizable Behaviors for Terminal Agents
Aug 23, 2026Yihang Yao, Bo Pang, Xuan Phi Nguyen +3Terminal Agents Generalization in RL
Salesforce AI Research · Carnegie Mellon University
HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning
Aug 22, 2026Yucan Guo, Xiaohan Wang, Miao Su +8Credit Assignment RL for LLM Agents
State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +1
SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning
Aug 20, 2026Dayang Liang, Lang Feng, Bo An +1RL for Language Models Agentic RL
Xiamen University, China
SPADE: Self-Play in Adaptive Synthetic Executable Environments
Aug 19, 2026Bo Liu, Simon Yu, Yiding Jiang +15Self-Play RL LLM Agent Self-Improvement
University of Washington · Stanford University · Northeastern University +6
Training Agents to Evolve with Their Harness: TaoLive Digital Avatar Agent Technical Report
Aug 16, 2026TaoLive AIGC LLM Team, Yuhan Sun, Wenhao Lin +7LLM Agent Training LLM Agent Harnesses
TaoLive AIGC LLM Team
Beyond Outcome Rewards: Step-Level Self-Distilled Policy Optimization for Deep Search Agents
Aug 13, 2026Haoze Wu, Chuqiao Kuang, Tianyi Zhuang +1Group Relative Policy Optimization Credit Assignment in RL
The Hong Kong University of Science and Technology · Huawei Technologies Ltd.
Scaling Automatic Research Agents via World Models
Aug 12, 2026Xiyuan Yang, Sheikh Sarwar, Jingru Cheng +8World Model Learning RL Post-Training
University of Illinois Urbana-Champaign · Amazon
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation
Aug 12, 2026Zhixin Zhang, Xinke Jiang, Zhibang Yang +5LLM-Guided RL LLM Agents
School of Computer Science, Peking University, Beijing, China · National Engineering Research Center for Software Engineering, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +2
Learning from Online User Feedback for Shopping Agents
Aug 12, 2026Haobo Zhang, Kelong Mao, Sulong Xu +2Large Language Model-Based Recommendation User Preference Modeling
Gaoling School of Artificial Intelligence, Renmin University of China
From Numbers to Judgment: Specialist LLM Agents and Reinforcement Learning for European Listed Real Estate
Aug 11, 2026Pardis Taghavi, Santosh BhavaniNumerical Reasoning in Language Models Multi-Agent LLM Systems
Santosh Bhavani∗
SKILLER: Language-Level Reinforcement Learning for Reusable Skill Extraction in Small Language Models
Aug 11, 2026Chenhao Dang, Siyuan Xiong, Conghui He +1RL for Language Models Small Language Models
1Shanghai Jiao Tong University · 2Shanghai Artificial Intelligence Laboratory · 3Harbin Institute of Technology, Shenzhen +1
TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling
Aug 11, 2026Yanyu Ren, Xizheng Wang, Xiao Liu +8Agentic RL Asynchronous RL
Tsinghua University · Zhongguancun Laboratory
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Aug 11, 2026Zelei Cheng, Amritansh Mishra, Sambit Sahu +1Efficient Attention Long-Horizon Agent Tasks
AI Foundations, Capital One
Bidirectional Context Self-Distillation for Reinforcement Learning of Skill-Based LLM Agents
Aug 10, 2026Tianjun Pan, Yuan Li, Hongda Wang +8LLM Agent Skill Learning Self-Distillation
Fudan University · Zhejiang University · Alibaba Group +1
MoRSE: Task-Oriented Multi-Agent System with Mixture of Role-Subtask Experts
Aug 10, 2026Peiwen Li, Shiyang Zhang, Yangtian Zhang +3Multi-Agent LLM Systems Mixture-of-Experts Language Models
Yale University, New Haven, CT, USA
Reading is not Reasoning: Bridging the Agentic Policy Gap in Vision-Text Compression
Aug 9, 2026Cheng Fan, Junyi Zhou, Tingzhang Luo +5Cross-Modal Knowledge Distillation Multimodal Token Compression
City University of Hong Kong · Huawei Technologies Ltd.
GraphThink: Graph-Enhanced LLM Thinking for Long-Horizon Embodied Task Planning
Aug 8, 2026Chen Li, Sijie Cheng, Yuelin Zhang +4Large Language Model-Based Robot Planning Robot Task Planning
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance, Beijing, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China +3

Sep 3, 2026, cs.LGJ/K move · Enter open · S save

Headroom-Drift Replay: A Primitive for Principled Replay Control in GRPO

Hyun Bin Park, Du-Seong Chang

Department of Artificial Intelligence, Sogang University, Seoul, Republic of Korea

RL for Language Model Reasoning Group Relative Policy Optimization Experience Replay RL Post-Training Policy Learning RL for LLM Agents

Open paper AskPDF arXiv

RL-based post-training for reasoning models is increasingly bottlenecked by repeated fresh rollout generation, particularly in agentic settings where environment interaction dominates wall-clock cost. Replay can reduce this burden by reusing past trajectories, but existing methods typically embed it within larger training pipelines involving exploration, experience restructuring, or mixed-policy optimization. This makes replay's own contribution difficult to isolate. We ask a focused question: how far can principled replay selection alone go? We introduce Headroom-Drift Replay, a group-level replay control primitive for GRPO that separates reuse into two decisions. Headroom ranks stored groups by remaining learning value, while Drift gates them by compatibility with the current policy. The fresh on-policy stream remains unchanged, and the method adds no auxiliary generation or training machinery. Across mathematical reasoning, multimodal reasoning, and Agentic Search benchmarks, this single intervention outperforms naive replay and matches or exceeds broader replay methods on Avg Mean@32. In Agentic Search, where environment interaction dominates cost, it delivers comparable quality at materially lower wall-clock time.

Momentum

Related topics

Latest papers 402