RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

70 papers in the last four weeks, up 141% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 402

All topics

From Imitation to Reward Discovery: On-Policy Warmup for Agentic RL
Sep 30, 2026Yitong Qiao, Tiantian He, Lei Liu +4Agentic RL Reinforcement Learning with Verifiable Rewards
Zhejiang University · Ant Healthcare, Ant Group
Mitigating the Length-Scaling Tax with Online Distillation
Sep 30, 2026Xu Wan, Wenyue Xu, Shengjie Zhao +1RL for Language Model Reasoning On-Policy Distillation
ByteDance Seed · Tongji University · Peking University
Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agents
Sep 30, 2026Huaiyu Fu, Heng Cao, Hao Wang +2LLM Agent Training RL Post-Training
Microsoft
EVOKE: Eliciting World Knowledge in Agents for Transferable Decision-Making
Sep 29, 2026Yuhan Guo, Jinming Liu, Liang Xu +8LLM Agent Training Generalization in RL
Shanghai Jiaotong University · Eastern Institute of Technology, Ningbo · Zhongguancun Academy, Beijing, China +1
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
Sep 29, 2026Rishabh Agrawal, Hejie Cui, Shasha Li +2Language Model Steering LLM Agent Self-Improvement
Google · University of Southern California
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Sep 29, 2026Haohao Qu, Yongcheng Jing, Chun Hin Chan +3Long-Context Modeling Recommender Systems
The Hong Kong Polytechnic University Hong Kong · Nanyang Technological University Singapore
Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents
Sep 29, 2026Qi Zhou, Yuanfan LiCredit Assignment in RL Long-Horizon LLM Agents
Xi’an Jiaotong University
VACE: Validation-Gated Alternating Co-Evolution of Agent Models and Harnesses
Sep 29, 2026Jiexing Qi, Yu He, Jun Liu +9Agentic RL Agent Harness Optimization
ICT AI Competence Center, Huawei Technologies Co., Ltd., Shanghai, China · Shanghai Jiao Tong University, Shanghai, China
Traverse: Learning When to Remember, Reset, and Redirect for Long-Horizon Web Search
Sep 29, 2026Jingyuan Ma, Lynx Aster, He Zhang +5Long-Horizon Agent Evaluation Web Search Agents
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · ByteDance
EASE: Behavior-Adaptive Skill Curation for Self-Evolving Agents
Sep 29, 2026Zhen Xiong, Qiaoyu TanLLM Agent Skill Learning LLM Agent Skill Retrieval
New York University · New York University Shanghai
BRIDGE: Bilevel Retrieval-Credit-Aware Agentic Reinforcement Learning
Sep 29, 2026Quan Xiao, Mingda Liu, Gaowen Liu +2Reinforcement Learning Agentic RAG
Department of Electrical and Computer Engineering, Cornell University · Institute of Integrated Research, Institute of Science Tokyo · Cisco Research
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Sep 28, 2026Zhiwei Zhang, Huayu Deng, Fei Zhao +4Supervised Fine-Tuning Self-Training for Language Models
AllSpark Team
Harness Learning Enables Generalizable Test-Time Adaptation
Sep 28, 2026Alvin Zhang, Xuecheng Liu, Zixuan Wang +6Test-Time Adaptation Meta-Learning
Reinforcing Agentic Creativity in Scientific Ideation with Night Science
Sep 28, 2026Priyanka Kargupta, Silviu Cucerzan, Shweti Mahajan +4AI Agents for Scientific Discovery Scientific Hypothesis Generation
University of Illinois Urbana-Champaign · Microsoft · Microsoft Research
Continuous Context Management
Sep 28, 2026William Hoy, Jingxuan Fan, Nurcin Celik +1Context Distillation Long-Horizon LLM Agents
University of Miami · Harvard University
ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
Sep 28, 2026Yang Li, Jinhan Yang, hai liu +8Agentic RL Credit Assignment in RL
University of Hong Kong · The Chinese University of Hong Kong · Jiangxi Science and Technology Normal University +1
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Sep 28, 2026Haodong Zhu, Yangyang Ren, Changbai Li +4Credit Assignment in RL Step-Level Credit Assignment in RL
Beihang University · Zhongguancun Academy · Communication University of China +1
Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
Sep 28, 2026Yangyang Ren, Haodong Zhu, Linlin Yang +3Agentic RL Step-Level Credit Assignment in RL
Beihang University · Zhongguancun Academy · Communication University of China +2
TIDE: Teacher-Student Transition via Informative Distillation and Exploration for Agentic RL
Sep 28, 2026Yibin Huang, Xinming Xu, Conghui ZhuAgentic RL On-Policy Distillation
Faculty of Computing, Harbin Institute of Technology · Tsinghua University
Proactive Dialogue Policy Optimization via Cognitive-State Transition
Sep 28, 2026Minghui Ma, Mengqi Chen, Bin Guo +1Policy Optimization Conversational Agents
Northwestern Polytechnical University
UniOPSD: Unifying Outcome and Hindsight Feedback for Agentic Reinforcement Learning
Sep 28, 2026Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai +8Agentic RL Credit Assignment in RL
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Peking University +3
SIPO: Selective-Inference Policy Optimization for Tree-Structured Agentic RL
Sep 28, 2026Zenghuang Fu, Ningqi Chen, Mingda Jia +10Agentic RL Reinforcement Learning
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · The University of Hong Kong +4
Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learning
Sep 28, 2026Lirui Luo, Kelong Mao, Heming Xia +10LLM Agent Memory Persistent Memory for Language Models
Marathoner: Ultra-Long-Horizon Autonomous Intelligence
Sep 28, 2026Zhang Ruiyang, Ou Jinpeng, Xie Yifan +4Long-Horizon Agent Tasks Long-Horizon LLM Agents
FIC, University of Macau · Ant Group · Peking University
GlyphBench: A Playground for Language-Model Reinforcement Learning
Sep 28, 2026Roger Creus Castanyer, Marc-Alexandre Côté, Matthew James Sargent +3RL Benchmarks RL for Language Models
Mila - Quebec Artificial Intelligence Institute · Université de Montréal · Vmax
Learning Perturbation Robust Policies for LLM Agents with Stable Optimization
Sep 28, 2026Pengxin Wang, Yuanzhe LI, Yuxin Ren +2RL Post-Training Robust RL
Department of Electrical and Computer Engineering University of Arizona
QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
Sep 27, 2026Weiqi Wang, Yuxin Zhou, Mouxiang Chen +9Reinforcement Learning Long-Horizon Agent Tasks
Alibaba Token Hub, Alibaba Group · University of Science and Technology of China · Tsinghua University
CompoWorld: Compositional Environment Scaling for General Agents
Sep 27, 2026Xiao-Wen Yang, Weiyi Xu, Wen Da +9Tool-Augmented Language Model Agents Tool-Using Agents
AllSpark Team
Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learning
Sep 27, 2026Chenlong Yin, Xiaolong Jin, Wei Zou +2Adversarial Attacks on LLMs LLM Red Teaming
The Pennsylvania State University · Purdue University
Beyond Timestamps: Decision-Aligned On-Policy Distillation for Long-Horizon Agents
Sep 27, 2026Mingju Chen, Can Lv, Jinrong Liu +3Credit Assignment in RL Reinforcement Learning with Verifiable Rewards
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University · Tsinghua University

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

From Imitation to Reward Discovery: On-Policy Warmup for Agentic RL

Yitong Qiao, Tiantian He, Lei Liu, Yue Shen+3

Zhejiang University · Ant Healthcare, Ant Group

Agentic RL Reinforcement Learning with Verifiable Rewards On-Policy Distillation Sparse-Reward RL Policy Learning RL for LLM Agents

Open paper AskPDF arXiv

Reinforcement learning with a verifiable reward (RLVR) offers a scalable approach to training language-model agents, yet sparse outcome rewards can leave early training with little signal for policy improvement. We identify an On-Policy Acceleration Phenomenon: in our main comparisons, RLVR initialized with on-policy distillation reaches high performance earlier in training and achieves both higher average performance during subsequent RLVR and higher final performance than the alternative baselines. Motivated by this observation, we study On-Policy Warmup (OPW), a teacher-guided stage in which the student trains with teacher supervision on its own interaction trajectories before transitioning to RLVR. Unlike imitation on fixed teacher-generated trajectories, OPW targets states induced by the student's own decisions, including imperfect actions and recovery situations. We provide a theoretical explanation by connecting on-policy reverse-KL distillation to trajectory-level distribution matching. Under a competent teacher and sufficiently small population distillation loss, this connection yields a lower bound on initial verifier success and a corresponding bound on reward-discovery complexity. For group-relative RLVR, we further characterize when increased success probability produces more reward-informative groups. Together, our findings support on-policy distillation as an effective warmup for agentic RLVR and identify initial reward discovery as a mechanism that can contribute to the observed acceleration.

Momentum

Related topics

Latest papers 402