RL for LLM Agents

RL: Reinforcement Learning · LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

70 papers in the last four weeks, up 141% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 402

All topics

ECHO: Terminal Agents Learn World Models for Free
May 23, 2026Vaishnavi Shrivastava, Piero Kauffmann, Ahmed Awadallah +1World Model Learning Computer-Use Agents
Microsoft Research
When Does Multi-Agent RL Improve LLM Workflows? Workflow, Scale, and Policy-Sharing Tradeoffs
May 22, 2026Yifan Zeng, Yiran Wu, Yaolun Zhang +4Multi-Agent LLM Systems LLM Agent Workflow Optimization
1Oregon State University · 2Pennsylvania State University · 3Adobe Inc. +1
Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents
May 22, 2026Jiazheng Kang, Bowen Zhang, Zixin Song +4Deep Research Agents Agentic Reasoning
Qwen Applications Business Group of Alibaba · Tsinghua University
From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning
May 22, 2026Ranxu zhang, zeyang li, Jiacheng Huang +5Agentic RL Reinforcement Learning
University of Science and Technology of China · Alibaba Group
What Training Data Teaches RL Memory Agents: An Empirical Study of Curriculum Effects in Memory-Augmented QA
May 21, 2026Xinjie He, Zhiyuan Lin, Su Liu +4Memory-Augmented Neural Networks Curriculum RL
Columbia University · Independent Researcher · Johns Hopkins University +1
Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning
May 21, 2026Banghao Chi, Yining Xie, Mingyuan Wu +9Tool-Using Agents LLM Agent Training
University of Illinois Urbana-Champaign · Meta
DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Agent Memory
May 21, 2026Jianing Yin, Tan Tang, Yingcai WuLLM Agent Memory Persistent Memory for Language Models
State Key Lab of CAD&CG, Zhejiang University
Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles
May 21, 2026Jinyang Wu, Guocheng Zhai, Ruihan Jin +7LLM Agent Orchestration Language Model Ensembles
1Tsinghua University · 2Zhejiang University · 3The Chinese University of Hong Kong +2
Mem- $π$ : Adaptive Memory through Learning When and What to Generate
May 20, 2026Xiaoqiang Wang, Chao Wang, Hadi Nekoei +5LLM Agent Memory RL for LLM Agents
1ServiceNow AI Research · 2Mila – Quebec AI Institute · 3Université de Montréal +3
Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents
May 20, 2026Chongrui Ye, Yuxiang Liu, Yu Wang +5Continual Learning for LLM Agents Continual Learning
University of Illinois Urbana-Champaign · University of California San Diego
Training Language Agents to Learn from Experience
May 19, 2026Yuval Shalev, Zifeng Ding, Mateja JamnikContinual Learning for LLM Agents Self-Improving Agents
University of Cambridge
Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents
May 19, 2026Wenjie Tang, Minne Li, Sijie Huang +2Reinforcement Learning Partially Observable RL
College of Computer, National University of Defense Technology · 2Intelligent Game and Decision Lab (IGDL) · Institute of Artificial Intelligence, Xiamen University
Memory-Augmented Reinforcement Learning Agent for CAD Generation
May 19, 2026Yin Xiaolong, Liu Yu, Shen Jiahang +4Text-to-CAD Generation RL for Generative Models
Zhejiang University · Ocean University of China
What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
May 19, 2026Xiaozhe Li, Tianyi Lyu, Yang Li +6Reinforcement Learning Credit Assignment in RL
1Tongji University · 2Shanghai AI Laboratory · 5Independent +2
Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints
May 18, 2026Jiayu Li, Enpei Zhang, Dawei Zhou +2Sequential MARL Sequential Decision Making
Stern School of Business · New York University · Department of Computer Science +2
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
May 18, 2026Wonjoong Kim, Yeonjun In, Sangwu Park +2Reward Modeling Credit Assignment in RL
KAIST · Yonsei University
GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents
May 18, 2026Xiongbin Wu, Zhihao Luo, Shanzhe Lei +7Reinforcement Learning Group Relative Policy Optimization
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · East China Normal University +2
HydroAgent: Closing the Gap Between Frontier LLMs and Human Experts in Hydrologic Model Calibration via Simulator-Grounded RL
May 18, 2026Zhi Li, Songkun Yan, Jie Cao +4Reinforcement Learning Model Calibration
Civil, Environmental, and Architectural Engineering, University of Colorado Boulder · Civil Engineering and Environmental Sciences, University of Oklahoma · Department of Computer Science, University of Oklahoma +3
Agent Bazaar: Enabling Economic Alignment in Multi-Agent Marketplaces
May 17, 2026Seth Karten, Cameron Crow, Chi JinAI Agent Benchmarks Multi-Agent Systems
Princeton University
Look Before You Leap: Autonomous Exploration for LLM Agents
May 15, 2026Ziang Ye, Wentao Shi, Yuxin Liu +6RL Exploration LLM Agents
University of Science and Technology of China · Meituan
ALSO: Adversarial Online Strategy Optimization for Social Agents
May 15, 2026Xiang Li, Liping Yi, Mingze Kong +3Non-Stationary RL Large Language Model-Based Social Simulation
School of Artificial Intelligence, Tianjin University, Tianjin, China · The Chinese University of Hong Kong, Shenzhen, China · East China Normal University, Shanghai, China.
Response-Conditioned Parallel-to-Sequential Orchestration for Multi-Agent Systems
May 15, 2026Nurbek Tastan, Alex Iacob, Lorenzo Sani +4Multi-Agent System Optimization Multi-Agent Orchestration
1MBZUAI, UAE · University of Cambridge, UK · 3Flower Labs, UK +1
Self-Distilled Agentic Reinforcement Learning
May 14, 2026Zhengxi Lu, Zhiyuan Yao, Zhuowen Han +8Agentic RL Self-Distillation
Zhejiang University · Meituan · Tsinghua University
COLLATOR: Compositional Multi-Agent Orchestration with Counterfactual Reinforcement Learning
May 14, 2026Xudong Chen, Yixin Liu, Hua Wei +1Multi-Agent System Optimization Multi-Agent Orchestration
SkillFlow: Flow-Driven Recursive Skill Evolution for Agentic Orchestration
May 13, 2026Mingda Zhang, Tiesunlong Shen, Haoran Luo +4LLM Agent Orchestration LLM Agent Self-Improvement
The Chinese University of Hong Kong, Shenzhen · National University of Singapore · Nanyang Technological University +1
Scaling Retrieval-Augmented Reasoning with Parallel Search and Explicit Merging
May 13, 2026Jiabei Liu, Wenyu Mao, Junfei Tan +4Multi-Hop QA Agentic RAG
University of Science and Technology of China · WeChat Technical Architecture Department, Tecent Inc.
GAGPO: Generalized Advantage Grouped Policy Optimization
May 13, 2026Siyuan Zhu, Chao Yu, Rongxin Yang +4Reinforcement Learning Group Relative Policy Optimization
School of Computer Science and Engineering, Sun Yat-sen University · Meituan
ICRL: Learning to Internalize Self-Critique with Reinforcement Learning
May 13, 2026Jianbo Lin, Xiaomin Yu, Yi Xin +7RL for Language Models LLM Self-Refinement
Hong Kong University of Science and Technology (Guangzhou) · Nanjing University · Sun Yat-sen University +3
Reinforced Collaboration in Multi-Agent Flow Networks
May 13, 2026Zheng Wang, Yuang Liu, Yangkai DingMulti-Agent LLM Systems Multi-Agent Collaboration
Huawei Technologies Co., Ltd.
Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
May 12, 2026Zhong Guan, Yongjian Guo, Haoran Sun +5Asynchronous RL Reinforcement Learning
Tianjin University · Tsinghua University · JDT AI Infra +1

May 23, 2026, cs.LGJ/K move · Enter open · S save

ECHO: Terminal Agents Learn World Models for Free

Vaishnavi Shrivastava, Piero Kauffmann, Ahmed Awadallah, Dimitris Papailiopoulos

Microsoft Research

World Model Learning Computer-Use Agents Terminal Agents LLM Agent Training Policy Learning RL for LLM Agents

Open paper AskPDF arXiv

CLI agents are the closest thing language models have to an embodied setting: the model emits commands, the terminal executes them, and the returned stream -- stdout, errors, files, logs, and traces -- records the consequences. We argue that this stream is a supervision signal, but standard agent RL discards it: GRPO-style training updates action tokens with sparse outcome-level rewards while ignoring environment responses already in the rollout. Failed rollouts provide little policy-gradient signal despite containing rich evidence about how the environment responds. We introduce ECHO (Environment Cross-entropy Hybrid Objective), a hybrid objective that combines the standard policy-gradient loss on action tokens with an auxiliary loss that trains the policy to predict environment observation tokens resulting from its own actions. ECHO reuses the same forward pass as GRPO, requires no additional rollouts, and turns terminal feedback into dense supervision for all rollouts. ECHO doubles GRPO pass@1 on TerminalBench-2.0: Qwen3-8B improves from 2.70% to 5.17%, and Qwen3-14B from 5.17% to 10.79%. ECHO also produces policies that better predict terminal dynamics, even on trajectories they did not generate: across held-out rollouts, it sharply reduces environment-token cross-entropy while GRPO alone barely changes it. From base Qwen3-8B, ECHO matches expert-SFT-then-GRPO performance on held-out terminal tasks without expert demonstrations, and recovers roughly half of the expert-SFT initialization benefit on TerminalBench-2.0. In some settings, the environment prediction loss alone enables verifier-free self-improvement, allowing policies to improve on unseen OOD tasks by learning only from environment interactions. Together, these results suggest that environment observations are not merely context for future actions, but a dense, on-policy supervision signal already present in every rollout.

Momentum

Related topics

Latest papers 402