Reinforcement Learning

Also known as RL

Reinforcement learning studies how an agent can choose actions over time to earn rewards, using experience to improve its decisions. Common approaches include value-based methods such as Q-learning, policy-gradient methods such as PPO, and model-based methods that learn or use a model of the environment; offline and hierarchical methods address learning from fixed data and long tasks. Applications range from Atari and MuJoCo benchmarks to robot control, traffic, games, and language-model agents that use tools or solve multi-step problems. A central challenge is making learning efficient and reliable when rewards are sparse or imperfect, environments change, and success in simulation or benchmarks may not transfer safely to deployment.

Latest papers

1,767 papers195 papers in the last four weeks, down 8% on the four weeks before. 1.3% of all new papers.

ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network
May 10, 2026Qian Chen, Junqiao Zhao, Hongtu Zhou +4Reinforcement Learning Transformer-Based RL
Tongji University
From Passive Reuse to Active Reasoning: Grounding Large Language Models for Neuro-Symbolic Experience Replay
May 10, 2026Yanan Xiao, Yixiang Tang, Zechen Feng +3Reinforcement Learning Experience Replay
1Affiliation 1 · 2Affiliation 2 · 3Affiliation 3
Functional Graphs for Predicting and Explaining Goal Failure in Sparse Goal-Conditioned RL
May 10, 2026Shalley DashReinforcement Learning Sparse-Reward RL
Associate Adjunct Professor, Business Analytics, Institute of Management Technology Ghaziabad, 201001
PiCA: Pivot-Based Credit Assignment for Search Agentic Reinforcement Learning
May 10, 2026Dongyi Liu, Yifan Niu, Qinwen Wang +2Reinforcement Learning Reward Shaping
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
Learning the Preferences of a Learning Agent
May 9, 2026Karim Abdel Sadek, Mark Bedaywi, Rhys Gould +1Reinforcement Learning Preference Learning
University of California, Berkeley
DARE: Difficulty-Adaptive Reinforcement Learning with Co-Evolved Difficulty Estimation
May 9, 2026Yang Zhou, Can Jin, Zihan Dong +7Reinforcement Learning RL for Language Model Reasoning
Rutgers University · Amazon · Washington University +1
Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs
May 9, 2026Xiaozhe Li, Xinyu Fang, Shengyuan Ding +5RL Benchmarks Reinforcement Learning
Tongji University1 · Shanghai AI Lab2 · Zhejiang University3 +3
BubbleSpec: Turning Long-Tail Bubbles into Speculative Rollout Drafts for Synchronous Reinforcement Learning
May 9, 2026Yuhang Xu, Kaibin Tian, Yang Tian +6Reinforcement Learning Speculative Decoding
1Shanghai Jiao Tong University · 2Bytedance
Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents
May 9, 2026Minzheng Wang, Run Luo, Yanbo Wang +6Reinforcement Learning Self-Play RL
MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · National University of Singapore +1
The Cancellation Hypothesis in Critic-Free RL: From Outcome Rewards to Token Credits
May 9, 2026Tianhao Cheng, Zeyu Huang, Zihan Qiu +5Reinforcement Learning Token-Level Credit Assignment
1Fudan University · 3The University of Edinburgh · 6Qwen Team, Alibaba Group +4
MARLaaS: Multi-Tenant Asynchronous Reinforcement Learning as a Service
May 8, 2026Timothy Tin Long Yu, Gursimran Singh, Ge Shi +3Asynchronous RL Reinforcement Learning
Huawei Technologies Canada
DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards
May 8, 2026Haoyu Hu, Xuandong Zhao, Xuhai "Orson'' Xu +1Reinforcement Learning Group Relative Policy Optimization
Cornell University · University of California, Berkeley · Columbia University
Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning
May 8, 2026Manish Bhattarai, Ismael Boureima, Nishath Rajiv Ranasinghe +2Reinforcement Learning RL for Language Model Reasoning
Los Alamos National Laboratory, Los Alamos, NM, USA.
Reinforcement Learning for Exponential Utility: Algorithms and Convergence in Discounted MDPs
May 8, 2026Gugan Thoppe, L. A. Prashanth, Ankur Naskar +1Markov Decision Processes Reinforcement Learning
Computer Science and Automation, Indian Institute of Science, Bengaluru · Computer Science and Engineering, Indian Institute of Technology Madras, Chennai · Tata Consultancy Services Limited, Hyderabad
Interpreting Reinforcement Learning Agents with Susceptibilities
May 8, 2026Chris Elliott, Einar Urdshals, David Quarel +1Reinforcement Learning Neural Network Interpretability
Timaeus
Actor-Critic Algorithm for Dynamic Expectile and CVaR
May 8, 2026Yudong Luo, Erick DelageReinforcement Learning Risk-Sensitive RL
1GERAD & Department of Decision Sciences, HEC Montr´eal, Canada · 2Mila - Quebec AI Institute, Canada
Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Works
May 8, 2026Wenhua Nie, Jianan Wu, Junlin Liu +6Reinforcement Learning Group Relative Policy Optimization
National Taiwan University
SEIF: Self-Evolving Reinforcement Learning for Instruction Following
May 8, 2026Qingyu Ren, Qianyu He, Jiajie Zhu +7RL for Language Models Reinforcement Learning
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University · School of Data Science, Fudan University · Ant Group
RELO: Reinforcement Learning to Localize for Visual Object Tracking
May 8, 2026Xin Chen, Chuanyu Sun, Jiao Xu +4Reinforcement Learning Visual Object Tracking
City University of Hong Kong · Dalian University of Technology · Hunyuan Team, Tencent
Beyond Linear Attention: Softmax Transformers Implement In-Context Reinforcement Learning
May 8, 2026Zixuan Xie, Xinyu Liu, Claire Chen +3Reinforcement Learning Transformer-Based RL
University of Virginia · California Institute of Technology · Purdue University
SparseRL-Sync: Lossless Weight Synchronization with ~100x Less Communication
May 8, 2026Lucas Hu, Ranchi Zhao, Isaac Zhu +4Reinforcement Learning Communication-Efficient Distributed Training
Scitix
Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
May 8, 2026Xiaoze Liu, Dhananjay Ram, Yuting Zhang +3RL for Language Models Reinforcement Learning
1Purdue University · *Work done during an internship at Amazon. · 2AWS Agentic AI
Improved Model-based Reinforcement Learning with Smooth Kernels
May 8, 2026Kun Long, Yuqiang Li, Xianyi WuReinforcement Learning Continuous-Time RL
School of Statistics, KLATASDS-MOE, East China Normal University, North Zhongshan Road, Shanghai, 200062, PR China.
Adaptive Negative Reinforcement for LLM Reasoning:Dynamically Balancing Correction and Diversity in RLVR
May 8, 2026Yash Ingle, Jaival Chauhan, Ankit Yadav +1Reinforcement Learning RL for Language Model Reasoning
Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat, India
Convergence and Emergence of In-Context Reinforcement Learning with Chain of Thought
May 8, 2026Zixuan Xie, Xinyu Liu, Rohan Chandra +1Reinforcement Learning Transformer
University of Virginia
Almost Sure Convergence Rates of Stochastic Approximation and Reinforcement Learning via a Poisson-Moreau Drift
May 8, 2026Xinyu Liu, Zixuan Xie, Shangtong ZhangReinforcement Learning Stochastic Approximation
University of Virginia
Actor-Critic with Active Importance Sampling
May 8, 2026Majid Molaei, Gabor Paczolay, Matteo Papini +2Reinforcement Learning Importance Sampling for RL
Politecnico di Milano Milan, Italy
From Causal Graphs to Markovian States: Multi-Order State Exposure for Deep RL
May 8, 2026Jiamin Xu, Jacqueline Maasch, Kyra GanMarkov Decision Processes Causal Representation Learning
Cornell Tech, New York, NY
Towards Differentially Private Reinforcement Learning with General Function Approximation
May 7, 2026Yi He, Xingyu ZhouRegret Minimization in RL Reinforcement Learning
Wayne State University, Detroit, USA
PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
May 7, 2026Minghao Yan, Bo Peng, Benjamin Coleman +11Test-Time Adaptation Reinforcement Learning
Google · University of Wisconsin–Madison · Google DeepMind

May 10, 2026J/K move · Enter open · S save

ACSAC: Adaptive Chunk Size Actor-Critic with Causal Transformer Q-Network

Qian Chen, Junqiao Zhao, Hongtu Zhou, Hang Yu+3

Tongji University

Reinforcement Learning Transformer-Based RL Offline RL Sparse-Reward RL Action Chunking Actor-Critic Methods

Open paper AskPDF arXiv

Long-horizon, sparse-reward tasks pose a fundamental challenge for reinforcement learning, since single-step TD learning suffers from bootstrapping error accumulation across successive Bellman updates. Actor-critic methods with action chunking address this by operating over temporally extended actions, which reduce the effective horizon, enable fast value backups, and support temporally consistent exploration. However, existing methods rely on a fixed chunk size and therefore cannot adaptively balance reactivity against temporal consistency. A large fixed chunk size reduces responsiveness to new observations, while a small one produces incoherent motions, forcing task-specific tuning of the chunk size. To address this limitation, we propose Adaptive Chunk Size Actor-Critic (ACSAC). ACSAC leverages a causal Transformer critic to evaluate expected returns for action chunks of different sizes. At each chunk boundary, it adaptively selects the chunk size that maximizes the expected return, supporting flexible, state-dependent chunk sizes without task-specific tuning. We prove that the ACSAC Bellman operator is a contraction whose unique fixed point is the action-value function of the adaptive policy. Experiments on OGBench demonstrate that ACSAC achieves state-of-the-art performance on long-horizon, sparse-reward manipulation tasks across both offline RL and offline-to-online RL settings.