Average-Reward RL

RL: Reinforcement Learning

Ask about this topic

Overview

Average-reward RL optimizes long-run reward per time step in continuing tasks, rather than discounting distant rewards; it is useful when there is no natural episode end or discount factor. Methods include differential temporal-difference and Q-learning, relative-value iteration, actor-critic, and policy-gradient or mirror-descent algorithms, with tabular, linear, and neural critics and often a single Markovian trajectory. Current work develops convergence, regret, and sample-complexity guarantees for communicating and weakly communicating MDPs, while extending learning to constrained and robust problems, options, and multi-agent or mean-field settings. A central tension is that guarantees often depend on mixing, span, or ergodicity assumptions, while optimizing asymptotic reward can still permit poor transient performance or obscure different recurrent-class rewards in multichain systems.

Latest papers

CardsList

35 papers9 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

  1. Almost Sure Convergence of Differential Temporal Difference Learning for Average Reward Markov Decision Processes

    Feb 18, 2026Ethan Blaser, Jiuqi Wang, Shangtong ZhangTemporal-Difference LearningAverage-Reward RL

    University of Virginia

  2. Reinforcement Learning in MDPs with Information-Ordered Policies

    Aug 5, 2025Zhongjun Zhang, Shipra Agrawal, Ilan Lobel +2Queueing TheoryPolicy Learning

    Department of Industrial Engineering and Management Sciences, Northwestern University · Department of Industrial Engineering and Operations Research, Columbia University · Technology, Operations, and Statistics, Stern School of Business, New York University +1

  3. Distributionally Robust Markov Games with Average Reward

    Aug 5, 2025Zachary Roch, Yue WangNash Equilibrium ComputationMulti-Agent System Optimization

    Department of Electrical and Computer Engineering, University of Central Florida, Orlando, Florida, USA. · Department of Computer Science, University of Central Florida, Orlando, Florida, USA.

  4. Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

    Jun 8, 2025Yang Xu, Swetha Ganesh, Vaneet AggarwalRobust Markov Decision ProcessesReinforcement Learning

    Purdue University, West Lafayette, Indiana, USA 47907

  5. Model-Free Robust Average-Reward Reinforcement Learning with Sample Complexity Analysis

    May 18, 2025Zachary Roch, George Atia, Yue WangReinforcement LearningAverage-Reward RL

    Department of Electrical and Computer Engineering, University of Central Florida, Orlando, Florida, USA. · Department of Computer Science, University of Central Florida, Orlando, Florida, USA.