Average Reward Markov Decision Process

Average Reward Markov Decision Processes (AMDPs) focus on finding optimal policies that maximize the long-run average reward in sequential decision-making problems. Current research emphasizes developing efficient algorithms, both model-based and model-free, with improved regret bounds and sample complexities, often leveraging techniques like policy gradient methods, value iteration, and function approximation within various model architectures (e.g., linear, kernel). These advancements are significant for improving the theoretical understanding and practical applicability of reinforcement learning in diverse fields, including robotics, control systems, and resource management, where long-term average performance is crucial.

Papers

October 14, 2024

Burning RED: Unlocking Subtask-Driven Reinforcement Learning and Risk-Awareness in Average-Reward Markov Decision Processes
Juan Sebastian Rojas, Chi-Guhn Lee
Reinforcement Learning Sequential Decision Unlocking Insight Average Reward Markov Decision Process Risk Awareness

October 10, 2024

The Plug-in Approach for Average-Reward and Discounted MDPs: Optimal Sample Complexity Analysis
Matthew Zurek, Yudong Chen
Markov Decision Process Sample Complexity Optimal Policy Low Rank Average Reward Markov Decision Process

July 9, 2024

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes
Guillermo Infante, Anders Jonsson, Vicenç Gómez
Hierarchical Reinforcement Learning Average Reward Markov Decision Process Average Reward Reinforcement Learning High Level Task

June 17, 2024

Constrained Reinforcement Learning with Average Reward Objective: Model-Based and Model-Free Algorithms
Vaneet Aggarwal, Washim Uddin Mondal, Qinbo Bai
Reinforcement Learning Primal Dual Constrained Reinforcement Learning Constrained Markov Decision Process Average Reward Markov Decision Process Model Free Algorithm Average Reward

June 3, 2024

Achieving Tractable Minimax Optimal Regret in Average Reward MDPs
Victor Boone, Zihan Zhang
Policy Gradient Regret Bound Regret Guarantee Average Reward Markov Decision Process Minimax Optimal Regret

May 27, 2024

Finding good policies in average-reward Markov Decision Processes without prior knowledge
Adrienne Tuynman, Rémy Degenne, Emilie Kaufmann
Sample Complexity Optimal Policy Prior Knowledge Average Reward Markov Decision Process High Performing Policy Policy Model Policy Identification

April 19, 2024

Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximation
Jianliang He, Han Zhong, Zhuoran Yang
Sample Efficient Average Reward Markov Decision Process Average Reward Optimal State

March 18, 2024

Span-Based Optimal Sample Complexity for Weakly Communicating and General Average Reward MDPs
Matthew Zurek, Yudong Chen
Sample Complexity Optimal Policy Average Reward Markov Decision Process Average Reward MDPs

March 11, 2024

On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes
Navdeep Kumar, Yashaswini Murthy, Itai Shufaro, Kfir Y. Levy, R. Srikant, Shie Mannor
Policy Gradient Global Convergence Tabular Markov Decision Process Average Reward Markov Decision Process Hybrid Reward Average Reward Reinforcement Learning

February 3, 2024

Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithm
Qinbo Bai, Washim Uddin Mondal, Vaneet Aggarwal
Optimal Policy Primal Dual Constrained Markov Decision Process Regret Guarantee Simple Regret Policy Gradient Algorithm Average Reward Markov Decision Process Policy Parametrization

November 22, 2023

Span-Based Optimal Sample Complexity for Average Reward MDPs
Matthew Zurek, Yudong Chen
Sample Complexity Optimal Policy Average Reward Markov Decision Process Average Reward MDPs

October 18, 2023

Quantum Speedups in Regret Analysis of Infinite Horizon Average-Reward Markov Decision Processes
Bhargav Ganguly, Yang Xu, Vaneet Aggarwal
Regret Analysis Quantum Advantage Infinite Horizon Average Reward Markov Decision Process Quantum Speedup Tabular Reinforcement Learning

October 13, 2023

Optimal Sample Complexity for Average Reward Markov Decision Processes
Shengbo Wang, Jose Blanchet, Peter Glynn
Markov Decision Process Sample Complexity Optimal Policy Policy Learning Average Reward Markov Decision Process Average Reward MDPs

September 5, 2023

Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes
Qinbo Bai, Washim Uddin Mondal, Vaneet Aggarwal
Regret Analysis O$ Regret Policy Gradient Algorithm Average Reward Markov Decision Process Linear Markov Decision Process

June 28, 2023

Sharper Model-free Reinforcement Learning for Average-reward Markov Decision Processes
Zihan Zhang, Qiaomin Xie
Optimal Policy Model Free Model Free Reinforcement Learning Average Reward Markov Decision Process

May 24, 2023

Inverse Reinforcement Learning with the Average Reward Criterion
Feiyang Wu, Jingyang Ke, Anqi Wu
Inverse Reinforcement Learning Policy Mirror Descent Average Reward Markov Decision Process Average Reward

January 2, 2023

Robust Average-Reward Markov Decision Processes
Yue Wang, Alvaro Velasquez, George Atia, Ashley Prater-Bennette, Shaofeng Zou
Average Reward Markov Decision Process Robust Reward Optimal Robust Policy

December 1, 2022

Near Sample-Optimal Reduction-based Policy Learning for Average Reward MDP
Jinghan Wang, Mengdi Wang, Lin F. Yang
Markov Decision Process Ground Truth Optimal Policy Deterministic Policy Average Reward Markov Decision Process Sample Efficient Policy Average Reward MDPs

June 7, 2022

Concentration bounds for SSP Q-learning for average cost MDPs
Shaan Ul Haque, Vivek Borkar
Q Learning Shortest Path Value Iteration Selective Focus Q$ Learning Average Reward Markov Decision Process

May 23, 2022

Logarithmic regret bounds for continuous-time average-reward Markov decision processes
Xuefeng Gao, Xun Yu Zhou
Reinforcement Learning Markov Decision Process Logarithmic Regret Instance Dependent Regret Average Reward Markov Decision Process Finite Time Regret Continuous Time Markov Decision Process

Average Reward Markov Decision Process

Papers

Burning RED: Unlocking Subtask-Driven Reinforcement Learning and Risk-Awareness in Average-Reward Markov Decision Processes

The Plug-in Approach for Average-Reward and Discounted MDPs: Optimal Sample Complexity Analysis

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes

Constrained Reinforcement Learning with Average Reward Objective: Model-Based and Model-Free Algorithms

Achieving Tractable Minimax Optimal Regret in Average Reward MDPs

Finding good policies in average-reward Markov Decision Processes without prior knowledge

Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximation

Span-Based Optimal Sample Complexity for Weakly Communicating and General Average Reward MDPs

On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes

Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithm

Span-Based Optimal Sample Complexity for Average Reward MDPs

Quantum Speedups in Regret Analysis of Infinite Horizon Average-Reward Markov Decision Processes

Optimal Sample Complexity for Average Reward Markov Decision Processes

Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes

Sharper Model-free Reinforcement Learning for Average-reward Markov Decision Processes

Inverse Reinforcement Learning with the Average Reward Criterion

Robust Average-Reward Markov Decision Processes

Near Sample-Optimal Reduction-based Policy Learning for Average Reward MDP

Concentration bounds for SSP Q-learning for average cost MDPs

Logarithmic regret bounds for continuous-time average-reward Markov decision processes