Episodic Markov Decision Process

Episodic Markov Decision Processes (EMDPs) model sequential decision-making problems where interactions conclude after a fixed number of steps, focusing on learning optimal policies to maximize cumulative rewards. Current research emphasizes developing provably efficient algorithms, particularly for model-free approaches and settings with function approximation, often employing techniques like upper confidence bounds, posterior sampling, and reference-advantage decomposition to handle stochasticity and improve sample efficiency. These advancements are significant for both theoretical understanding of reinforcement learning and practical applications, enabling faster and more robust learning in complex environments with limited data.

Papers

May 10, 2023

Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation
Yifei Min, Jiafan He, Tianhao Wang, Quanquan Gu
Multi Agent Reinforcement Learning Multiple Agent Cooperative Multi Agent Reinforcement Learning Linear Function Approximation Communication Complexity Episodic Markov Decision Process Asynchronous Communication

April 27, 2023

A Best-of-Both-Worlds Algorithm for Constrained MDPs with Long-Term Constraints
Jacopo Germano, Francesco Emanuele Stradi, Gianmarco Genalti, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti
Online Learning Constrained Markov Decision Process Episodic Markov Decision Process Best of Both World Algorithm Long Term Constraint

March 31, 2023

Online Reinforcement Learning in Markov Decision Process Using Linear Programming
Vincent Leon, S. Rasoul Etesami
Markov Decision Process Online Reinforcement Learning Linear Programming Episodic Markov Decision Process Stochastic Reward Regret Rate

February 8, 2023

A Near-Optimal Algorithm for Safe Reinforcement Learning Under Instantaneous Hard Constraints
Ming Shi, Yingbin Liang, Ness Shroff
Reinforcement Learning Safe Reinforcement Learning Regret Analysis Episodic Markov Decision Process Near Optimal Algorithm

December 27, 2022

Model-Based Reinforcement Learning with Multinomial Logistic Function Approximation
Taehyun Hwang, Min-hwan Oh
Model Based Reinforcement Learning Efficient Reinforcement Learning Episodic Markov Decision Process Multinomial Logistic Linear Markov Decision Process

October 24, 2022

Opportunistic Episodic Reinforcement Learning
Xiaoxiao Wang, Nader Bouacida, Xueying Guo, Xin Liu
Reinforcement Learning Episodic Markov Decision Process Episodic Reinforcement Learning Exploration Exploitation Trade Opportunistic Routing

October 3, 2022

Square-root regret bounds for continuous-time episodic Markov decision processes
Xuefeng Gao, Xun Yu Zhou
Reinforcement Learning Markov Decision Process Episodic Markov Decision Process Episodic Reinforcement Learning Continuous Time Markov Decision Process Continuous MDPs

September 21, 2022

Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Models
Rui Miao, Zhengling Qi, Xiaoke Zhang
Markov Decision Process Policy Evaluation Episodic Markov Decision Process Nonparametric Instrumental Variable Proximal Causal

September 7, 2022

On the Convergence of Monte Carlo UCB for Random-Length Episodic MDPs
Zixuan Dong, Che Wang, Keith Ross
Reinforcement Learning Early Stage Convergence Q Learning Monte Carlo Finite Horizon Q Function Episodic Markov Decision Process

August 17, 2022

Nearly Optimal Latent State Decoding in Block MDPs
Yassir Jedra, Junghyun Lee, Alexandre Proutière, Se-Young Yun
Latent State Near Optimal Policy Episodic Markov Decision Process Reward Free

August 11, 2022

Best Policy Identification in Linear MDPs
Jerome Taupin, Yassir Jedra, Alexandre Proutiere
Optimal Policy Episodic Markov Decision Process Linear MDPs Optimal Sampling Policy Identification

July 14, 2022

Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning
Baichuan Huang, Abdeslam Boularias, Jingjin Yu
Action Space Monte Carlo Tree Search Long Horizon Episodic Markov Decision Process Episodic Reinforcement Learning

June 24, 2022

Joint Representation Training in Sequential Tasks with Shared Structure
Aldo Pacchiano, Ofir Nachum, Nilseh Tripuraneni, Peter Bartlett
Reinforcement Learning Joint Representation Episodic Markov Decision Process Parallel Algorithm Sequential Task Shared Model

June 23, 2022

Provably Efficient Model-Free Constrained RL with Linear Function Approximation
Arnob Ghosh, Xingyu Zhou, Ness Shroff
Primal Dual Sublinear Regret Model Free Reinforcement Learning Linear Function Approximation Episodic Markov Decision Process Constraint Violation

June 19, 2022

Guarantees for Epsilon-Greedy Reinforcement Learning with Function Approximation
Christoph Dann, Yishay Mansour, Mehryar Mohri, Ayush Sekhari, Karthik Sridharan
Reinforcement Learning Function Approximation Episodic Markov Decision Process Formal Guarantee Epsilon Greedy Myopic Exploration

June 2, 2022

Incrementality Bidding via Reinforcement Learning under Mixed and Delayed Rewards
Ashwinkumar Badanidiyuru, Zhe Feng, Tianxi Li, Haifeng Xu
Reinforcement Learning Episodic Markov Decision Process Automated Conversion Novel Reinforcement Learning Mixed Speech Incremental Model Reward Feedback Reward Delay

May 23, 2022

Learning to branch with Tree MDPs
Lara Scavuzzo, Feng Yang Chen, Didier Chételat, Maxime Gasse, Andrea Lodi, Neil Yorke-Smith, Karen Aardal
Reinforcement Learning LeArning Abstract Policy Gradient Episodic Markov Decision Process

March 17, 2022

Near Instance-Optimal PAC Reinforcement Learning for Deterministic MDPs
Andrea Tirinzoni, Aymen Al-Marjani, Emilie Kaufmann
Reinforcement Learning Markov Decision Process Optimal Policy Episodic Markov Decision Process Probably Approximately Correct Minimax Optimal Algorithm

February 22, 2022

A policy gradient approach for optimization of smooth risk measures
Nithia Vijayan, Prashanth L. A
Optimization Purpose Policy Gradient Episodic Markov Decision Process Risk Sensitive Reinforcement Learning

February 10, 2022

Settling the Communication Complexity for Distributed Offline Reinforcement Learning
Juliusz Krysztof Ziomek, Jun Wang, Yaodong Yang
Offline Reinforcement Learning Contextual Bandit Offline Reinforcement Learning Algorithm Communication Complexity Episodic Markov Decision Process Minimax Rate

Episodic Markov Decision Process

Papers

Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation

A Best-of-Both-Worlds Algorithm for Constrained MDPs with Long-Term Constraints

Online Reinforcement Learning in Markov Decision Process Using Linear Programming

A Near-Optimal Algorithm for Safe Reinforcement Learning Under Instantaneous Hard Constraints

Model-Based Reinforcement Learning with Multinomial Logistic Function Approximation

Opportunistic Episodic Reinforcement Learning

Square-root regret bounds for continuous-time episodic Markov decision processes

Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Models

On the Convergence of Monte Carlo UCB for Random-Length Episodic MDPs

Nearly Optimal Latent State Decoding in Block MDPs

Best Policy Identification in Linear MDPs

Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning

Joint Representation Training in Sequential Tasks with Shared Structure

Provably Efficient Model-Free Constrained RL with Linear Function Approximation

Guarantees for Epsilon-Greedy Reinforcement Learning with Function Approximation

Incrementality Bidding via Reinforcement Learning under Mixed and Delayed Rewards

Learning to branch with Tree MDPs

Near Instance-Optimal PAC Reinforcement Learning for Deterministic MDPs

A policy gradient approach for optimization of smooth risk measures

Settling the Communication Complexity for Distributed Offline Reinforcement Learning