Episodic Reinforcement Learning

Episodic reinforcement learning (ERL) focuses on learning optimal policies within finite-length episodes, aiming to maximize cumulative rewards over a defined horizon. Current research emphasizes improving sample efficiency and addressing challenges like sparse rewards, model uncertainty, and multi-agent interactions, often employing model-free and model-based algorithms, including variations of Q-learning, actor-critic methods, and Monte Carlo Tree Search. These advancements are significant for real-world applications such as robotics and autonomous systems, where efficient learning from limited interactions and adaptation to unforeseen circumstances are crucial.

Papers

February 28, 2023

Exploiting Multiple Abstractions in Episodic RL via Reward Shaping
Roberto Cipollone, Giuseppe De Giacomo, Marco Favorito, Luca Iocchi, Fabio Patrizi
Reinforcement Learning Markov Decision Process Reward Shaping Active Exploration Episodic Reinforcement Learning Level Abstraction

February 16, 2023

Quantum Computing Provides Exponential Regret Improvement in Episodic Reinforcement Learning
Bhargav Ganguly, Yulian Wu, Di Wang, Vaneet Aggarwal
Quantum Computing Quantum Algorithm Regret Analysis Finite Horizon Episodic Reinforcement Learning Quantum Reinforcement Learning

February 7, 2023

Online Reinforcement Learning with Uncertain Episode Lengths
Debmalya Mandal, Goran Radanovic, Jiarui Gan, Adish Singla, Rupak Majumdar
Reinforcement Learning Algorithm Online Reinforcement Learning Episodic Reinforcement Learning Uncertain Episode Length

November 23, 2022

Introspection-based Explainable Reinforcement Learning in Episodic and Non-episodic Scenarios
Niclas Schroeter, Francisco Cruz, Stefan Wermter
Episodic Reinforcement Learning Explainable Reinforcement Learning Uncertain Episode Length

October 24, 2022

Opportunistic Episodic Reinforcement Learning
Xiaoxiao Wang, Nader Bouacida, Xueying Guo, Xin Liu
Reinforcement Learning Episodic Markov Decision Process Episodic Reinforcement Learning Exploration Exploitation Trade Opportunistic Routing

October 17, 2022

You Only Live Once: Single-Life Reinforcement Learning
Annie S. Chen, Archit Sharma, Sergey Levine, Chelsea Finn
Disaster Response Single Agent Reinforcement Learning Episodic Reinforcement Learning

October 15, 2022

Near-Optimal Regret Bounds for Multi-batch Reinforcement Learning
Zihan Zhang, Yuhang Jiang, Yuan Zhou, Xiangyang Ji
Optimal Regret Finite Horizon Episodic Reinforcement Learning Batch Reinforcement Learning

October 5, 2022

October 4, 2022

A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games
Wei Xiong, Han Zhong, Chengshuai Shi, Cong Shen, Tong Zhang
Posterior Sampling Markov Game Multi Agent Decoupling Coefficient Self Play Episodic Reinforcement Learning Zero Sum Markov Game

October 3, 2022

Square-root regret bounds for continuous-time episodic Markov decision processes
Xuefeng Gao, Xun Yu Zhou
Reinforcement Learning Markov Decision Process Episodic Markov Decision Process Episodic Reinforcement Learning Continuous Time Markov Decision Process Continuous MDPs

August 29, 2022

Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learning
Anshuka Rangi, Haifeng Xu, Long Tran-Thanh, Massimo Franceschetti
Reinforcement Learning Continuum Limit Poisoning Attack Robust Reinforcement Learning Episodic Reinforcement Learning Reward Hacking

August 26, 2022

Socially Fair Reinforcement Learning
Debmalya Mandal, Jiarui Gan
Reinforcement Learning Markov Decision Process Reward Function Episodic Reinforcement Learning Gini Index

August 23, 2022

A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learning
Christoph Dann, Mehryar Mohri, Tong Zhang, Julian Zimmert
Reinforcement Learning Contextual Bandit Posterior Sampling Episodic Reinforcement Learning Linear MDPs Based Exploration

July 14, 2022

Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning
Baichuan Huang, Abdeslam Boularias, Jingjin Yu
Action Space Monte Carlo Tree Search Long Horizon Episodic Markov Decision Process Episodic Reinforcement Learning

May 23, 2022

Computationally Efficient Horizon-Free Reinforcement Learning for Linear Mixture MDPs
Dongruo Zhou, Quanquan Gu
Contextual Bandit Episodic Reinforcement Learning Linear Mixture Markov Decision Process Heterogeneous Bagging

May 11, 2022

A State-Distribution Matching Approach to Non-Episodic Reinforcement Learning
Archit Sharma, Rehaan Ahmad, Chelsea Finn
Reinforcement Learning Reinforcement Learning Algorithm Based Policy Episodic Reinforcement Learning Distribution Matching Sparse Reward Continuous Control

March 7, 2022

Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation
Alexander Long, Alan Blair, Herke van Hoof
Reinforcement Learning Algorithm Value Function Tetromino Pixel Data Efficient Episodic Reinforcement Learning Polygonal Environment Lazy Learning

March 3, 2022

The Best of Both Worlds: Reinforcement Learning with Logarithmic Regret and Policy Switches
Grigoris Velegkas, Zhuoran Yang, Amin Karbasi
Reinforcement Learning World Event Regret Minimization Regret Guarantee Episodic Reinforcement Learning Logarithmic Regret Instance Dependent Regret Policy Switching

December 29, 2021

Sequential memory improves sample and memory efficiency in Episodic Control
Ismael T. Freire, Adrián F. Amil, Paul F. M. J. Verschure
External Sample Bio Inspired Episodic Reinforcement Learning Memory Efficiency Episodic Control Sequential Memory

Episodic Reinforcement Learning

Papers

Exploiting Multiple Abstractions in Episodic RL via Reward Shaping

Quantum Computing Provides Exponential Regret Improvement in Episodic Reinforcement Learning

Online Reinforcement Learning with Uncertain Episode Lengths

Introspection-based Explainable Reinforcement Learning in Episodic and Non-episodic Scenarios

Opportunistic Episodic Reinforcement Learning

You Only Live Once: Single-Life Reinforcement Learning

Near-Optimal Regret Bounds for Multi-batch Reinforcement Learning

Reward-Mixing MDPs with a Few Latent Contexts are Learnable

Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planning

A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games

Square-root regret bounds for continuous-time episodic Markov decision processes

Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learning

Socially Fair Reinforcement Learning

A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learning

Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning

Computationally Efficient Horizon-Free Reinforcement Learning for Linear Mixture MDPs

A State-Distribution Matching Approach to Non-Episodic Reinforcement Learning

Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximation

The Best of Both Worlds: Reinforcement Learning with Logarithmic Regret and Policy Switches

Sequential memory improves sample and memory efficiency in Episodic Control