Episodic Reinforcement Learning

Episodic reinforcement learning (ERL) focuses on learning optimal policies within finite-length episodes, aiming to maximize cumulative rewards over a defined horizon. Current research emphasizes improving sample efficiency and addressing challenges like sparse rewards, model uncertainty, and multi-agent interactions, often employing model-free and model-based algorithms, including variations of Q-learning, actor-critic methods, and Monte Carlo Tree Search. These advancements are significant for real-world applications such as robotics and autonomous systems, where efficient learning from limited interactions and adaptation to unforeseen circumstances are crucial.

Papers

December 28, 2021

Exponential Family Model-Based Reinforcement Learning via Score Matching
Gene Li, Junbo Li, Anmol Kabra, Nathan Srebro, Zhaoran Wang, Zhuoran Yang
Model Based Reinforcement Learning O$ Regret Score Matching Ridge Regression Episodic Reinforcement Learning Efficient Estimation Optimistic Algorithm

December 19, 2021

Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex models
Lukasz Szpruch, Tanut Treetanthiploet, Yufei Zhang
Model Based Reinforcement Learning Finite Horizon Episodic Reinforcement Learning Exploration Exploitation Trade Performance Gap Convex Model

December 17, 2021

Autonomous Reinforcement Learning: Formalism and Benchmarking
Archit Sharma, Kelvin Xu, Nikhil Sardana, Abhishek Gupta, Karol Hausman, Sergey Levine, Chelsea Finn
Reinforcement Learning Benchmark Platform Episodic Reinforcement Learning Analysis Formalism Autonomous Reinforcement Learning

December 3, 2021

Episodic Policy Gradient Training
Hung Le, Majid Abdolshah, Thommen K. George, Kien Do, Dung Nguyen, Svetha Venkatesh
Reinforcement Learning Policy Gradient Episodic Memory Episodic Reinforcement Learning Episodic Training

November 26, 2021

Learning Long-Term Reward Redistribution via Randomized Return Decomposition
Zhizhou Ren, Ruihan Guo, Yuan Zhou, Jian Peng
Reinforcement Learning Episodic Reinforcement Learning Proxy Reward Reward Redistribution Episodic Reward Return Decomposition

November 15, 2021

Optimism and Delays in Episodic Reinforcement Learning
Benjamin Howson, Ciara Pike-Burke, Sarah Filippi
Simple Regret Episodic Reinforcement Learning Significant Delay Change Optimistic Algorithm Delay Distribution

October 18, 2021

Optimistic Policy Optimization is Provably Efficient in Non-stationary MDPs
Han Zhong, Zhongren Chen, Zhuoran Yang, Zhaoran Wang, Csaba Szepesvári
Markov Decision Process Policy OpTimization Policy Optimization Episodic Reinforcement Learning Stationary Kernel Optimistic Policy Gradient