Episodic Reinforcement Learning

Episodic reinforcement learning (ERL) focuses on learning optimal policies within finite-length episodes, aiming to maximize cumulative rewards over a defined horizon. Current research emphasizes improving sample efficiency and addressing challenges like sparse rewards, model uncertainty, and multi-agent interactions, often employing model-free and model-based algorithms, including variations of Q-learning, actor-critic methods, and Monte Carlo Tree Search. These advancements are significant for real-world applications such as robotics and autonomous systems, where efficient learning from limited interactions and adaptation to unforeseen circumstances are crucial.

Papers

November 9, 2024

Model Selection for Average Reward RL with Application to Utility Maximization in Repeated Games
Alireza Masoumian, James R. Wright
Optimal Policy Model Selection Task Utility Episodic Reinforcement Learning Balancing Efficiency Memory Constraint Stable Regret Sequential Game

July 18, 2024

Optimistic Q-learning for average reward and episodic reinforcement learning
Priyank Agrawal, Shipra Agrawal
Q Learning Regret Minimization Episodic Reinforcement Learning Model Free Algorithm Average Reward Average Reward Reinforcement Learning

July 14, 2024

Learning to Steer Markovian Agents under Model Uncertainty
Jiawei Huang, Vinzenz Thoma, Zebang Shen, Heinrich H. Nax, Niao He
LeArning Abstract Multi Agent System Model Uncertainty Learning Dynamic Episodic Reinforcement Learning Steering Technique Markovian Policy

June 29, 2024

Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning
Gautham Vasan, Yan Wang, Fahim Shahriar, James Bergstra, Martin Jagersand, A. Rupam Mahmood
Robot Learning Sparse Reward Episodic Reinforcement Learning Conditioned Policy Goal Oriented Reinforcement Learning

June 5, 2024

"Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations
Muhan Hou, Koen Hindriks, A.E. Eiben, Kim Baraka
Reinforcement Learning Noisy Demonstration Single Example Episodic Reinforcement Learning Agent Exploration Demonstration Trajectory

June 3, 2024

Combinatorial Multivariant Multi-Armed Bandits with Applications to Episodic Reinforcement Learning and Beyond
Xutong Liu, Siwei Wang, Jinhang Zuo, Han Zhong, Xuchuang Wang, Zhiyong Wang, Shuai Li, Mohammad Hajiesmaili, John C. S. Lui, Wei Chen
Financial Application Multi Armed Bandit Regret Bound Episodic Reinforcement Learning Upper Confidence Bound Combinatorial Multi Armed Bandit Multivariate Probability Distribution

May 29, 2024

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning
Avik Kar, Rahul Singh
Optimal Policy Finite Horizon State Action Space Episodic Reinforcement Learning Metric Space Continuous MDPs

April 30, 2024

Provably Efficient Information-Directed Sampling Algorithms for Multi-Agent Reinforcement Learning
Qiaosheng Zhang, Chenjia Bai, Shuyue Hu, Zhen Wang, Xuelong Li
Multi Agent Reinforcement Learning Markov Game Episodic Reinforcement Learning Sampling Based Algorithm General Sum Markov Game Information Directed Sampling

March 1, 2024

Snapshot Reinforcement Learning: Leveraging Prior Trajectories for Efficiency
Yanxiao Zhao, Yangge Qian, Tianyi Wang, Jingyang Shan, Xiaolin Qin
Deep Reinforcement Learning High Efficiency Episodic Reinforcement Learning Synthetic Trajectory Single Snapshot Baseline Algorithm

January 21, 2024

Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learning
Ge Li, Hongyi Zhou, Dominik Roth, Serge Thilges, Fabian Otto, Rudolf Lioutikov, Gerhard Neumann
Reinforcement Learning Black Box Episodic Reinforcement Learning Binary Forward Exploration Step Reinforcement Learning Policy Update

January 19, 2024

Episodic Reinforcement Learning with Expanded State-reward Space
Dayang Liang, Yaru Zhang, Yunlong Liu
Deep Reinforcement Learning Episodic Memory Model Free Deep Reinforcement Learning Episodic Reinforcement Learning Prompt Expansion State Reward Episodic Control

January 18, 2024

Learning Non-myopic Power Allocation in Constrained Scenarios
Arindam Chowdhury, Santiago Paternain, Gunjan Verma, Ananthram Swami, Santiago Segarra
Wireless Network Episodic Reinforcement Learning Power Allocation Power Constraint

July 26, 2023

Actions Speak What You Want: Provably Sample-Efficient Reinforcement Learning of the Quantal Stackelberg Equilibrium from Strategic Feedbacks
Siyu Chen, Mengdi Wang, Zhuoran Yang
Reinforcement Learning Optimal Policy Past Action Episodic Reinforcement Learning Sample Efficient Reinforcement Learning Entropy Regularized Feedback Mechanism Stackelberg Equilibrium

May 15, 2023

Horizon-free Reinforcement Learning in Adversarial Linear Mixture MDPs
Kaixuan Ji, Qingyue Zhao, Jiafan He, Weitong Zhang, Quanquan Gu
Regret Bound Episodic Reinforcement Learning Adversarial Reinforcement Learning Adversarial Markov Decision Process Planning Horizon Model Free Policy

April 15, 2023

STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning
Sirui Chen, Zhaowei Zhang, Yaodong Yang, Yali Du
Multi Agent Reinforcement Learning Temporal Attention Episodic Reinforcement Learning Temporal Credit Assignment Global Reward Return Decomposition

April 11, 2023

A Tale of Sampling and Estimation in Discounted Reinforcement Learning
Alberto Maria Metelli, Mirco Mutti, Marcello Restelli
Reinforcement Learning Estimation Task Policy OpTimization Open Sampling Episodic Reinforcement Learning Cautionary TALE Markov Reward Process Moving Horizon

March 7, 2023

Chance-Aware Lane Change with High-Level Model Predictive Control Through Curriculum Reinforcement Learning
Yubin Wang, Yulin Li, Zengqi Peng, Hakim Ghazzai, Jun Ma
Episodic Reinforcement Learning Curriculum Reinforcement Learning

February 28, 2023

Exploiting Multiple Abstractions in Episodic RL via Reward Shaping
Roberto Cipollone, Giuseppe De Giacomo, Marco Favorito, Luca Iocchi, Fabio Patrizi
Reinforcement Learning Markov Decision Process Reward Shaping Active Exploration Episodic Reinforcement Learning Level Abstraction

February 16, 2023

Quantum Computing Provides Exponential Regret Improvement in Episodic Reinforcement Learning
Bhargav Ganguly, Yulian Wu, Di Wang, Vaneet Aggarwal
Quantum Computing Quantum Algorithm Regret Analysis Finite Horizon Episodic Reinforcement Learning Quantum Reinforcement Learning

February 7, 2023

Online Reinforcement Learning with Uncertain Episode Lengths
Debmalya Mandal, Goran Radanovic, Jiarui Gan, Adish Singla, Rupak Majumdar
Reinforcement Learning Algorithm Online Reinforcement Learning Episodic Reinforcement Learning Uncertain Episode Length

Episodic Reinforcement Learning

Papers

Model Selection for Average Reward RL with Application to Utility Maximization in Repeated Games

Optimistic Q-learning for average reward and episodic reinforcement learning

Learning to Steer Markovian Agents under Model Uncertainty

Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning

"Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations

Combinatorial Multivariant Multi-Armed Bandits with Applications to Episodic Reinforcement Learning and Beyond

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning

Provably Efficient Information-Directed Sampling Algorithms for Multi-Agent Reinforcement Learning

Snapshot Reinforcement Learning: Leveraging Prior Trajectories for Efficiency

Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learning

Episodic Reinforcement Learning with Expanded State-reward Space

Learning Non-myopic Power Allocation in Constrained Scenarios

Actions Speak What You Want: Provably Sample-Efficient Reinforcement Learning of the Quantal Stackelberg Equilibrium from Strategic Feedbacks

Horizon-free Reinforcement Learning in Adversarial Linear Mixture MDPs

STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning

A Tale of Sampling and Estimation in Discounted Reinforcement Learning

Chance-Aware Lane Change with High-Level Model Predictive Control Through Curriculum Reinforcement Learning

Exploiting Multiple Abstractions in Episodic RL via Reward Shaping

Quantum Computing Provides Exponential Regret Improvement in Episodic Reinforcement Learning

Online Reinforcement Learning with Uncertain Episode Lengths