Episodic RL

RL: Reinforcement Learning

Latest papers

11 papers0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Closing the Log Gap: Minimax-Optimal Regret for Episodic RL
Oct 29, 2026Guy Zamir, Victor Boone, Matthew Zurek +1Episodic RL Reinforcement Learning
Department of Computer Sciences, University of Wisconsin–Madison · IRIT, Université de Toulouse, CNRS, Toulouse INP, Toulouse, France
Provably Efficient Reinforcement Learning in Continuous-Time Episodic MDPs with Poisson Decision Epochs
Sep 19, 2026Kenny Guo, Valentio Iverson, Sahan Wijetunga +1Markov Decision Processes Continuous-Time RL
Department of Economics, Yale University, New Haven, Connecticut, USA · Department of Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · Department of Mathematics, University of California, Los Angeles, Los Angeles, California, USA
Decentralized Multi-Player Q-Learning in Episodic Markov Decision Processes with Information Asymmetry
Aug 13, 2026Larissa Xu, King Bi, William ChangEpisodic RL Regret Minimization in RL
Department of Mathematics University of California, Los Angeles Los Angeles, USA · Department of Computer Science University of California, Los Angeles Los Angeles, USA
Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learning
Jun 3, 2026Zicheng Zhao, Yu Lan, Chengzhengxu Li +2Episodic RL Reinforcement Learning
Xi’an Jiaotong University · Queen Mary University of London
Pure Exploration for a Good Policy in Reinforcement Learning with Bandit Feedback
May 22, 2026Zitian Li, Wang Chi CheungReinforcement Learning RL Exploration
Department of Industrial Systems Engineering & Management National University of Singapore Engineering Drive 2 Block E1A #06-25 Singapore 117576
Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration
May 21, 2026Lily Goli, Justin Kerr, Daniele Reda +3Episodic RL Reinforcement Learning
University of Toronto · Wayve · Vector Institute +2
ReversedQ: Opportunities for Faster Q-Learning in Episodic Online Reinforcement Learning
May 20, 2026Sofia R. Miskala-Dinc, Aviva PrinsEpisodic RL Reinforcement Learning
University of Maryland College Park, MD, United States
Tighter Regret Bounds for Contextual Action-Set Reinforcement Learning
May 15, 2026Zijun Chen, Zihan ZhangRegret Minimization in RL Reinforcement Learning
Department of Computer Science and Engineering Hong Kong University of Science and Technology
Extending Differential Temporal Difference Methods for Episodic Problems
May 6, 2026Kris De Asis, Mohamed Elsayed, Jiamin HeEpisodic RL Reinforcement Learning
1Openmind Research Institute · Department of Computing Science, University of Alberta, Canada · 3Alberta Machine Intelligence Institute (Amii)
Learning From Failures: Efficient Reinforcement Learning Control with Episodic Memory
Mar 7, 2026Chenyang MiaoEpisodic RL Reinforcement Learning
Xi’an Jiaotong University
Agentic Episodic Control
Jun 2, 2025Xidong Yang, Wenhao Li, Junjie Sheng +4Episodic RL Agentic RL
East China Normal University · Tongji University · Shanghai Jiao Tong University +2

Oct 29, 2026J/K move · Enter open · S save

Closing the Log Gap: Minimax-Optimal Regret for Episodic RL

Guy Zamir, Victor Boone, Matthew Zurek, Yudong Chen

Department of Computer Sciences, University of Wisconsin–Madison · IRIT, Université de Toulouse, CNRS, Toulouse INP, Toulouse, France

Episodic RL Reinforcement Learning Regret Minimization in RL Minimax Regret UCB Algorithms Model-Based RL

Open paper AskPDF arXiv

In this paper, we study the regret minimization problem in episodic tabular MDPs. Our algorithm achieves the first expected regret bound scaling as

\sqrt{H^3 S A K}

for sufficiently large

K

, where

H

is the episode length,

K

is the number of episodes, and

S

and

A

are the numbers of states and actions, respectively. Our method is the first to match the minimax lower bound without extra polylog factors. We introduce a variant of UCBVI which does not maintain strictly optimistic estimates, and uses MOSS-style exploration bonuses whose logarithmic terms adapt to both visit counts and estimated variances. A key challenge is controlling the propagation of errors caused by under-optimism, which we accomplish in an occupancy-weighted way, avoiding logarithmic overhead that would be incurred by uniform-over-rows concentration. Our analysis develops time-uniform and variance-aware concentration bounds that may be of independent interest. As a corollary we also develop an algorithm for

(\varepsilon, δ)

-PAC best-policy identification, which is the first to match the

Ω\big(H^3 SA\log(\frac{1}δ)\varepsilon^{-2}\big)

minimax lower bound up to constants in some regimes.