Regret Minimization in RL

RL: Reinforcement Learning

Overview

Regret minimization studies how an agent can learn an effective policy while keeping the cumulative reward it gives up, compared with an optimal or hindsight benchmark, as small as possible. Common approaches include optimistic exploration such as UCB, posterior or Thompson sampling, policy optimization, and primal-dual methods for constrained problems; variance-aware and gap-dependent analyses aim to improve guarantees when the environment is favorable. Work spans tabular finite- and infinite-horizon Markov decision processes, linear and kernel function approximation, adversarial or changing rewards, and settings with delayed, preference-based, or risk-sensitive feedback. Many settings now have near-matching upper and lower regret bounds, but a central tension remains between guarantees that adapt to problem structure and algorithms that remain computationally efficient under weaker assumptions.

Latest papers

CardsList

70 papers13 papers in the last four weeks, up 18% on the four weeks before. 0.1% of all new papers.

  1. Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferences

    Feb 26, 2026Junghyun Lee, Minju Hong, Kwang-Sung Jun +2Regret Minimization in RLPreference-Based RL

  2. Understanding Gap-Dependent Regret for Optimism-Based Reinforcement Learning with Linear Function Approximation

    Feb 23, 2026Haochen Zhang, Zhong Zheng, Lingzhou XueRegret Minimization in RLReinforcement Learning

  3. Regret and Sample Complexity of Online Q-Learning via Concentration of Stochastic Approximation with Time-Inhomogeneous Markov Chains

    Feb 18, 2026Rahul Singh, Siddharth Chandak, Eric Moulines +2Regret Minimization in RLQ-Learning

  4. Data- and Variance-dependent Regret Bounds for Online Tabular MDPs

    Feb 2, 2026Mingyi Li, Taira Tsuchiya, Kenji YamanishiRegret Minimization in RLMarkov Decision Processes

  5. Reinforcement Learning with Multi-Step Lookahead Information Via Adaptive Batching

    Jan 15, 2026Nadav MerlisReinforcement LearningRegret Minimization in RL

  6. Stagewise Reinforcement Learning and the Geometry of the Regret Landscape

    Jan 12, 2026Chris Elliott, Einar Urdshals, David Quarel +2Reinforcement LearningBayesian RL

  7. Reinforcement Learning in MDPs with Information-Ordered Policies

    Aug 5, 2025Zhongjun Zhang, Shipra Agrawal, Ilan Lobel +2Queueing TheoryPolicy Learning

  8. Safe Learning Under Irreversible Dynamics via Asking for Help

    Feb 19, 2025Benjamin Plaut, Juan Liévano-Karim, Hanlin Zhu +1Regret Minimization in RLReinforcement Learning

  9. Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning

    Sep 22, 2024Jia Wan, Sean R. Sinclair, Devavrat Shah +1Regret Minimization in RLMarkov Decision Processes

  10. Thompson Sampling for Infinite-Horizon Discounted Decision Processes

    May 14, 2024Daniel Adelman, Cagla Keceli, Alba V. Olivares-NadalThompson SamplingRegret Minimization in RL