Reward Free Reinforcement Learning

Reward-free reinforcement learning (RL) focuses on efficiently exploring an environment without initial reward information to subsequently learn optimal policies for any given reward function. Current research emphasizes developing sample-efficient algorithms, often employing model-based approaches and linear function approximation, to address the challenge of exploring diverse states and actions effectively. This research area is significant because it improves the data efficiency of RL, enabling faster learning and deployment in real-world applications where reward signals may be delayed, sparse, or expensive to obtain. The development of provably efficient algorithms and the exploration of optimal exploration strategies are key current focuses.

Papers

June 24, 2024

Uncertainty-Aware Reward-Free Exploration with General Function Approximation
Junkai Zhang, Weitong Zhang, Dongruo Zhou, Quanquan Gu
Reinforcement Learning Unsupervised Reinforcement Learning Reward Free Reinforcement Learning Reward Free Exploration

May 28, 2024

Offline Oracle-Efficient Learning for Contextual MDPs via Layerwise Exploration-Exploitation Tradeoff
Jian Qian, Haichen Hu, David Simchi-Levi
Contextual Bandit Action Free Offline Exploration Exploitation Trade Near Optimal Algorithm Reward Free Reinforcement Learning Contextual Markov Decision Process Contextual MDPs

December 16, 2023

Asymmetric Norms to Approximate the Minimum Action Distance
Lorenzo Steccanella, Anders Jonsson
Goal Conditioned Policy Learning Norm Reward Free Reinforcement Learning

October 25, 2023

Towards Control-Centric Representations in Reinforcement Learning from Images
Chen Liu, Hongyu Zang, Xin Li, Yong Heng, Yifei Wang, Zhen Fang, Yisen Wang, Mingzhong Wang
Reinforcement Learning Bisimulation Metric DeepMind Control Suite Reward Free Reinforcement Learning Image Based Reinforcement Learning Control Centric Representation

August 23, 2023

Diverse Policies Converge in Reward-free Markov Decision Processe
Fanqi Lin, Shiyu Huang, Weiwei Tu
Reinforcement Learning Diverse Policy Reward Free Reinforcement Learning Diversity Reinforcement Learning

July 26, 2023

Reinforcement Learning by Guided Safe Exploration
Qisong Yang, Thiago D. Simão, Nils Jansen, Simon H. Tindemans, Matthijs T. J. Spaan
Reinforcement Learning Reinforcement Learning Agent Reward Free Reward Free Reinforcement Learning

March 20, 2023

Improved Sample Complexity for Reward-free Reinforcement Learning under Low-rank MDPs
Yuan Cheng, Ruiquan Huang, Jing Yang, Yingbin Liang
Sample Complexity Low Rank Reward Free Reinforcement Learning Reward Free Exploration

March 17, 2023

Optimal Horizon-Free Reward-Free Exploration for Linear Mixture MDPs
Junkai Zhang, Weitong Zhang, Quanquan Gu
Reward Free Reward Free Reinforcement Learning Reward Free Exploration Linear Mixture Markov Decision Process Reward Aware

October 3, 2022

Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximation
Dan Qiao, Yu-Xiang Wang
Reinforcement Learning Optimal Policy Linear Function Approximation Linear MDPs Reward Free Reinforcement Learning Optimal Deployment

June 28, 2022

Safe Exploration Incurs Nearly No Additional Sample Complexity for Reward-free RL
Ruiquan Huang, Jing Yang, Yingbin Liang
Sample Complexity Near Optimal Policy Safe Exploration Reward Free Reinforcement Learning Reward Free Exploration Reinforcement Learning Paradigm

June 21, 2022

On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RL
Jinglin Chen, Aditya Modi, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal
Low Rank Sample Efficiency Reward Free Reinforcement Learning Reward Free Exploration Reward Aware

June 13, 2022

Provable Benefit of Multitask Representation Learning in Reinforcement Learning
Yuan Cheng, Songtao Feng, Jing Yang, Hong Zhang, Yingbin Liang
Reinforcement Learning Low Rank Provable Benefit Multi Task Representation Reward Free Reward Free Reinforcement Learning

May 31, 2022

One Policy is Enough: Parallel Exploration with a Single Policy is Near-Optimal for Reward-Free Reinforcement Learning
Pedro Cisneros-Velarde, Boxiang Lyu, Sanmi Koyejo, Mladen Kolar
Reinforcement Learning Markov Game Health Policy Recommendation Near Optimal Diverse Policy Reward Free Reinforcement Learning Simultaneous Exploration

May 4, 2022

State Representation Learning for Goal-Conditioned Reinforcement Learning
Lorenzo Steccanella, Anders Jonsson
State Representation Goal Conditioned Reinforcement Learning Goal Conditioned Policy Reward Free Reinforcement Learning Multi Goal

February 25, 2022

Learning Dynamic Mechanisms in Unknown Environments: A Reinforcement Learning Approach
Shuang Qiu, Boxiang Lyu, Qinglin Meng, Zhaoran Wang, Zhuoran Yang, Michael I. Jordan
Markov Decision Process Unknown Environment Reinforcement Learning Approach Mechanism Design Learning Mechanism Reward Free Reinforcement Learning

January 26, 2022

Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processes
Andrew Wagenmaker, Yifang Chen, Max Simchowitz, Simon S. Du, Kevin Jamieson
Reinforcement Learning Linear MDPs Reward Free Reinforcement Learning

November 23, 2021

Adaptive Multi-Goal Exploration
Jean Tarbouriech, Omar Darwiche Domingues, Pierre Ménard, Matteo Pirotta, Michal Valko, Alessandro Lazaric
Deep Reinforcement Learning Reward Free Reinforcement Learning Goal Conditioned Exploration