Offline Deep Reinforcement Learning

Offline deep reinforcement learning (offline DRL) aims to train reinforcement learning agents using pre-collected datasets, eliminating the need for costly and potentially risky real-world interaction. Current research emphasizes improving the robustness and sample efficiency of algorithms like Conservative Q-Learning (CQL) and Advantage Weighted Actor-Critic (AWAC), often focusing on techniques like pre-training with synthetic data and careful policy selection methods to mitigate overfitting and improve generalization. This field is significant because it enables the application of DRL to safety-critical domains and facilitates more standardized and reproducible research through the development of comprehensive benchmark datasets and open-source libraries.

Papers

August 15, 2024

D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
Rafael Rafailov, Kyle Hatch, Anikait Singh, Laura Smith, Aviral Kumar, Ilya Kostrikov, Philippe Hansen-Estruch, Victor Kolev, Philip Ball, Jiajun Wu, Chelsea Finn, Sergey Levine
Real World Data Driven Diverse Datasets Offline Deep Reinforcement Learning Offline RL Algorithm

March 19, 2024

Simple Ingredients for Offline Reinforcement Learning
Edoardo Cetin, Andrea Tirinzoni, Matteo Pirotta, Alessandro Lazaric, Yann Ollivier, Ahmed Touati
Offline Reinforcement Learning Diverse Datasets Complete Recipe Heterogeneous Source Offline Deep Reinforcement Learning

December 4, 2023

When is Offline Policy Selection Sample Efficient for Reinforcement Learning?
Vincent Liu, Prabhat Nagarajan, Andrew Patterson, Martha White
Reinforcement Learning Policy Evaluation Hyperparameter Tuning Offline Policy Offline Deep Reinforcement Learning

November 6, 2023

Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimization
Kun Lei, Zhengmao He, Chenhao Lu, Kaizhe Hu, Yang Gao, Huazhe Xu
Reinforcement Learning Online Reinforcement Learning Online Service Offline Policy Policy Improvement Online Fine Tuning Offline Deep Reinforcement Learning

October 1, 2023

Pre-training with Synthetic Data Helps Offline Reinforcement Learning
Zecheng Wang, Che Wang, Zixuan Dong, Keith Ross
Synthetic Data Offline Reinforcement Learning Pre Training Decision Transformer Multiple Gait Offline Deep Reinforcement Learning

September 6, 2023

ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning
Linkang Du, Min Chen, Mingyang Sun, Shouling Ji, Peng Cheng, Jiming Chen, Zhikun Zhang
Open Source Trajectory Data DRL Algorithm Offline Deep Reinforcement Learning

January 31, 2023

Learning Vision-based Robotic Manipulation Tasks Sequentially in Offline Reinforcement Learning Settings
Sudhir Pratap Yadav, Rajendra Nagar, Suril V. Shah
Deep Reinforcement Learning Multi Task Learning Offline Reinforcement Learning Manipulation Task Vision Based Sequential Learning Offline Deep Reinforcement Learning

November 6, 2022

Spatio-temporal Incentives Optimization for Ride-hailing Services with Offline Deep Reinforcement Learning
Yanqiu Wu, Qingyang Li, Zhiwei Qin
Markov Decision Process Resource Allocation Spatio Temporal Learning Offline Deep Reinforcement Learning Ride Hailing Service

October 13, 2022

CORL: Research-oriented Deep Offline Reinforcement Learning Library
Denis Tarasov, Alexander Nikulin, Dmitry Akimov, Vladislav Kurenkov, Sergey Kolesnikov
Robust Evaluation Offline Deep Reinforcement Learning CLEVR X Dataset Offline to Online Reinforcement Learning

August 1, 2022

A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning
Zaharah A. Bukhsh, Nils Jansen, Hajo Molegraaf
Deep Reinforcement Learning Deep Q Learning Online Service Conservative Q Learning Maintenance Planning Offline Deep Reinforcement Learning

May 21, 2022

User-Interactive Offline Reinforcement Learning
Phillip Swazinna, Steffen Udluft, Thomas Runkler
Offline Reinforcement Learning New Hyperparameter Offline Deep Reinforcement Learning Offline RL Algorithm Hyperparameter Importance

April 25, 2022

Deep Reinforcement Learning for Online Routing of Unmanned Aerial Vehicles with Wireless Power Transfer
Kaiwen Li, Tao Zhang, Rui Wang, Ling Wang
Deep Reinforcement Learning Unmanned Aerial Vehicle Routing Problem Wireless Power Transfer Offline Deep Reinforcement Learning

March 13, 2022

DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learning
Jinxin Liu, Hongyin Zhang, Donglin Wang
Offline Reinforcement Learning Reward Function Offline Deep Reinforcement Learning

March 6, 2022

Offline Deep Reinforcement Learning for Dynamic Pricing of Consumer Credit
Raad Khraishi, Ramin Okhrati
Dynamic Pricing Conservative Q Learning Offline Deep Reinforcement Learning Consumer Credit

November 29, 2021

Pessimistic Model Selection for Offline Deep Reinforcement Learning
Chao-Han Huck Yang, Zhengling Qi, Yifan Cui, Pin-Yu Chen
Deep Reinforcement Learning Optimal Policy Model Selection Sequential Decision Real World Scenario Offline Deep Reinforcement Learning

November 6, 2021

d3rlpy: An Offline Deep Reinforcement Learning Library
Takuma Seno, Michita Imai
Offline Reinforcement Learning Algorithm Policy Algorithm Offline Deep Reinforcement Learning