Offline RL

Offline reinforcement learning (RL) aims to train agents using pre-collected data, avoiding the need for costly and potentially unsafe online interactions. Current research focuses on addressing the challenges of distribution shift (avoiding overestimation of unseen actions) and improving the efficiency and robustness of algorithms, including those leveraging techniques like denoising score matching, implicit Q-learning, and diffusion models. These advancements are significant because they enable the application of RL to real-world scenarios where online data collection is impractical or impossible, impacting fields such as robotics and personalized medicine.

Papers

June 14, 2024

Binary Reward Labeling: Bridging Offline Preference and Reward-Based Reinforcement Learning
Yinglun Xu, David Zhu, Rohan Gumaste, Gagandeep Singh
Reinforcement Learning Offline Reinforcement Learning Offline Preference Offline RL Scalar Reward Offline Preference Based Reinforcement Learning

June 11, 2024

CDSA: Conservative Denoising Score-based Algorithm for Offline Reinforcement Learning
Zeyuan Liu, Kai Yang, Xiu Li
Offline Reinforcement Learning Offline RL Denoising Score

November 30, 2023

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
Haruka Kiyohara, Ren Kishimoto, Kosuke Kawakami, Ken Kobayashi, Kazuhide Nakata, Yuta Saito
Offline Reinforcement Learning Policy Evaluation Policy Learning Python Library Scope Sensitive Offline RL

July 24, 2023

June 1, 2023

Improving Offline RL by Blending Heuristics
Sinong Geng, Aldo Pacchiano, Andrey Kolobov, Ching-An Cheng
Offline RL Algorithm Offline RL Dual Source Blending Attack

May 31, 2023

Efficient Diffusion Policies for Offline Reinforcement Learning
Bingyi Kang, Xiao Ma, Chao Du, Tianyu Pang, Shuicheng Yan
Reinforcement Learning Offline Reinforcement Learning Policy Gradient Diffusion Policy Offline RL

April 20, 2023

IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies
Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, Sergey Levine
Actor Critic Diffusion Policy Implicit Language Semantic Actor Offline RL Implicit Q Learning

March 3, 2023

Decision Transformer under Random Frame Dropping
Kaizhe Hu, Ray Chen Zheng, Yang Gao, Huazhe Xu
Deep Reinforcement Learning Decision Transformer Offline RL RL Method

November 27, 2022

Domain Generalization for Robust Model-Based Offline Reinforcement Learning
Alan Clark, Shoaib Ahmed Siddiqui, Robert Kirk, Usman Anwar, Stephen Chung, David Krueger
Offline Reinforcement Learning Domain Generalization Robust Version Model Based Offline Reinforcement Learning Offline RL Based Offline RL

November 2, 2022

Dual Generator Offline Reinforcement Learning
Quan Vuong, Aviral Kumar, Sergey Levine, Yevgen Chebotar
Generative Adversarial Network Offline Reinforcement Learning Offline RL