Offline Reinforcement Learning Algorithm

Offline reinforcement learning (RL) aims to train agents using pre-collected datasets, eliminating the need for costly online interaction with the environment. Current research focuses on addressing challenges like limited data, distribution shifts between training and deployment, and the impact of data quality on performance, employing techniques such as conservative Q-learning, diffusion models, and data augmentation methods to improve policy learning and generalization. These advancements are significant for real-world applications where online learning is impractical or unsafe, particularly in robotics, autonomous driving, and other domains with high-stakes decision-making.

Papers

November 1, 2022

Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
Paria Rashidinejad, Hanlin Zhu, Kunhe Yang, Stuart Russell, Jiantao Jiao
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Conservative Policy Augmented Lagrangian Single Policy Concentrability

October 17, 2022

Boosting Offline Reinforcement Learning via Data Rebalancing
Yang Yue, Bingyi Kang, Xiao Ma, Zhongwen Xu, Gao Huang, Shuicheng Yan
Offline Reinforcement Learning Distribution Data Offline Reinforcement Learning Algorithm

October 12, 2022

Semi-Supervised Offline Reinforcement Learning with Action-Free Trajectories
Qinqing Zheng, Mikael Henaff, Brandon Amos, Aditya Grover
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Many Trajectory Offline Dataset

October 7, 2022

BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets
Chen Gong, Zhou Yang, Yunpeng Bai, Junda He, Jieke Shi, Kecen Li, Arunesh Sinha, Bowen Xu, Xinwen Hou, David Lo, Tianhao Wang
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Hidden Backdoor

June 11, 2022

Federated Offline Reinforcement Learning
Doudou Zhou, Yufeng Zhang, Aaron Sonabend-W, Zhaoran Wang, Junwei Lu, Tianxi Cai
Markov Decision Process Offline Reinforcement Learning Policy Optimization Offline Reinforcement Learning Algorithm Dynamic Treatment Regime Federated Offline Reinforcement Learning

June 7, 2022

On the Role of Discount Factor in Offline Reinforcement Learning
Hao Hu, Yiqin Yang, Qianchuan Zhao, Chongjie Zhang
Offline Reinforcement Learning Integral Role Sample Efficient Offline Reinforcement Learning Algorithm Offline Adaptation Discount Factor

June 6, 2022

RORL: Robust Offline Reinforcement Learning via Conservative Smoothing
Rui Yang, Chenjia Bai, Xiaoteng Ma, Zhaoran Wang, Chongjie Zhang, Lei Han
Reinforcement Learning Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Linear MDPs Robust Offline Reinforcement Learning Conservative Value Estimation

June 2, 2022

Offline Reinforcement Learning with Differential Privacy
Dan Qiao, Yu-Xiang Wang
Differential Privacy Offline Reinforcement Learning Data Driven Offline Reinforcement Learning Algorithm Differential Privacy Guarantee

May 26, 2022

Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes
Miao Lu, Yifei Min, Zhaoran Wang, Zhuoran Yang
Markov Decision Process Offline Reinforcement Learning Offline Reinforcement Learning Algorithm C POMDPs Text Based Confounders

March 17, 2022

Semi-Markov Offline Reinforcement Learning for Healthcare
Mehdi Fatemi, Mary Wu, Jeremy Petch, Walter Nelson, Stuart J. Connolly, Alexander Benz, Anthony Carnicelli, Marzyeh Ghassemi
Reinforcement Learning Markov Decision Process Offline Reinforcement Learning Optimal Policy Healthcare System Offline Reinforcement Learning Algorithm

February 10, 2022

Settling the Communication Complexity for Distributed Offline Reinforcement Learning
Juliusz Krysztof Ziomek, Jun Wang, Yaodong Yang
Offline Reinforcement Learning Contextual Bandit Offline Reinforcement Learning Algorithm Communication Complexity Episodic Markov Decision Process Minimax Rate

January 31, 2022

Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
Denis Yarats, David Brandfonbrener, Hao Liu, Michael Laskin, Pieter Abbeel, Alessandro Lazaric, Lerrel Pinto
Practical Algorithm Raw Data Offline Reinforcement Learning Policy Reinforcement Learning Offline Reinforcement Learning Algorithm Exploratory Data Analysis

December 31, 2021

Single-Shot Pruning for Offline Reinforcement Learning
Samin Yeasar Arnob, Riyasat Ohib, Sergey Plis, Doina Precup
Deep Reinforcement Learning Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Shot Pruning

November 22, 2021

Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Offline Multi Agent Reinforcement Learning Rectification Network First Order Policy

November 6, 2021

d3rlpy: An Offline Deep Reinforcement Learning Library
Takuma Seno, Michita Imai
Offline Reinforcement Learning Algorithm Policy Algorithm Offline Deep Reinforcement Learning