Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

June 13, 2023

Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective
Zeyu Zhang, Yi Su, Hui Yuan, Yiran Wu, Rishab Balasubramanian, Qingyun Wu, Huazheng Wang, Mengdi Wang
Offline Reinforcement Learning Policy Learning Stable Rank Reinforcement Learning Approach Offline Reinforcement Learning Algorithm Click Model

June 12, 2023

Improving Offline-to-Online Reinforcement Learning with Q-Ensembles
Kai Zhao, Yi Ma, Jianye Hao, Jinyi Liu, Yan Zheng, Zhaopeng Meng
Reinforcement Learning Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Ensemble Based Q Ensemble

June 11, 2023

Policy Regularization with Dataset Constraint for Offline Reinforcement Learning
Yuhang Ran, Yi-Chen Li, Fuxiang Zhang, Zongzhang Zhang, Yang Yu
Offline Reinforcement Learning Policy Regularization Action Sampling

June 10, 2023

HIPODE: Enhancing Offline Reinforcement Learning with High-Quality Synthetic Data from a Policy-Decoupled Approach
Shixi Lian, Yi Ma, Jinyi Liu, Yan Zheng, Zhaopeng Meng
Synthetic Data Offline Reinforcement Learning Reinforcement Learning Model Augmentation Policy Policy Decomposition

June 9, 2023

Iteratively Refined Behavior Regularization for Offline Reinforcement Learning
Xiaohan Hu, Yi Ma, Chenjun Xiao, Yan Zheng, Jianye Hao
Offline Reinforcement Learning Conservative Policy Behavior Regularization

June 8, 2023

June 7, 2023

Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL
Peng Cheng, Xianyuan Zhan, Zhihao Wu, Wenjia Zhang, Shoucheng Song, Han Wang, Youfang Lin, Li Jiang
Offline Reinforcement Learning Object Surface Offline Reinforcement Learning Algorithm State Action Space Passive Symmetry Sample Efficient Offline Reinforcement Learning

June 6, 2023

June 5, 2023

Survival Instinct in Offline Reinforcement Learning
Anqi Li, Dipendra Misra, Andrey Kolobov, Ching-An Cheng
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm State of the Art Robustness

June 1, 2023

May 31, 2023

May 29, 2023

Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism
Zihao Li, Zhuoran Yang, Mengdi Wang
Reinforcement Learning Offline Reinforcement Learning Human Feedback Near Optimal Policy Dynamic Discrete Choice

May 26, 2023

Future-conditioned Unsupervised Pretraining for Decision Transformer
Zhihui Xie, Zichuan Lin, Deheng Ye, Qiang Fu, Wei Yang, Shuai Li
Offline Reinforcement Learning Pre Training Multi Scenario Decision Transformer Unsupervised Pre Training Pre Trained Reinforcement Learning

May 24, 2023

May 23, 2023

When should we prefer Decision Transformers for Offline Reinforcement Learning?
Prajjwal Bhargava, Rohan Chitnis, Alborz Geramifard, Shagun Sodhani, Amy Zhang
Imitation Learning Offline Reinforcement Learning Q Learning Decision Transformer Conservative Q Learning Robotics Benchmark