Offline Reinforcement Learning Algorithm

Offline reinforcement learning (RL) aims to train agents using pre-collected datasets, eliminating the need for costly online interaction with the environment. Current research focuses on addressing challenges like limited data, distribution shifts between training and deployment, and the impact of data quality on performance, employing techniques such as conservative Q-learning, diffusion models, and data augmentation methods to improve policy learning and generalization. These advancements are significant for real-world applications where online learning is impractical or unsafe, particularly in robotics, autonomous driving, and other domains with high-stakes decision-making.

Papers

September 4, 2024

Tractable Offline Learning of Regular Decision Processes
Ahana Deb, Roberto Cipollone, Anders Jonsson, Alessandro Ronca, Mohammad Sadegh Talebi
Reinforcement Learning Offline Reinforcement Learning Algorithm Non Markovian Finite State Automaton

August 22, 2024

Domain Adaptation for Offline Reinforcement Learning with Limited Samples
Weiqin Chen, Sandipan Mishra, Santiago Paternain
Domain Adaptation Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Limited Sample Target Dataset

June 13, 2024

Is Value Learning Really the Main Bottleneck in Offline RL?
Seohong Park, Kevin Frans, Sergey Levine, Aviral Kumar
Reinforcement Learning Imitation Learning Offline Reinforcement Learning Policy Learning Offline Reinforcement Learning Algorithm Major Challenge Bottleneck Value Learning

June 11, 2024

Integrating Domain Knowledge for handling Limited Data in Offline RL
Briti Gangopadhyay, Zhao Wang, Jia-Fong Yeh, Shingo Takamatsu
Offline Reinforcement Learning Domain Knowledge Limited Data Offline Reinforcement Learning Algorithm Logic Regularization

June 5, 2024

UDQL: Bridging The Gap between MSE Loss and The Optimal Value Function in Offline Reinforcement Learning
Yu Zhang, Rui Yu, Zhipeng Yao, Wenyuan Zhang, Jun Wang, Liming Zhang
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Mean Squared Error Optimal Value Function Operator Prediction

May 29, 2024

Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning
Tianle Zhang, Jiayi Guan, Lin Zhao, Yihang Li, Dongjiang Li, Zecui Zeng, Lei Sun, Yue Chen, Xuelong Wei, Lusong Li, Xiaodong He
Offline Reinforcement Learning Optimal Policy Offline Reinforcement Learning Algorithm Diffusion Policy Action Noise

April 18, 2024

TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents
Chen Gong, Kecen Li, Jin Yao, Tianhao Wang
Reinforcement Learning Offline Reinforcement Learning Anti Forgetting Offline Reinforcement Learning Algorithm

April 9, 2024

Policy-Guided Diffusion
Matthew Thomas Jackson, Michael Tryfan Matthews, Cong Lu, Benjamin Ellis, Shimon Whiteson, Jakob Foerster
Diffusion Explainer Autoregressive Model Offline Reinforcement Learning Algorithm Diffusion Policy

April 3, 2024

AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset
Dongsu Lee, Chanin Eom, Minhae Kwon
Reinforcement Learning Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Autonomous Driving Benchmark

February 20, 2024

Align Your Intents: Offline Imitation Learning via Optimal Transport
Maksim Bobrin, Nazar Buzun, Dmitrii Krylov, Dmitry V. Dylov
Offline Reinforcement Learning Optimal Transport LD Align Offline Reinforcement Learning Algorithm Offline Imitation Offline Imitation Learning Intent Assurance

February 5, 2024

Contrastive Diffuser: Planning Towards High Return States via Contrastive Learning
Yixiang Shan, Zhengbang Zhu, Ting Long, Qifan Liang, Yi Chang, Weinan Zhang, Liang Yin
Contrastive Learning Offline Reinforcement Learning Task Planning Policy Learning Offline Reinforcement Learning Algorithm Multi View Diffuser Return Trajectory

February 4, 2024

January 6, 2024

On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond
Thanh Nguyen-Tang, Raman Arora
Offline Reinforcement Learning Posterior Sampling Offline Reinforcement Learning Algorithm Data Diversity Sample Efficient Offline Reinforcement Learning

November 19, 2023

Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets
Kun Yang, Cong Shen, Jing Yang, Shu-ping Yeh, Jerry Sydir
Reinforcement Learning Offline Reinforcement Learning Q Learning Offline Reinforcement Learning Algorithm Optimal Reinforcement Learning Wireless Network Optimization Dataset Mixture

October 27, 2023

Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage
Kishan Panaganti, Zaiyan Xu, Dileep Kalathil, Mohammad Ghavamzadeh
Offline Reinforcement Learning Constructive Approach Distribution Shift Limited Field Offline Reinforcement Learning Algorithm Distributionally Robust Distribution Shift Problem Single Policy Concentrability

October 19, 2023

Towards Robust Offline Reinforcement Learning under Diverse Data Corruption
Rui Yang, Han Zhong, Jiawei Xu, Amy Zhang, Chongjie Zhang, Lei Han, Tong Zhang
Offline Reinforcement Learning Offline Reinforcement Learning Algorithm Robust Offline Reinforcement Learning Data Corruption Implicit Q Learning

October 18, 2023

Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning
Jianlan Luo, Perry Dong, Jeffrey Wu, Aviral Kumar, Xinyang Geng, Sergey Levine
Offline Reinforcement Learning Skill Learning Offline Reinforcement Learning Algorithm

October 6, 2023

Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets
Zhang-Wei Hong, Aviral Kumar, Sathwik Karnik, Abhishek Bhandwaldar, Akash Srivastava, Joni Pajarinen, Romain Laroche, Abhishek Gupta, Pulkit Agrawal
Reinforcement Learning Offline Reinforcement Learning Imbalanced Datasets Offline Reinforcement Learning Algorithm Uniform Sampling Offline Policy Learning

September 29, 2023

Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness
Xiaoyu Wen, Xudong Yu, Rui Yang, Haoyuan Chen, Chenjia Bai, Zhen Wang
Reinforcement Learning High Uncertainty Anticipation Offline Reinforcement Learning Algorithm Near Optimal Policy Policy Improvement Robust Offline Reinforcement Learning Continual Offline Reinforcement Learning

Offline Reinforcement Learning Algorithm

Papers

Tractable Offline Learning of Regular Decision Processes

Domain Adaptation for Offline Reinforcement Learning with Limited Samples

Is Value Learning Really the Main Bottleneck in Offline RL?

Integrating Domain Knowledge for handling Limited Data in Offline RL

UDQL: Bridging The Gap between MSE Loss and The Optimal Value Function in Offline Reinforcement Learning

Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning

TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents

Policy-Guided Diffusion

AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset

Align Your Intents: Offline Imitation Learning via Optimal Transport

Contrastive Diffuser: Planning Towards High Return States via Contrastive Learning

The Virtues of Pessimism in Inverse Reinforcement Learning

DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching

On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond

Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets

Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage

Towards Robust Offline Reinforcement Learning under Diverse Data Corruption

Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning

Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets

Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness