Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

December 2, 2022

Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discovery
Yiqin Yang, Hao Hu, Wenzhe Li, Siyuan Li, Jun Yang, Qianchuan Zhao, Chongjie Zhang
Offline Reinforcement Learning External Control Reinforcement Learning Algorithm Flow Mood Simple Primitive Hierarchical Policy

December 1, 2022

Launchpad: Learning to Schedule Using Offline and Online RL Methods
Vanamala Venkataswamy, Jake Grigsby, Andrew Grimshaw, Yanjun Qi
Offline Reinforcement Learning Data Driven Policy OpTimization Task Scheduling Action Free Offline RL Method

November 30, 2022

November 29, 2022

November 28, 2022

November 27, 2022

Domain Generalization for Robust Model-Based Offline Reinforcement Learning
Alan Clark, Shoaib Ahmed Siddiqui, Robert Kirk, Usman Anwar, Stephen Chung, David Krueger
Offline Reinforcement Learning Domain Generalization Robust Version Model Based Offline Reinforcement Learning Offline RL Based Offline RL

November 23, 2022

On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation
Thanh Nguyen-Tang, Ming Yin, Sunil Gupta, Svetha Venkatesh, Raman Arora
Offline Reinforcement Learning Information Theoretic Linear Function Approximation Function Approximation Instance Dependent

November 21, 2022

November 20, 2022

Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows
Dmitriy Akimov, Vladislav Kurenkov, Alexander Nikulin, Denis Tarasov, Sergey Kolesnikov
Reinforcement Learning Latent Space Offline Reinforcement Learning Normalizing Flow Latent State Training Agent Latent Action Space

November 17, 2022

On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning
Shiro Takagi
Fine Tuning Transformer Based Offline Reinforcement Learning Mixed Effect Transformer Based Model Meaningful Representation Pre Trained Transformer Different Modality

November 15, 2022

November 9, 2022

Leveraging Offline Data in Online Reinforcement Learning
Andrew Wagenmaker, Aldo Pacchiano
Reinforcement Learning Offline Reinforcement Learning Optimal Policy Online Reinforcement Learning Offline Data Offline Dataset

November 8, 2022

ARMOR: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data
Tengyang Xie, Mohak Bhardwaj, Nan Jiang, Ching-An Cheng
Offline Reinforcement Learning Offline Data Model Driven Adversarial Model Based Offline RL Baseline Policy

November 6, 2022

Wall Street Tree Search: Risk-Aware Planning for Offline Reinforcement Learning
Dan Elbaz, Gal Novik, Oren Salzman
Offline Reinforcement Learning Tree Search Risk Aware Reward Maximizing