Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

March 16, 2023

Goal-conditioned Offline Reinforcement Learning through State Space Partitioning
Mianchu Wang, Yue Jin, Giovanni Montana
Offline Reinforcement Learning Goal Conditioned Policy Policy Improvement Offline Learning Offline Goal Conditioned Reinforcement Learning

March 14, 2023

Adaptive Policy Learning for Offline-to-Online Reinforcement Learning
Han Zheng, Xufang Luo, Pengfei Wei, Xuan Song, Dongsheng Li, Jing Jiang
Reinforcement Learning Offline Reinforcement Learning Offline Dataset Offline to Online Reinforcement Learning Policy Based Reinforcement Learning

March 13, 2023

Deploying Offline Reinforcement Learning with Human Feedback
Ziniu Li, Ke Xu, Liu Liu, Lanqing Li, Deheng Ye, Peilin Zhao
Reinforcement Learning Offline Reinforcement Learning Human Feedback Offline Reinforcement Learning Algorithm Policy Parameterization Robot Locomotion

March 9, 2023

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
Mitsuhiko Nakamoto, Yuexiang Zhai, Anikait Singh, Max Sobol Mark, Yi Ma, Chelsea Finn, Aviral Kumar, Sergey Levine
Offline Reinforcement Learning Q Learning Offline Reinforcement Learning Algorithm Conservative Q Learning Online Fine Tuning

March 7, 2023

March 3, 2023

Learning to Influence Human Behavior with Offline Reinforcement Learning
Joey Hong, Sergey Levine, Anca Dragan
LeArning Abstract Offline Reinforcement Learning AI Agent Optimal Behavior Legible Behavior

February 28, 2023

February 27, 2023

February 24, 2023

February 22, 2023

Behavior Proximal Policy Optimization
Zifeng Zhuang, Kun Lei, Jinxin Liu, Donglin Wang, Yilang Guo
Offline Reinforcement Learning Proximal Policy Optimization Offline Policy Policy Actor Critic

February 21, 2023

February 16, 2023

Dual RL: Unification and New Methods for Reinforcement and Imitation Learning
Harshit Sikchi, Qinqing Zheng, Amy Zhang, Scott Niekum
Reinforcement Learning Imitation Learning Offline Reinforcement Learning Positive Reinforcement New Method Anti Unification Dual Reinforcement Learning

February 14, 2023

Conservative State Value Estimation for Offline Reinforcement Learning
Liting Chen, Jie Yan, Zhengdao Shao, Lu Wang, Qingwei Lin, Saravan Rajmohan, Thomas Moscibroda, Dongmei Zhang
Offline Reinforcement Learning Conservative Q Learning Conservative Value Estimation Function Estimation

February 5, 2023

Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage
Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
Offline Reinforcement Learning Higher Coverage Rate Minimax Optimal Minimax Optimization Problem Multiple Realizability

Offline Reinforcement Learning

Papers

Goal-conditioned Offline Reinforcement Learning through State Space Partitioning

Adaptive Policy Learning for Offline-to-Online Reinforcement Learning

Deploying Offline Reinforcement Learning with Human Feedback

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning

On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samples

Graph Decision Transformer

Learning to Influence Human Behavior with Offline Reinforcement Learning

Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learning

The In-Sample Softmax for Offline Reinforcement Learning

(Re)$^2$H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning

The Provable Benefits of Unsupervised Data Sharing for Offline Reinforcement Learning

VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation

Neural Laplace Control for Continuous-time Delayed Systems

Plume: A Framework for High Performance Deep RL Network Controllers via Prioritized Trace Sampling

Behavior Proximal Policy Optimization

Adversarial Model for Offline Reinforcement Learning

Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management

Dual RL: Unification and New Methods for Reinforcement and Imitation Learning

Conservative State Value Estimation for Offline Reinforcement Learning

Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage