Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

October 12, 2022

October 11, 2022

Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning
Tung Nguyen, Qinqing Zheng, Aditya Grover
Offline Reinforcement Learning Behavior Cloning Decision Transformer Return Trajectory

October 9, 2022

October 7, 2022

October 5, 2022

Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based Reinforcement Learning
David Brandfonbrener, Stephen Tu, Avi Singh, Stefan Welker, Chad Boodoo, Nikolai Matni, Jake Varley
Reinforcement Learning Offline Reinforcement Learning Robotic Task Data Collection Action Free Offline Deformable Object Manipulation Task

October 3, 2022

Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient
Ming Yin, Mengdi Wang, Yu-Xiang Wang
Reinforcement Learning Offline Reinforcement Learning Q Learning Differentiable Function

September 30, 2022

S2P: State-conditioned Image Synthesis for Data Augmentation in Offline Reinforcement Learning
Daesol Cho, Dongseok Shim, H. Jin Kim
Data Augmentation Offline Reinforcement Learning Image Synthesis Based Offline RL

September 29, 2022

Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling
Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, Jun Zhu
Offline Reinforcement Learning Generative Modeling Policy Parameterization Unseen Action

September 27, 2022

DCE: Offline Reinforcement Learning With Double Conservative Estimates
Chen Zhao, Kai Xing Huang, Chun yuan
Offline Reinforcement Learning Traditional Reinforcement Learning Conservative Value Estimation

September 19, 2022

September 18, 2022

Offline Reinforcement Learning with Instrumental Variables in Confounded Markov Decision Processes
Zuyue Fu, Zhengling Qi, Zhaoran Wang, Zhuoran Yang, Yanxun Xu, Michael R. Kosorok
Markov Decision Process Offline Reinforcement Learning Policy Learning Instrumental Variable Policy Learning Method

September 15, 2022

Can Offline Reinforcement Learning Help Natural Language Understanding?
Ziqi Zhang, Yile Wang, Yue Zhang, Donglin Wang
Pre Trained Model Offline Reinforcement Learning Implicit Knowledge Visually Grounded

September 14, 2022

C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning
Ci Chen, Pingyu Xiang, Haojian Lu, Yue Wang, Rong Xiong
Non Humanoid Robot Robot Person Offline Reinforcement Learning Behavior Cloning Co Design Morphological Differentiation Based Method Dual Network

September 8, 2022

Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL
Taku Yamagata, Ahmed Khalil, Raul Santos-Rodriguez
Offline Reinforcement Learning Q Learning Policy Learning Dynamic Programming Decision Transformer Sequence Modeling

September 2, 2022

Dialogue Evaluation with Offline Reinforcement Learning
Nurul Lubis, Christian Geishauser, Hsien-Chin Lin, Carel van Niekerk, Michael Heck, Shutong Feng, Milica Gašić
Offline Reinforcement Learning Dialogue System Dialogue Evaluation Dialogue Benchmark

August 23, 2022

Strategic Decision-Making in the Presence of Information Asymmetry: Provably Efficient RL with Algorithmic Instruments
Mengxin Yu, Zhuoran Yang, Jianqing Fan
Markov Decision Process Offline Reinforcement Learning Optimal Policy Decision Making Speech Presence Policy Optimization Efficient Reinforcement Learning Information Asymmetry Algorithmic System

Offline Reinforcement Learning

Papers

Real World Offline Reinforcement Learning with Realistic Data Source

A Unified Framework for Alternating Offline Model Training and Policy Learning

Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning

State Advantage Weighting for Offline RL

The Role of Coverage in Online Reinforcement Learning

Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimization

BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets

Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based Reinforcement Learning

Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient

S2P: State-conditioned Image Synthesis for Data Augmentation in Offline Reinforcement Learning

Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling

DCE: Offline Reinforcement Learning With Double Conservative Estimates

Latent Plans for Task-Agnostic Offline Reinforcement Learning

Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning

Offline Reinforcement Learning with Instrumental Variables in Confounded Markov Decision Processes

Can Offline Reinforcement Learning Help Natural Language Understanding?

C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning

Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL

Dialogue Evaluation with Offline Reinforcement Learning

Strategic Decision-Making in the Presence of Information Asymmetry: Provably Efficient RL with Algorithmic Instruments