Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

December 28, 2023

Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?
Gunshi Gupta, Tim G. J. Rudner, Rowan Thomas McAllister, Adrien Gaidon, Yarin Gal
Offline Reinforcement Learning Underlying Causal Active Sampling

December 21, 2023

Critic-Guided Decision Transformer for Offline Reinforcement Learning
Yuanfu Wang, Chao Yang, Ying Wen, Yu Liu, Yu Qiao
Offline Reinforcement Learning Value Function Decision Transformer Critic Model Return Conditioned Supervised Learning

December 19, 2023

December 18, 2023

December 16, 2023

Advancing RAN Slicing with Offline Reinforcement Learning
Kun Yang, Shu-ping Yeh, Menglei Zhang, Jerry Sydir, Jing Yang, Cong Shen
Reinforcement Learning Offline Reinforcement Learning Near Optimal Policy Radio Resource Management Radio Access Network Slicing

December 15, 2023

Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation
Girolamo Macaluso, Alessandro Sestini, Andrew D. Bagdanov
Reinforcement Learning Data Augmentation Offline Reinforcement Learning Small Datasets Big Gain Offline Pre Training

December 10, 2023

The Generalization Gap in Offline Reinforcement Learning
Ishita Mediratta, Qingfei You, Minqi Jiang, Roberta Raileanu
Strong Generalization Offline Reinforcement Learning Online Reinforcement Learning Generalization Gap Offline Learning

December 7, 2023

MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operator
Xiao-Yin Liu, Xiao-Hu Zhou, Guotao Li, Hao Li, Mei-Jiang Gui, Tian-Yu Xiang, De-Xing Huang, Zeng-Guang Hou
Offline Reinforcement Learning Model Based Offline Reinforcement Learning Bellman Operator Conservative Policy

December 6, 2023

Evaluation of Active Feature Acquisition Methods for Static Feature Settings
Henrik von Kleist, Alireza Zamanian, Ilya Shpitser, Narges Ahmidi
Global Evaluation Offline Reinforcement Learning Different Missingness Static Feature Active Feature Acquisition

December 3, 2023

Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings
Henrik von Kleist, Alireza Zamanian, Ilya Shpitser, Narges Ahmidi
Global Evaluation Offline Reinforcement Learning Set Based Feature Acquisition Time Dependent Feature Active Feature Acquisition

November 30, 2023

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
Haruka Kiyohara, Ren Kishimoto, Kosuke Kawakami, Ken Kobayashi, Kazuhide Nakata, Yuta Saito
Offline Reinforcement Learning Policy Evaluation Policy Learning Python Library Scope Sensitive Offline RL

November 29, 2023

November 28, 2023

November 27, 2023

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning
Jianxiong Li, Shichao Lin, Tianyu Shi, Chujie Tian, Yu Mei, Jian Song, Xianyuan Zhan, Ruimin Li
Offline Reinforcement Learning Data Driven Approach Traffic Simulator Sample Efficient Offline Reinforcement Learning Traffic Flow Theory

November 25, 2023

Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning
Melrose Roderick, Gaurav Manek, Felix Berkenkamp, J. Zico Kolter
Offline Reinforcement Learning Policy Reinforcement Learning Actor Critic Algorithm

November 19, 2023

Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets
Kun Yang, Cong Shen, Jing Yang, Shu-ping Yeh, Jerry Sydir
Reinforcement Learning Offline Reinforcement Learning Q Learning Offline Reinforcement Learning Algorithm Optimal Reinforcement Learning Wireless Network Optimization Dataset Mixture

Offline Reinforcement Learning

Papers

Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?

Critic-Guided Decision Transformer for Offline Reinforcement Learning

CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement Learning

Neural Network Approximation for Pessimistic Offline Reinforcement Learning

Learning a Diffusion Model Policy from Rewards via Q-Score Matching

Robot Crowd Navigation in Dynamic Environment with Offline Reinforcement Learning

Advancing RAN Slicing with Offline Reinforcement Learning

Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation

The Generalization Gap in Offline Reinforcement Learning

MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operator

Evaluation of Active Feature Acquisition Methods for Static Feature Settings

Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation

Self-Driving Telescopes: Autonomous Scheduling of Astronomical Observation Campaigns with Offline Reinforcement Learning

Stable Online and Offline Reinforcement Learning for Antibody CDRH3 Design

End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight

Digital Twin-Enhanced Deep Reinforcement Learning for Resource Management in Networks Slicing

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning

Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets