Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

December 3, 2023

Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings
Henrik von Kleist, Alireza Zamanian, Ilya Shpitser, Narges Ahmidi
Global Evaluation Offline Reinforcement Learning Set Based Feature Acquisition Time Dependent Feature Active Feature Acquisition

November 30, 2023

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
Haruka Kiyohara, Ren Kishimoto, Kosuke Kawakami, Ken Kobayashi, Kazuhide Nakata, Yuta Saito
Offline Reinforcement Learning Policy Evaluation Policy Learning Python Library Scope Sensitive Offline RL

November 29, 2023

November 28, 2023

November 27, 2023

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning
Jianxiong Li, Shichao Lin, Tianyu Shi, Chujie Tian, Yu Mei, Jian Song, Xianyuan Zhan, Ruimin Li
Offline Reinforcement Learning Data Driven Approach Traffic Simulator Sample Efficient Offline Reinforcement Learning Traffic Flow Theory

November 25, 2023

Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning
Melrose Roderick, Gaurav Manek, Felix Berkenkamp, J. Zico Kolter
Offline Reinforcement Learning Policy Reinforcement Learning Actor Critic Algorithm

November 19, 2023

Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets
Kun Yang, Cong Shen, Jing Yang, Shu-ping Yeh, Jerry Sydir
Reinforcement Learning Offline Reinforcement Learning Q Learning Offline Reinforcement Learning Algorithm Optimal Reinforcement Learning Wireless Network Optimization Dataset Mixture

November 15, 2023

Supported Trust Region Optimization for Offline Reinforcement Learning
Yixiu Mao, Hongchang Zhang, Chen Chen, Yi Xu, Xiangyang Ji
Offline Reinforcement Learning Trust Region Trust Region Policy Optimization

November 3, 2023

AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation
Daiki E. Matsunaga, Jongmin Lee, Jaeseok Yoon, Stefanos Leonardos, Pieter Abbeel, Kee-Eung Kim
Offline Reinforcement Learning DIstribution Estimation Offline Multi Agent Reinforcement Learning Distribution State

October 31, 2023

October 28, 2023

October 27, 2023

October 26, 2023

CROP: Conservative Reward for Model-based Offline Policy Optimization
Hao Li, Xiao-Hu Zhou, Xiao-Liang Xie, Shi-Qi Liu, Zhen-Qiu Feng, Xiao-Yin Liu, Mei-Jiang Gui, Tian-Yu Xiang, De-Xing Huang, Bo-Xian Yao, Zeng-Guang Hou
Offline Reinforcement Learning Model Based Offline Reinforcement Learning Offline Policy Conservative Reward

Offline Reinforcement Learning

Papers

Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation

Self-Driving Telescopes: Autonomous Scheduling of Astronomical Observation Campaigns with Offline Reinforcement Learning

Stable Online and Offline Reinforcement Learning for Antibody CDRH3 Design

End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight

Digital Twin-Enhanced Deep Reinforcement Learning for Resource Management in Networks Slicing

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning

Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets

Supported Trust Region Optimization for Offline Reinforcement Learning

AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation

A Tractable Inference Perspective of Offline RL

Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving

Offline RL with Observation Histories: Analyzing and Improving Sample Complexity

Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning

Robust Offline Reinforcement learning with Heavy-Tailed Rewards

Unsupervised Behavior Extraction via Random Intent Priors

Bridging Distributionally Robust Learning and Offline RL: An Approach to Mitigate Distribution Shift and Partial Data Coverage

Guided Data Augmentation for Offline Reinforcement Learning and Imitation Learning

CROP: Conservative Reward for Model-based Offline Policy Optimization