Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

April 25, 2024

Offline Reinforcement Learning with Behavioral Supervisor Tuning
Padmanaba Srinivasan, William Knottenbelt
Offline Reinforcement Learning Policy Generalization Offline Dataset Human Supervisor

April 19, 2024

Single-Task Continual Offline Reinforcement Learning
Sibo Gai, Donglin Wang
Offline Reinforcement Learning Continual Learning Continual Reinforcement Learning Offline Visual Representation Learning

April 18, 2024

TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents
Chen Gong, Kecen Li, Jin Yao, Tianhao Wang
Reinforcement Learning Offline Reinforcement Learning Anti Forgetting Offline Reinforcement Learning Algorithm

April 16, 2024

Offline Trajectory Generalization for Offline Reinforcement Learning
Ziqi Zhao, Zhaochun Ren, Liu Yang, Fajie Yuan, Pengjie Ren, Zhumin Chen, jun Ma, Xin Xin
Offline Reinforcement Learning Trajectory Wise Reward Trajectory Generalization

April 15, 2024

Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RL
Fangwei Zhong, Kui Wu, Hai Ci, Churan Wang, Hao Chen
Offline Reinforcement Learning Object Tracking Visual Tracking Visual Foundation Model Robust Tracking Egocentric Vision

April 14, 2024

Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts
Jing-Cheng Pang, Si-Hang Yang, Kaiyuan Li, Jiaji Zhang, Xiong-Hui Chen, Nan Tang, Yang Yu
Reinforcement Learning Offline Reinforcement Learning Informed Agent

April 11, 2024

Leveraging Domain-Unlabeled Data in Offline Reinforcement Learning across Two Domains
Soichiro Nishimori, Xin-Qiang Cai, Johannes Ackermann, Masashi Sugiyama
Offline Reinforcement Learning Domain Name Positive Unlabeled Learning

April 9, 2024

Diverse Randomized Value Functions: A Provably Pessimistic Approach for Offline Reinforcement Learning
Xudong Yu, Chenjia Bai, Hongyi Guo, Changhong Wang, Zhen Wang
Offline Reinforcement Learning Uncertainty Quantification Efficient Policy Learning Value Estimation

April 7, 2024

Percentile Criterion Optimization in Offline Reinforcement Learning
Elita A. Lobo, Cyrus Cousins, Yair Zick, Marek Petrik
Reinforcement Learning Offline Reinforcement Learning Robust Policy Ambiguity Set

April 6, 2024

Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning
Yeda Song, Dongwook Lee, Gunhee Kim
Offline Reinforcement Learning Compositional Model Transductive Approach Compositional Operation

April 3, 2024

March 29, 2024

March 27, 2024

Scaling Vision-and-Language Navigation With Offline RL
Valay Bundele, Mahesh Bhupati, Biplab Banerjee, Aditya Grover
Offline Reinforcement Learning Vision and Language Navigation Reward Conditioned

March 26, 2024

Uncertainty-aware Distributional Offline Reinforcement Learning
Xiaocong Chen, Siyu Wang, Tong Yu, Lina Yao
Offline Reinforcement Learning Epistemic Uncertainty Robust Offline Reinforcement Learning Risk Aware Policy

March 25, 2024

Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data
Zeyu Jia, Alexander Rakhlin, Ayush Sekhari, Chen-Yu Wei
Offline Reinforcement Learning Integral Role Trajectory Data Offline Policy Evaluation Multiple Realizability Bellman Completeness

March 20, 2024

GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot
Wenxuan Song, Han Zhao, Pengxiang Ding, Can Cui, Shangke Lyu, Yaning Fan, Donglin Wang
Multi Task Learning Offline Reinforcement Learning Mixture of Expert Robot Learning Quadrupedal Robot Generalist Robot

March 19, 2024

Offline Reinforcement Learning

Papers

Offline Reinforcement Learning with Behavioral Supervisor Tuning

Single-Task Continual Offline Reinforcement Learning

TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents

Offline Trajectory Generalization for Offline Reinforcement Learning

Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RL

Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts

Leveraging Domain-Unlabeled Data in Offline Reinforcement Learning across Two Domains

Diverse Randomized Value Functions: A Provably Pessimistic Approach for Offline Reinforcement Learning

Percentile Criterion Optimization in Offline Reinforcement Learning

Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning

Grid-Mapping Pseudo-Count Constraint for Offline Reinforcement Learning

AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset

EnCoMP: Enhanced Covert Maneuver Planning with Adaptive Threat-Aware Visibility Estimation using Offline Reinforcement Learning

CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning

Scaling Vision-and-Language Navigation With Offline RL

Uncertainty-aware Distributional Offline Reinforcement Learning

Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data

GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot

Simple Ingredients for Offline Reinforcement Learning

Sample Complexity of Offline Distributionally Robust Linear Markov Decision Processes