Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

November 18, 2024

Enhancing Decision Transformer with Diffusion-Based Trajectory Branch Generation
Zhihong Liu, Long Qian, Zeyang Liu, Lipeng Wan, Xingyu Chen, Xuguang Lan
Offline Reinforcement Learning Faithful Generation Decision Transformer Trajectory Generation Sequence Modeling Feasible Trajectory Trajectory Refinement

November 17, 2024

An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces
Alex Beeson, David Ireland, Giovanni Montana
Offline Reinforcement Learning Action Space Comprehensive Investigation

November 14, 2024

Iterative Batch Reinforcement Learning via Safe Diversified Model-based Policy Search
Amna Najib, Stefan Depeweg, Phillip Swazinna
Reinforcement Learning Offline Reinforcement Learning Policy Learning Batch Reinforcement Learning Policy Search Model Based Policy

November 12, 2024

November 11, 2024

Streetwise Agents: Empowering Offline RL Policies to Outsmart Exogenous Stochastic Disturbances in RTC
Aditya Soni, Mayukh Das, Anjaly Parayil, Supriyo Ghosh, Shivam Shandilya, Ching-An Cheng, Vishak Gopal, Sami Khairy, Gabriel Mittag, Yasaman Hosseinkashi, Chetan Bansal
Offline Reinforcement Learning Data Driven Decision Exogenous Variable Traffic Agent Stochastic Disturbance Exogenous Information

November 8, 2024

Real-World Offline Reinforcement Learning from Vision Language Model Feedback
Sreyas Venkataraman, Yufei Wang, Ziyu Wang, Zackory Erickson, David Held
Offline Reinforcement Learning Human Feedback Inverse Reinforcement Learning Policy Learning Real Robot

November 7, 2024

November 2, 2024

Prompt Tuning with Diffusion for Few-Shot Pre-trained Policy Generalization
Shengchao Hu, Wanru Zhao, Weixiong Lin, Li Shen, Ya Zhang, Dacheng Tao
Strong Generalization Offline Reinforcement Learning Diffusion Explainer Prompt Tuning Conditional Diffusion Model Conditional Generative Health Policy Recommendation

October 31, 2024

Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers
Kai Yan, Alexander G. Schwing, Yu-Xiong Wang
Reinforcement Learning Offline Reinforcement Learning Decision Transformer Online Fine Tuning Deeper ViT S 54

October 30, 2024

Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation
Samuele Peri, Alessio Russo, Gabor Fodor, Pablo Soldati
Reinforcement Learning Offline Reinforcement Learning Adaptation Concern Sequence Modeling Radio Access Network Conservative Q Learning Downlink Communication

October 28, 2024

Offline Reinforcement Learning With Combinatorial Action Spaces
Matthew Landers, Taylor W. Killian, Hugo Barnes, Thomas Hartvigsen, Afsaneh Doryab
Offline Reinforcement Learning Action Space Reinforcement Learning Problem Branch Prediction

October 27, 2024

Q-Distribution guided Q-learning for offline reinforcement learning: Uncertainty penalized Q-value via consistency model
Jing Zhang, Linjiajie Fang, Kexin Shi, Wenjia Wang, Bing-Yi Jing
Reinforcement Learning Offline Reinforcement Learning Q Learning Consistency Model Q Value

October 25, 2024

Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression
Yixiu Mao, Cheems Wang, Chen Chen, Yun Qu, Xiangyang Ji
Offline Reinforcement Learning Action Feature Copy Suppression Optimistic Algorithm Finite State

October 23, 2024

Scalable Offline Reinforcement Learning for Mean Field Games
Axel Brunnbauer, Julian Lemmel, Zahra Babaiee, Sophie Neubauer, Radu Grosu
Offline Reinforcement Learning Robust Policy Mean Field Game Mean Field Reinforcement Learning

October 21, 2024

Offline reinforcement learning for job-shop scheduling problems
Imanol Echeverria, Maialen Murua, Roberto Santana
Deep Reinforcement Learning Offline Reinforcement Learning Combinatorial Optimization Problem Job Shop Scheduling Job Shop Scheduling Problem Flexible Job Shop Scheduling Problem

October 17, 2024

Rethinking Optimal Transport in Offline Reinforcement Learning
Arip Asadulaev, Rostislav Korst, Alexander Korotin, Vage Egiazarian, Andrey Filchenkov, Evgeny Burnaev
Offline Reinforcement Learning Expert Level Performance Control Problem

October 16, 2024

Off-dynamics Conditional Diffusion Planners
Wen Zheng Terence Ng, Jianda Chen, Tianwei Zhang
Offline Reinforcement Learning Conditional Diffusion Model Diffusion Planner Large Scale Real World Offline Dataset Inverse Dynamic

October 15, 2024

DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation
Jaehyun Park, Yunho Kim, Sejin Kim, Byung-Jun Lee, Sundong Kim
Offline Reinforcement Learning Adaptive Importance Sparse Reward Environment Adaptive Decision Making Implicit Q Learning

Offline Reinforcement Learning

Papers

Enhancing Decision Transformer with Diffusion-Based Trajectory Branch Generation

An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces

Iterative Batch Reinforcement Learning via Safe Diversified Model-based Policy Search

Doubly Mild Generalization for Offline Reinforcement Learning

Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learning

Streetwise Agents: Empowering Offline RL Policies to Outsmart Exogenous Stochastic Disturbances in RTC

Real-World Offline Reinforcement Learning from Vision Language Model Feedback

Constrained Latent Action Policies for Model-Based Offline Reinforcement Learning

Hypercube Policy Regularization Framework for Offline Reinforcement Learning

Prompt Tuning with Diffusion for Few-Shot Pre-trained Policy Generalization

Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers

Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation

Offline Reinforcement Learning With Combinatorial Action Spaces

Q-Distribution guided Q-learning for offline reinforcement learning: Uncertainty penalized Q-value via consistency model

Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression

Scalable Offline Reinforcement Learning for Mean Field Games

Offline reinforcement learning for job-shop scheduling problems

Rethinking Optimal Transport in Offline Reinforcement Learning

Off-dynamics Conditional Diffusion Planners

DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation