Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to train agents using pre-collected data, eliminating the need for costly and potentially risky online interactions with the environment. Current research focuses on addressing challenges like distributional shift (mismatch between training and target data) and improving generalization across diverse tasks, employing model architectures such as transformers, convolutional networks, and diffusion models, along with algorithms like conservative Q-learning and decision transformers. These advancements are significant for deploying RL in real-world applications where online learning is impractical or unsafe, impacting fields ranging from robotics and healthcare to personalized recommendations and autonomous systems.

Papers

May 29, 2024

May 28, 2024

May 27, 2024

May 23, 2024

May 22, 2024

Offline RL via Feature-Occupancy Gradient Ascent
Gergely Neu, Nneka Okolo
Markov Decision Process Offline Reinforcement Learning Linear Programming Gradient Ascent

May 20, 2024

Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?
Yang Dai, Oubo Ma, Longfei Zhang, Xingxing Liang, Shengchao Hu, Mengzhu Wang, Shouling Ji, Jincai Huang, Li Shen
Offline Reinforcement Learning Trajectory Optimization Decision Transformer Mamba Based E RNN

May 18, 2024

Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses
Thanh Nguyen, Tung M. Luu, Tri Ton, Chang D. Yoo
Adversarial Attack Offline Reinforcement Learning Robust Policy Adversarial DEfense

May 14, 2024

Reinformer: Max-Return Sequence Modeling for Offline RL
Zifeng Zhuang, Dengyun Peng, Jinxin Liu, Ziqi Zhang, Donglin Wang
Offline Reinforcement Learning Sequence Modeling Sequence Model

May 9, 2024

An Overview of Machine Learning-Enabled Optimization for Reconfigurable Intelligent Surfaces-Aided 6G Networks: From Reinforcement Learning to Large Language Models
Hao Zhou, Chengming Hu, Xue Liu
Reinforcement Learning Multi Agent Reinforcement Learning Offline Reinforcement Learning Deep Q Learning G Network Reconfigurable Intelligent Learning Based Optimization

May 8, 2024

Offline Model-Based Optimization via Policy-Guided Gradient Search
Yassine Chemingui, Aryan Deshwal, Trong Nghia Hoang, Janardhan Rao Doppa
Offline Reinforcement Learning Policy Gradient Surrogate Model Action Free Offline Learned Optimizers Model Based Optimization Offline Optimization

May 7, 2024

Improving Offline Reinforcement Learning with Inaccurate Simulators
Yiwen Hou, Haoyuan Sun, Jinming Ma, Feng Wu
Generative Adversarial Network Offline Reinforcement Learning Accurate Simulation Offline Dataset

May 6, 2024

Out-of-Distribution Adaptation in Offline RL: Counterfactual Reasoning via Causal Normalizing Flows
Minjae Cho, Jonathan P. How, Chuangchuang Sun
Offline Reinforcement Learning Policy Learning Counterfactual Reasoning Offline Policy Evaluation Causal Reinforcement Learning Graph Out of Distribution Adaptation Graphical Normalizing Flow

April 30, 2024

Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learning
Chenjia Bai, Lingxiao Wang, Jianye Hao, Zhuoran Yang, Bin Zhao, Zhen Wang, Xuelong Li
Offline Reinforcement Learning Value Iteration Task Specific Policy Offline Dataset

Offline Reinforcement Learning

Papers

Long-Horizon Rollout via Dynamics Diffusion for Offline Reinforcement Learning

Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning

Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination

HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning

Q-value Regularized Transformer for Offline Reinforcement Learning

Any-step Dynamics Model Improves Future Predictions for Online and Offline Reinforcement Learning

GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learning

Trajectory Data Suffices for Statistically Efficient Learning in Offline RL with Linear $q^\pi$-Realizability and Concentrability

State-Constrained Offline Reinforcement Learning

Offline Reinforcement Learning from Datasets with Structured Non-Stationarity

Exclusively Penalized Q-learning for Offline Reinforcement Learning

Offline RL via Feature-Occupancy Gradient Ascent

Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?

Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses

Reinformer: Max-Return Sequence Modeling for Offline RL

An Overview of Machine Learning-Enabled Optimization for Reconfigurable Intelligent Surfaces-Aided 6G Networks: From Reinforcement Learning to Large Language Models

Offline Model-Based Optimization via Policy-Guided Gradient Search

Improving Offline Reinforcement Learning with Inaccurate Simulators

Out-of-Distribution Adaptation in Offline RL: Counterfactual Reasoning via Causal Normalizing Flows

Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learning