Offline Learning

Offline learning in reinforcement learning (RL) aims to train effective policies using only pre-collected datasets, avoiding the need for costly online interaction with the environment. Current research focuses on improving generalization to unseen tasks and environments, addressing distribution shifts between training and deployment data, and developing more efficient algorithms, including those based on successor representations, decision transformers, and various forms of Q-learning with conservatism. These advancements are crucial for deploying RL in real-world scenarios where online learning is impractical or risky, with applications ranging from robotics and game playing to wireless network optimization and healthcare.

Papers

October 17, 2024

Preference Aligned Diffusion Planner for Quadrupedal Locomotion Control
Xinyi Yuan, Zhiwei Shang, Zifan Wang, Chenkai Wang, Zhao Shan, Zhenchao Qi, Meixin Zhu, Chenjia Bai, Xuelong Li
Quadrupedal Locomotion Legged Robot Locomotion Diffusion Planner Offline Learning

May 12, 2024

Ensemble Successor Representations for Task Generalization in Offline-to-Online Reinforcement Learning
Changhong Wang, Xudong Yu, Chenjia Bai, Qiaosheng Zhang, Zhen Wang
Reinforcement Learning Online Adaptation Task Generalization Successor Representation Offline Learning Offline to Online Reinforcement Learning

February 28, 2024

ELA: Exploited Level Augmentation for Offline Learning in Zero-Sum Games
Shiqi Lei, Kanghoon Lee, Linjing Li, Jinkyoo Park, Jiachen Li
Imitation Learning Offline Reinforcement Learning Zero Sum Game Offline Learning

January 16, 2024

Solving Continual Offline Reinforcement Learning with Decision Transformer
Kaixin Huang, Li Shen, Chen Zhao, Chun Yuan, Dacheng Tao
Offline Reinforcement Learning Decision Transformer Offline Learning Offline RL Algorithm Continual Offline Reinforcement Learning

December 10, 2023

The Generalization Gap in Offline Reinforcement Learning
Ishita Mediratta, Qingfei You, Minqi Jiang, Roberta Raileanu
Strong Generalization Offline Reinforcement Learning Online Reinforcement Learning Generalization Gap Offline Learning

June 14, 2023

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
Nikhil Vyas, Depen Morwani, Rosie Zhao, Gal Kaplun, Sham Kakade, Boaz Barak
Deep Learning Stochastic Gradient Descent Online Learning Implicit Bias Offline Learning Gradient Step Rating Meaningfulness Multi Epoch Training

May 15, 2023

An Offline Time-aware Apprenticeship Learning Framework for Evolving Reward Functions
Xi Yang, Ge Gao, Min Chi
Reward Function Expectation Maximization Future Trajectory Offline Learning Apprenticeship Learning

March 16, 2023

Goal-conditioned Offline Reinforcement Learning through State Space Partitioning
Mianchu Wang, Yue Jin, Giovanni Montana
Offline Reinforcement Learning Goal Conditioned Policy Policy Improvement Offline Learning Offline Goal Conditioned Reinforcement Learning

February 24, 2023

Multi-Agent Reinforcement Learning with Common Policy for Antenna Tilt Optimization
Adriano Mendo, Jose Outes-Carnero, Yak Ng-Molina, Juan Ramiro-Moreno
Multi Agent Reinforcement Learning Wireless Network Diverse Policy Network Simulation Offline Learning Antenna Tilt

June 9, 2022

Mildly Conservative Q-Learning for Offline Reinforcement Learning
Jiafei Lyu, Xiaoteng Ma, Xiu Li, Zongqing Lu
Strong Generalization Offline Reinforcement Learning Conservative Q Learning Offline Learning

May 20, 2022

Towards biologically plausible Dreaming and Planning in recurrent spiking networks
Cristiano Capone, Pier Stanislao Paolucci
Reinforcement Learning Task Planning Network Programming Neuromorphic Hardware Dream Report Offline Learning

May 13, 2022

Tighter Regret Analysis and Optimization of Online Federated Learning
Dohyeok Kwon, Jonghwan Park, Songnam Hong
Optimization Purpose Sublinear Regret Regret Analysis Communication Efficient Cumulative Regret Online Gradient Descent Offline Learning

March 28, 2022

Offline-Online Learning of Deformation Model for Cable Manipulation with Graph Neural Networks
Changhao Wang, Yuyou Zhang, Xiang Zhang, Zheng Wu, Xinghao Zhu, Shiyu Jin, Te Tang, Masayoshi Tomizuka
Graph Neural Network Dynamic Model Robot Policy Deformable Linear Object Model Predictive Controller Offline Learning Deformation Model Large Deformation Cable Manipulation

March 11, 2022

Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism
Ming Yin, Yaqi Duan, Mengdi Wang, Yu-Xiang Wang
Offline Reinforcement Learning Value Iteration Variance Information Linear Representation Offline Learning

February 23, 2022

Reinforcement Learning in Practice: Opportunities and Challenges
Yuxi Li
Reinforcement Learning Machine Learning Technical Challenge Recommender System Emerging Opportunity Practice Mode Human Like RL Offline Learning

February 12, 2022

Coupling Online-Offline Learning for Multi-distributional Data Streams
Zhilin Zhao, Longbing Cao, Yuanyu Wan
Online Optimization Offline Learning Action on the Fly

November 11, 2021

Offline Contextual Bandits for Wireless Network Optimization
Miguel Suau, Alexandros Agapitos, David Lynch, Derek Farrell, Mingqi Zhou, Aleksandar Milenovic
Artificial Intelligence Real World Offline Learning Offline Contextual Bandit Cellular Traffic Wireless Network Optimization