Online Reinforcement Learning

Online reinforcement learning (RL) focuses on training agents to make optimal decisions in dynamic environments through continuous interaction and feedback. Current research emphasizes improving sample efficiency, particularly through offline data pre-training and techniques like prioritized experience replay and ensemble methods, as well as exploring novel model architectures such as Kolmogorov-Arnold Networks. These advancements aim to address challenges like reward sparsity, distribution shifts between offline and online data, and the need for safe and reliable learning in high-stakes applications such as robotics and healthcare, ultimately leading to more robust and efficient RL agents.

Papers

February 4, 2023

Online Reinforcement Learning in Non-Stationary Context-Driven Environments
Pouya Hamadanian, Arash Nasr-Esfahany, Malte Schwarzkopf, Siddartha Sen, Mohammad Alizadeh
Policy OpTimization Context Aware Non Stationary Online Reinforcement Learning Constrained Policy Optimization

January 30, 2023

Guiding Online Reinforcement Learning with Action-Free Offline Pretraining
Deyao Zhu, Yuhui Wang, Jürgen Schmidhuber, Mohamed Elhoseiny
Decision Transformer Online Reinforcement Learning Action Free Offline

December 30, 2022

Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks
Trevor Ablett, Bryan Chan, Jonathan Kelly
LeArning Abstract Imitation Learning Multi Task Environment Exploration Online Reinforcement Learning State Action Pair Adversarial Imitation Learning Adversarial Imitation

December 7, 2022

Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemble
Chao Li
Reinforcement Learning Deep Reinforcement Learning Noisy Demonstration Online Reinforcement Learning Expert Demonstration Policy Constraint

November 14, 2022

Towards Data-Driven Offline Simulations for Online Reinforcement Learning
Shengpu Tang, Felipe Vieira Frujeri, Dipendra Misra, Alex Lamb, John Langford, Paul Mineiro, Sebastian Kochman
Reinforcement Learning Online Reinforcement Learning Offline Policy Evaluation Adaptive Agent

November 9, 2022

Leveraging Offline Data in Online Reinforcement Learning
Andrew Wagenmaker, Aldo Pacchiano
Reinforcement Learning Offline Reinforcement Learning Optimal Policy Online Reinforcement Learning Offline Data Offline Dataset

October 13, 2022

Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecture
Hong Liu, Zhijian Ou, Yi Huang, Junlan Feng
Task Oriented Online Reinforcement Learning Dialog System User Simulator Simple Generative Dialog History

October 12, 2022

Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems
Felix Feit, Andreas Metzger, Klaus Pohl
Reinforcement Learning Online Reinforcement Learning Explainable Reinforcement Learning Self Adaptive System

October 9, 2022

The Role of Coverage in Online Reinforcement Learning
Tengyang Xie, Dylan J. Foster, Yu Bai, Nan Jiang, Sham M. Kakade
Offline Reinforcement Learning Integral Role Sample Complexity Online Reinforcement Learning Sample Efficient Higher Coverage Rate

September 22, 2022

Bypassing the Simulation-to-reality Gap: Online Reinforcement Learning using a Supervisor
Benjamin David Evans, Johannes Betz, Hongrui Zheng, Herman A. Engelbrecht, Rahul Mangharam, Hendrik W. Jordaan
Deep Reinforcement Learning Pre Training Online Reinforcement Learning Reality Gap Supervisory Performance

August 31, 2022

A stabilizing reinforcement learning approach for sampled systems with partially unknown models
Lukas Beckenbach, Pavel Osinenko, Stefan Streif
Reinforcement Learning System Description Online Reinforcement Learning Reinforcement Learning Approach Adaptive Cruise Control Closed Loop Stability

August 24, 2022

Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Path
Muhammad Aneeq uz Zaman, Alec Koppel, Sujay Bhatt, Tamer Başar
Mean Field Online Reinforcement Learning Mean Field Game Sample Path

August 15, 2022

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Online Reinforcement Learning Reward Design Oral Self Care

July 29, 2022

Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning
Shuang Qiu, Lingxiao Wang, Chenjia Bai, Zhuoran Yang, Zhaoran Wang
Reinforcement Learning Contrastive Learning Online Reinforcement Learning Contrastive Self Supervised Learning Efficient Policy Learning Upper Confidence Bound Confidence Aware Contrastive

July 25, 2022

Online Reinforcement Learning for Periodic MDP
Ayush Aniket, Arpan Chattopadhyay
Reinforcement Learning Markov Decision Process Online Reinforcement Learning Calibrated Confidence Piecewise Deterministic Markov Process

July 18, 2022

A framework for online, stabilizing reinforcement learning
Grigory Yaremenko, Georgiy Malaniya, Pavel Osinenko
Reinforcement Learning New Framework Mobile Robot Online Service Online Reinforcement Learning Stability Constraint

July 12, 2022

PAC Reinforcement Learning for Predictive State Representations
Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
Markov Decision Process Online Reinforcement Learning C POMDPs Thermodynamic Observables Predictive State Representation

June 10, 2022

Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?
Xiang Li, Jinghuan Shang, Srijan Das, Michael S. Ryoo
Reinforcement Learning Self Supervised Learning Tetromino Pixel Online Reinforcement Learning Self Supervised Loss Contrastive Reinforcement Learning

June 8, 2022

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines
Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Reinforcement Learning Reinforcement Learning Algorithm Online Reinforcement Learning State Aware Guideline Digital Intervention

May 17, 2022

Moral reinforcement learning using actual causation
Tue Herlau
Online Reinforcement Learning Causal Direction Actual Cause Moral Judgment Reinforcement Learning System

Online Reinforcement Learning

Papers

Online Reinforcement Learning in Non-Stationary Context-Driven Environments

Guiding Online Reinforcement Learning with Action-Free Offline Pretraining

Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks

Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemble

Towards Data-Driven Offline Simulations for Online Reinforcement Learning

Leveraging Offline Data in Online Reinforcement Learning

Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecture

Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems

The Role of Coverage in Online Reinforcement Learning

Bypassing the Simulation-to-reality Gap: Online Reinforcement Learning using a Supervisor

A stabilizing reinforcement learning approach for sampled systems with partially unknown models

Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Path

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care

Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning

Online Reinforcement Learning for Periodic MDP

A framework for online, stabilizing reinforcement learning

PAC Reinforcement Learning for Predictive State Representations

Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines

Moral reinforcement learning using actual causation