Continual Reinforcement Learning

Continual reinforcement learning (CRL) focuses on training agents that can continuously adapt to a sequence of changing tasks without forgetting previously acquired skills. Current research emphasizes improving data efficiency through techniques like data augmentation and experience replay, as well as developing robust model architectures that mitigate catastrophic forgetting, including those based on world models, hierarchical structures, and neural ensembles. These advancements are crucial for deploying reinforcement learning agents in real-world scenarios, such as robotics, autonomous driving, and dynamic communication networks, where continuous adaptation and lifelong learning are essential.

Papers

September 18, 2023

Self-Sustaining Multiple Access with Continual Deep Reinforcement Learning for Dynamic Metaverse Applications
Hamidreza Mazandarani, Masoud Shokrnezhad, Tarik Taleb, Richard Li
Continual LEArning Continual Reinforcement Learning Multi Channel Multiple Access Double Deep Q Learning Metaverse System Adaptive AI

July 20, 2023

A Definition of Continual Reinforcement Learning
David Abel, André Barreto, Benjamin Van Roy, Doina Precup, Hado van Hasselt, Satinder Singh
Lifelong Learning Definition Defining Redefinition Continual Reinforcement Learning Reinforcement Learning Problem Visual Reinforcement

March 13, 2023

Loss of Plasticity in Continual Deep Reinforcement Learning
Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, Marlos C. Machado
Continual LEArning Interest Loss Plasticity Rule Continual Reinforcement Learning Deep Reinforcement Learning Agent Value Based Deep Reinforcement Learning

December 31, 2022

Self-Activating Neural Ensembles for Continual Reinforcement Learning
Sam Powers, Eliot Xing, Abhinav Gupta
Intelligent Agent Skill Learning Continual Reinforcement Learning Task Boundary Neuronal Ensemble

November 30, 2022

General policy mapping: online continual reinforcement learning inspired on the insect brain
Angel Yanguas-Gil, Sandeep Madireddy
Reinforcement Learning Algorithm Efficient Reinforcement Learning Continual Reinforcement Learning Lifelong Reinforcement Learning Policy Generalization Insect Machine Hybrid System Backward Transfer

November 29, 2022

The Effectiveness of World Models for Continual Reinforcement Learning
Samuel Kessler, Mateusz Ostaszewski, Michał Bortkiewicz, Mateusz Żarski, Maciej Wołczyk, Jack Parker-Holder, Stephen J. Roberts, Piotr Miłoś
Continual LEArning World Model Experience Replay Efficient Reinforcement Learning Continual Reinforcement Learning Binary Forward Exploration Agnostic Continual Learning

November 14, 2022

Hierarchically Structured Task-Agnostic Continual Learning
Heinke Hihn, Daniel A. Braun
Reinforcement Learning Continual LEArning Continual Reinforcement Learning Continual Learning Algorithm Agnostic Continual Learning Continual Learning Paradigm

October 21, 2022

Continual Vision-based Reinforcement Learning with Group Symmetries
Shiqi Liu, Mengdi Xu, Piede Huang, Yongkang Liu, Kentaro Oguchi, Ding Zhao
Continual Reinforcement Learning RL Algorithm

September 28, 2022

Disentangling Transfer in Continual Reinforcement Learning
Maciej Wołczyk, Michał Zając, Razvan Pascanu, Łukasz Kuciński, Piotr Miłoś
Continual Learning Continual Learning Benchmark Continuous Environment Continual Reinforcement Learning

September 4, 2022

Reinforced Continual Learning for Graphs
Appan Rakaraddi, Siew Kei Lam, Mahardhika Pratama, Marcus De Carvalho
Graph Neural Network Continual LEArning Graph Drawing Continual Learning Continual Learning Benchmark Continual Reinforcement Learning

September 1, 2022

Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization
Tiantian Zhang, Zichuan Lin, Yuxing Wang, Deheng Ye, Qiang Fu, Wei Yang, Xueqian Wang, Bin Liang, Bo Yuan, Xiu Li
Dynamic Environment Deep RL Continual Reinforcement Learning Conditioned Policy

August 8, 2022

Continual Reinforcement Learning with TELLA
Neil Fendley, Cash Costello, Eric Nguyen, Gino Perrotta, Corey Lowman
Continual Learning Reinforcement Learning Agent Lifelong Learning Continual Reinforcement Learning Lifelong Machine Learning Learning Curriculum

August 1, 2022

Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewards
Yongle Luo, Yuxin Wang, Kun Dong, Qiang Zhang, Erkang Cheng, Zhiyong Sun, Bo Song
Reinforcement Learning Sparse Reward Continual Reinforcement Learning Guided Exploration Sequential Manipulation Hindsight Experience Replay Unsupervised Exploration

April 12, 2022

Dynamic Dialogue Policy for Continual Reinforcement Learning
Christian Geishauser, Carel van Niekerk, Nurul Lubis, Michael Heck, Hsien-Chin Lin, Shutong Feng, Milica Gašić
Continual LEArning Continual Reinforcement Learning Dialogue Policy Continual Learning Model Dialog Policy Learning

March 14, 2022

L2Explorer: A Lifelong Reinforcement Learning Assessment Environment
Erik C. Johnson, Eric Q. Nguyen, Blake Schreurs, Chigozie S. Ewulum, Chace Ashcraft, Neil M. Fendley, Megan M. Baker, Alexander New, Gautam K. Vallabha
Reinforcement Learning Lifelong Learning Continual Reinforcement Learning Continual Learning Algorithm Lifelong Reinforcement Learning Reinforcement Learning Solution

February 28, 2022

Avalanche RL: a Continual Reinforcement Learning Library
Nicolò Lucchesi, Antonio Carta, Vincenzo Lomonaco, Davide Bacciu
Continual Learning Continual Reinforcement Learning Reinforcement Learning Library Habitat Simulator

December 13, 2021

Continual Learning In Environments With Polynomial Mixing Times
Matthew Riemer, Sharath Chandra Raparthy, Ignacio Cases, Gopeshh Subbaraj, Maximilian Puelma Touzel, Irina Rish
Continual LEArning Environment Feature Continual Reinforcement Learning Mixing Time

December 10, 2021

Quantum Architecture Search via Continual Reinforcement Learning
Esther Ye, Samuel Yen-Chi Chen
Reinforcement Learning Quantum Circuit Deep Q Learning Continual Reinforcement Learning Quantum Architecture Search Quantum Architecture Quantum Circuit Synthesis