Continual Reinforcement Learning

Continual reinforcement learning (CRL) focuses on training agents that can continuously adapt to a sequence of changing tasks without forgetting previously acquired skills. Current research emphasizes improving data efficiency through techniques like data augmentation and experience replay, as well as developing robust model architectures that mitigate catastrophic forgetting, including those based on world models, hierarchical structures, and neural ensembles. These advancements are crucial for deploying reinforcement learning agents in real-world scenarios, such as robotics, autonomous driving, and dynamic communication networks, where continuous adaptation and lifelong learning are essential.

Papers

November 5, 2024

Hierarchical Orchestra of Policies
Thomas P Cannon, Özgür Simsek
Catastrophic Forgetting Part Whole Hierarchy Prior Policy Continual Reinforcement Learning Lifelong Reinforcement Learning Co Orchestration Sequential Task

October 14, 2024

Continual Deep Reinforcement Learning to Prevent Catastrophic Forgetting in Jamming Mitigation
Kemal Davaslioglu, Sastry Kompella, Tugba Erpek, Yalin E. Sagduyu
Deep Reinforcement Learning Catastrophic Forgetting Continual Reinforcement Learning Anti Jamming

August 24, 2024

Data Augmentation for Continual RL via Adversarial Gradient Episodic Memory
Sihao Wu, Xingyu Zhao, Xiaowei Huang
Reinforcement Learning Data Augmentation Continual Reinforcement Learning Gradient Episodic Memory

June 6, 2024

HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learning
Quentin Delfosse, Jannis Blüml, Bjarne Gregori, Kristian Kersting
Reinforcement Learning Robust Version Artificial Agent Continual Reinforcement Learning Learning Environment

May 20, 2024

Continual Deep Reinforcement Learning for Decentralized Satellite Routing
Federico Lozano-Cuadra, Beatriz Soret, Israel Leyva-Mayorga, Petar Popovski
Continual LEArning Multi Agent Multi Agent Deep Reinforcement Learning Continual Reinforcement Learning Autonomous Decision Making

April 30, 2024

Continual Model-based Reinforcement Learning for Data Efficient Wireless Network Optimisation
Cengis Hasan, Alexandros Agapitos, David Lynch, Alberto Castagna, Giorgio Cruciata, Hao Wang, Aleksandar Milenovic
Efficient Optimization Worst Case User Throughput Continual Reinforcement Learning Cell Library Characterization Wireless Network Optimization

April 19, 2024

Single-Task Continual Offline Reinforcement Learning
Sibo Gai, Donglin Wang
Offline Reinforcement Learning Continual Learning Continual Reinforcement Learning Offline Visual Representation Learning

April 2, 2024

K-percent Evaluation for Lifelong RL
Golnaz Mesbahi, Parham Mohammad Panahi, Olya Mastikhina, Martha White, Adam White
Deep RL DQN Agent Continual Reinforcement Learning Lifelong Reinforcement Learning

March 8, 2024

Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learning
Hongjoon Ahn, Jinu Hyeon, Youngmin Oh, Bosun Hwang, Taesup Moon
Complete Recipe Continual Reinforcement Learning RL Agent Negative Transfer Global Reset Feature Plasticity Loss Task Engagement

January 30, 2024

Augmenting Replay in World Models for Continual Reinforcement Learning
Luke Yang, Levin Kuhlmann, Gideon Kowadlo
World Model Model Free Continual Reinforcement Learning RePLAy Loss Efficient Generated Object Replay

January 25, 2024

Hierarchical Continual Reinforcement Learning via Large Language Model
Chaofan Pan, Xin Yang, Hao Wang, Wei Wei, Tianrui Li
Large Language Model Reinforcement Learning Hierarchical Reinforcement Learning Continual Reinforcement Learning Policy Making

December 18, 2023

Prediction and Control in Continual Reinforcement Learning
Nishanth Anand, Doina Precup
Continual LEArning Human Prediction External Control Value Function Continual Reinforcement Learning Transient Dynamic

December 2, 2023

Harnessing Discrete Representations For Continual Reinforcement Learning
Edan Meyer, Adam White, Marlos C. Machado
Reinforcement Learning Discrete Representation Continual Reinforcement Learning Continuous Representation Categorical Representation

November 20, 2023

Replay-enhanced Continual Reinforcement Learning
Tiantian Zhang, Kevin Zehua Shen, Zichuan Lin, Bo Yuan, Xueqian Wang, Xiu Li, Deheng Ye
Continual LEArning Catastrophic Forgetting Continual Reinforcement Learning Replay Based State of the Art Continual

November 12, 2023

Towards Continual Reinforcement Learning for Quadruped Robots
Giovanni Minelli, Vassilis Vassiliades
Non Humanoid Robot Robot Person Quadruped Robot Continual Reinforcement Learning Continual Learning Scenario Skill Transfer

September 25, 2023

Continual Driving Policy Optimization with Closed-Loop Individualized Curricula
Haoyi Niu, Yizhou Xu, Xingjian Jiang, Jianming Hu
Autonomous Vehicle Naturalistic Driving Continual Reinforcement Learning Safety Critical Scenario Driving Scenario Generation

September 18, 2023

Self-Sustaining Multiple Access with Continual Deep Reinforcement Learning for Dynamic Metaverse Applications
Hamidreza Mazandarani, Masoud Shokrnezhad, Tarik Taleb, Richard Li
Continual LEArning Continual Reinforcement Learning Multi Channel Multiple Access Double Deep Q Learning Adaptive AI Metaverse System

July 20, 2023

A Definition of Continual Reinforcement Learning
David Abel, André Barreto, Benjamin Van Roy, Doina Precup, Hado van Hasselt, Satinder Singh
Lifelong Learning Definition Defining Redefinition Continual Reinforcement Learning Reinforcement Learning Problem Visual Reinforcement

March 13, 2023

Loss of Plasticity in Continual Deep Reinforcement Learning
Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, Marlos C. Machado
Continual LEArning Interest Loss Plasticity Rule Continual Reinforcement Learning Deep Reinforcement Learning Agent Value Based Deep Reinforcement Learning

December 31, 2022

Self-Activating Neural Ensembles for Continual Reinforcement Learning
Sam Powers, Eliot Xing, Abhinav Gupta
Intelligent Agent Skill Learning Continual Reinforcement Learning Task Boundary Neuronal Ensemble