Lifelong Reinforcement Learning

Lifelong reinforcement learning (LRL) focuses on developing agents capable of continuously learning and adapting to new tasks throughout their operational lifetime, without catastrophic forgetting of previously acquired skills. Current research emphasizes improving sample efficiency and robustness to environmental changes through techniques like selective experience replay, parameter-free optimization methods, and biologically-inspired architectures incorporating neuromodulation or modularity. These advancements are crucial for deploying reinforcement learning agents in real-world scenarios characterized by non-stationary environments and limited training data, impacting fields such as robotics, autonomous driving, and medical imaging.

Papers

November 5, 2024

Hierarchical Orchestra of Policies
Thomas P Cannon, Özgür Simsek
Catastrophic Forgetting Part Whole Hierarchy Prior Policy Continual Reinforcement Learning Lifelong Reinforcement Learning Co Orchestration Sequential Task

November 1, 2024

Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayesian Theory
Zhi Zhang, Chris Chow, Yasi Zhang, Yanchao Sun, Haochen Zhang, Eric Hanchen Jiang, Han Liu, Furong Huang, Yuchen Cui, Oscar Hernan Madrid Padilla
PAC Bayesian Single Agent Reinforcement Learning Statistical Guarantee Lifelong Reinforcement Learning Policy Distribution

October 3, 2024

AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs
Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, Chaowei Xiao
Jailbreak Attack Attack Success Rate Lifelong Reinforcement Learning

September 5, 2024

CHIRPs: Change-Induced Regret Proxy metrics for Lifelong Reinforcement Learning
John Birkbeck, Adam Sobey, Federico Cerutti, Katherine Heseltine Hurley Flynn, Timothy J. Norman
Reinforcement Learning Agent Agent Performance Lifelong Reinforcement Learning Bell and Whistle Static Regret

August 15, 2024

Lifelong Reinforcement Learning via Neuromodulation
Sebastian Lee, Samuel Liebana Garcia, Claudia Clopath, Will Dabney
Reinforcement Learning Adaptive Learning Neuroscience Research Lifelong Reinforcement Learning Non Stationary Multi Armed Bandit

May 29, 2024

Statistical Context Detection for Deep Lifelong Reinforcement Learning
Jeffery Dick, Saptarshi Nath, Christos Peridis, Eseoghene Benjamin, Soheil Kolouri, Andrea Soltoggio
Context Learning Lifelong Reinforcement Learning Latent Action Task Recognition

May 26, 2024

Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning
Aneesh Muppidi, Zhiyu Zhang, Heng Yang
Online Convex Optimization Hyperparameter Selection Lifelong Reinforcement Learning Pace Adaptive Parameter Free

April 26, 2024

Beyond Imitation: A Life-long Policy Learning Framework for Path Tracking Control of Autonomous Driving
C. Gong, C. Lu, Z. Li, Z. Liu, J. Gong, X. Chen
Autonomous Driving Imitation Learning Learning Based Feature Imitation Lifelong Reinforcement Learning Path Following Control

April 2, 2024

K-percent Evaluation for Lifelong RL
Golnaz Mesbahi, Parham Mohammad Panahi, Olya Mastikhina, Martha White, Adam White
Deep RL DQN Agent Continual Reinforcement Learning Lifelong Reinforcement Learning

June 8, 2023

A framework for dynamically training and adapting deep reinforcement learning models to different, low-compute, and continuously changing radiology deployment environments
Guangyao Zheng, Shuhao Lai, Vladimir Braverman, Michael A. Jacobs, Vishwa S. Parekh
Deep Reinforcement Learning Training Data New Framework Medical Imaging COreset Selection Lifelong Reinforcement Learning Low Compute

May 31, 2023

Multi-environment lifelong deep reinforcement learning for medical imaging
Guangyao Zheng, Shuhao Lai, Vladimir Braverman, Michael A. Jacobs, Vishwa S. Parekh
Deep Reinforcement Learning Medical Imaging Lifelong Reinforcement Learning

May 18, 2023

Sharing Lifelong Reinforcement Learning Knowledge via Modulating Masks
Saptarshi Nath, Christos Peridis, Eseoghene Ben-Iwhiwhu, Xinran Liu, Shirin Dora, Cong Liu, Soheil Kolouri, Andrea Soltoggio
Lifelong Learning Multiple Agent Single Agent Lifelong Reinforcement Learning

February 22, 2023

Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging
Guangyao Zheng, Samson Zhou, Vladimir Braverman, Michael A. Jacobs, Vishwa S. Parekh
Medical Imaging Experience Replay COreset Selection Lifelong Reinforcement Learning Coreset Size

January 24, 2023

Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learning
Safa Alver, Doina Precup
Model Based Reinforcement Learning Learning Model Lifelong Reinforcement Learning Robust Planning

January 21, 2023

The configurable tree graph (CT-graph): measurable problems in partially observable and distal reward environments for lifelong reinforcement learning
Andrea Soltoggio, Eseoghene Ben-Iwhiwhu, Christos Peridis, Pawel Ladosz, Jeffery Dick, Praveen K. Pilly, Soheil Kolouri
Reward Function Sparse Reward Sparse Reward Environment Lifelong Reinforcement Learning Memory Less Agent Solved Problem Hierarchical Reward

December 21, 2022

Lifelong Reinforcement Learning with Modulating Masks
Eseoghene Ben-Iwhiwhu, Saptarshi Nath, Praveen K. Pilly, Soheil Kolouri, Andrea Soltoggio
Catastrophic Forgetting Lifelong Learning Lifelong Reinforcement Learning Biological Learning

December 8, 2022

System Design for an Integrated Lifelong Reinforcement Learning Agent for Real-Time Strategy Games
Indranil Sur, Zachary Daniels, Abrar Rahman, Kamil Faber, Gianmarco J. Gallardo, Tyler L. Hayes, Cameron E. Taylor, Mustafa Burak Gurbuz, James Smith, Sahana Joshi, Nathalie Japkowicz, Michael Baron, Zsolt Kira, Christopher Kanan, Roberto Corizzo, Ajay Divakaran, Michael Piacentino, Jesse Hostetler, Aswin Raghavan
Continual Learning Modular Framework Lifelong Reinforcement Learning Real Time Strategy Lifelong Machine Learning Lifelong Learning System

November 30, 2022

General policy mapping: online continual reinforcement learning inspired on the insect brain
Angel Yanguas-Gil, Sandeep Madireddy
Reinforcement Learning Algorithm Efficient Reinforcement Learning Continual Reinforcement Learning Lifelong Reinforcement Learning Policy Generalization Insect Machine Hybrid System Backward Transfer

October 20, 2022

Model-based Lifelong Reinforcement Learning with Bayesian Exploration
Haotian Fu, Shangqun Yu, Michael Littman, George Konidaris
Lifelong Reinforcement Learning Based Exploration

August 9, 2022

Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2
Zachary Daniels, Aswin Raghavan, Jesse Hostetler, Abrar Rahman, Indranil Sur, Michael Piacentino, Ajay Divakaran
Application Proficiency Deep RL Generative Replay Deep Reinforcement Learning Agent Lifelong Reinforcement Learning StarCraft II