Continual Learning

Follow this topic Ask about this topic

Momentum

26 papers in the last four weeks, up 189% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 241

All topics

Realistic Continual Learning Approach using Pre-trained Models
Apr 11, 2024Nadia Nasri, Carlos Gutiérrez-Álvarez, Sergio Lafuente-Arroyo +2Replay-Based Continual Learning Continual Learning
University of Alcala, Department of Signal Theory and Communications, GRAM Research group, Alcala de Henares, 28805, Spain · Univesity of Alicante, Department of Computer Science and Artificial Intelligence, San Vicente del Raspeig, 03690, Spain

Apr 11, 2024, cs.LGJ/K move · Enter open · S save

Realistic Continual Learning Approach using Pre-trained Models

Nadia Nasri, Carlos Gutiérrez-Álvarez, Sergio Lafuente-Arroyo, Saturnino Maldonado-Bascón+1

University of Alcala, Department of Signal Theory and Communications, GRAM Research group, Alcala de Henares, 28805, Spain · Univesity of Alicante, Department of Computer Science and Artificial Intelligence, San Vicente del Raspeig, 03690, Spain

Replay-Based Continual Learning Continual Learning Class-Incremental Learning Catastrophic Forgetting Pretraining Parameter-Efficient Adaptation+1

Open paper AskPDF arXiv

Continual learning (CL) evaluates adaptability in learning solutions to retain knowledge. Our research addresses the challenge of catastrophic forgetting, where models lose proficiency in previously learned tasks as they acquire new ones. While numerous solutions have been proposed, existing experimental setups often rely on idealized class-incremental learning scenarios. We introduce Realistic Continual Learning (RealCL), a novel CL paradigm where class distributions across tasks are random. We also present CLARE (Continual Learning Approach with pRE-trained models for RealCL scenarios), a pre-trained model-based solution designed to integrate new knowledge while preserving past learning. Our contributions include pioneering RealCL as a generalization of traditional CL setups, proposing CLARE as an adaptable approach for RealCL tasks, and conducting extensive experiments demonstrating its effectiveness across various RealCL scenarios. Notably, CLARE outperforms existing models on RealCL benchmarks, highlighting its versatility in unpredictable learning environments. Code to reproduce all our experiments can be found at https://github.com/gramuah/clare.

Momentum

Related topics

Latest papers 241

Realistic Continual Learning Approach using Pre-trained Models