Incremental Learning

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 63% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 103

All topics

CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling
Oct 6, 2026Maoqi Liu, Quan Fang, Yufei HeContinual Learning for LLMs Low-Rank Adaptation
Beijing University of Posts and Telecommunications · National University of Singapore
HiER-BLS: A Hierarchy-Guided and Error-Correcting Robust Incremental Broad Learning System
Oct 5, 2026Gongli Zhang, C. L. Philip Chen, Zhulin LiuHierarchical Representation Learning Neural Network Robustness
Guangdong Provincial Key Laboratory of Computational AI Models and Cognitive Intelligence, the School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China, also with the Pazhou Lab, Guangzhou 510335, China, and also with the Engineering Research Center of the Ministry of Education on Health Intelligent Perception and Paralleled Digital-Human, Guangzhou 510641, China
Width Expansion as a Method for Class Incremental Learning
Sep 29, 2026A. L. S. Conde, Y. Elkhatib, C. M. RanieriClass-Incremental Learning Dynamic Neural Networks
Institute of Geosciences and Exact Sciences, São Paulo State University (UNESP), Av. 24-A, 1515, Rio Claro, 13506-692, SP, Brazil · School of Computing Science, University of Glasgow, Glasgow, G12 8QQ, Scotland, United Kingdom
Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detection
Sep 29, 2026Yuankun Xie, Xiaoxuan Guo, Xiaopeng Wang +3Audio Deepfake Detection Prompt Learning
The Hong Kong Polytechnic University, Hong Kong SAR, China · Communication University of China, Beijing, China · Beijing Institute of Technology, Beijing, China
Normative Loss Landscape Navigation: A Trajectory-Based Approach to Mitigating Forgetting in Incremental Learning
Sep 28, 2026Isabelle Aguilar, Zayn Andre Zainal, Luis Fernando Herbozo Contreras +2Class-Incremental Learning Incremental Learning
Re:Cognize -- Open-Set Comic Character Re-Identification
Sep 27, 2026Aaditya Baranwal, Madhav Kataria, Yogesh S Rawat +1Self-Training Open-Set Recognition
Institute of Artificial Intelligence, University of Central Florida, Orlando, FL, USA · University of Central Florida
A Lightweight Plastic-Memory Framework for Graph Few-Shot Class-Incremental Learning
Sep 22, 2026Zihan Mei, Zhili Qin, Tongze Zhang +3Class-Incremental Learning Few-Shot Learning
University of Electronic Science and Technology of China
Lightweight Ranking Heads: Accelerating Multi-Task Experimentation in Production Recommender Systems
Sep 21, 2026Sanjay Surendranath Girija, Aniruddh Nath, Li Wei +6Learning to Rank Multi-Task Learning
Google LLC Mountain View, CA, USA
Muon Can Outperform Dedicated Continual Learning Methods
Sep 21, 2026Sebastian George Sincari, Bogdan Alexandru Gheorghe, Antonio BarbalauContinual Learning Muon Optimizer
Faculty of Mathematics and Computer Science, University of Bucharest, Bucharest, Romania · Faculty of Mathematics and Computer Science University of Bucharest Bucharest, Romania · Bitdefender, Bucharest, Romania +1
MECAIL: Communication-Aware Incremental Learning for Object Detection with 14.6 KB Spatiotemporal Experts
Sep 21, 2026Matthias Neuwirth-Trapp, Maarten Bieshaar, Danda Paudel +3Intelligent Transportation Systems Mixture of Experts
ETH Zurich, Zurich, Switzerland. · Bosch Research, Hildesheim, Germany. · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria.
Uncertainty-Aware Continual Learning for Open-World Intent Discovery Under an evolving Label Space
Sep 15, 2026Pisante Aida, Formentin SimoneContinual Learning Incremental Learning
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy.
CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework
Sep 15, 2026Yunxiang Fu, Meng Lou, Zicheng Liao +1Class-Incremental Learning Continual Learning
School of Computing and Data Science, The University of Hong Kong · Hong Kong Generative AI Research and Development Center
Investigating catastrophic forgetting in sound event classification
Sep 12, 2026Riccardo Casciotti, Annamaria MesarosClass-Incremental Learning Incremental Learning
Signal Processing Research Center Tampere University Tampere, Finland
Learning Orthogonal Multi-Index Models Beyond Small Initialization: Incremental Learning, Competitive Dynamics and Symmetry
Sep 11, 2026Mo Zhou, Weihang Xu, Simon S. Du +1Shallow Neural Networks Orthogonal Representation Learning
University of Washington · Amazon, Inc.
Hyperbolic Geometry for Open-World Object Detection in Remote Sensing Imagery
Sep 10, 2026Wuzhou Li, Jiawei Zhou, Shenghang Wang +1Open-Vocabulary Object Detection Continual Learning
School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan 430072, China · Electronic Information School, Wuhan University, Wuhan 430072, China · Electrical and Computer Engineering, Ohio State University, Columbus, OH, USA +1
Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners
Sep 3, 2026David Milec, Spyridon Samothrakis, Michael Fairbank +1Gradient-Boosted Decision Trees Game-Playing Agents
Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic · School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom · Maastricht University, the Netherlands
A Sensor-Adaptive Incremental Learning Framework for Artifact Detection in Satellite Precipitation Data
Sep 1, 2026Andres F. Monsalve, Hernan A. Moreno, Christian D. KummerowRemote Sensing Incremental Learning
University of Texas at El Paso, El Paso, TX, USA · Colorado State University, Fort Collins, CO, USA
One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning
Aug 31, 2026Yunxiang Fu, Meng Lou, Yizhou YuClass-Incremental Learning Incremental Learning
School of Computing and Data Science The University of Hong Kong
Knowing Beyond the Known: Reinforced Knowledge Specification for Multi-Label Class-Incremental Learning
Aug 31, 2026Aoting Zhang, Dongbao Yang, Chang Liu +3Class-Incremental Learning Multi-Label Classification
Institute of Information Engineering, Chinese Academy of2026 Sciences, Beijing, 100085, China · School of Cyber Security, University of Chinese Academy ofAug Sciences, Beijing, 101408, China · Nankai University, Tianjin, 300350, China +1
In-Cell Learning: Language Models That Update Their Own Weights in Sequence Without Changing the File They Ship
Aug 21, 2026Zifeng Liu, Yaxin Lu, Xuanhan Wu +6LLM Quantization Knowledge Editing
Big Data and Artificial Intelligence Center, The Third Affiliated Hospital of Sun Yat-sen University · Institute for Frontier Interdisciplinary Research in Health Sciences and Technology, Sun Yat-sen University · Sun Yat-sen University Institute of Artificial Intelligence +7
Uniform Herding: Exemplar Replay with Representation Refresh
Aug 13, 2026Krishna SubediClass-Incremental Learning Replay-Based Continual Learning
Neryva
Incremental Evaluation and Training in Relational Deep Learning
Aug 13, 2026Jakub Peleška, Gustav ŠírConcept Drift Online Continual Learning
Czech Technical University in Prague
BPG: Balancing Plasticity and Generalization for Domain Incremental Learning
Aug 11, 2026Qiang Wang, Songlin Dong, Shaokun Wang +5Domain-Incremental Learning Incremental Learning
College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, China · Shenzhen University of Advanced Technology, Shenzhen, China · Harbin Institute of Technology, Shenzhen, China +1
NeuroGuard: Neural Gradient Update Aware of Representation Damage
Aug 8, 2026Taigo Sakai, Kazuhito HottaClass-Incremental Learning Adaptive Loss Weighting
Meijo University, 1-501 Shiogamaguchi, Tempaku-ku, Nagoya 468-8502, Japan
MAGIC-SSCIL: Manifold Anchoring and Geometric Incremental Calibration for Semi-Supervised Class Incremental Learning
Aug 5, 2026Yousef Abdi, Mohammad Asadpour, Yousef SeyfariClass-Incremental Learning Incremental Learning
Faculty of Electrical and Computer Engineering, Universityof Tabriz, Tabriz, Iran · Faculty of Engineering, University of Maragheh, Maragheh,Iran
NeuMoSync: End-to-End Neuromodulatory Control for Plasticity and Adaptability in Continual Learning
Aug 5, 2026Seyed Roozbeh Razavi Rohani, Khashayar Khajavi, Wesley Chung +2Class-Incremental Learning Dynamic Neural Networks
Department of Computer Science Simon Fraser University · Department of Computer Science McGill Univeristy, Mila - Quebec AI Institute
Plasticity of Growing and Elastic Neural Networks in Online Continual Learning
Aug 2, 2026Jeong Min Kong, Richard S. SuttonLoss of Plasticity Dynamic Neural Networks
Department of Electrical and Computer Engineering University of California, Los Angeles (UCLA) · Department of Computing Science University of Alberta Alberta Machine Intelligence Institute (Amii)
AdaHAT: Adaptive Hard Attention to the Task in Task-Incremental Learning
Aug 2, 2026Pengxiang Wang, Hongbo Bo, Jun Hong +2Stability-Plasticity Dilemma Incremental Learning
School of Mathematical Sciences, Peking University, Beijing, China · Population Health Sciences Institute, Newcastle University, Newcastle, UK · School of Engineering Mathematics and Technology, University of Bristol, Bristol, UK +1
Relative Parameter Importance in Task-Agnostic Replay-Free Continual Learning
Aug 1, 2026Malavika Suresh, Ikechukwu Nkisi-Orji, Nirmalie WiratungaClass-Incremental Learning Domain-Incremental Learning
Robert Gordon University
The Art of Not Forgetting A Local Learning Architecture for Continual Learning
Jul 29, 2026Ashmith Atmuri, Yashaswini Rao BhogarajulaMemory-Augmented Neural Networks Continual Learning
Arkadhi Research

Oct 6, 2026, cs.AIJ/K move · Enter open · S save

CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

Maoqi Liu, Quan Fang, Yufei He

Beijing University of Posts and Telecommunications · National University of Singapore

Continual Learning for LLMs Low-Rank Adaptation Incremental Learning Catastrophic Forgetting

Open paper AskPDF arXiv

Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, recent advances implement low-rank adaptation with orthogonal projections (e.g., O-LoRA) to isolate task parameters. However, we reveal that such strict geometric constraints trigger an "Orthogonality Dilemma": rigid parameter isolation impedes the transfer and accumulation of shared representations across semantically related tasks. In this work, we propose a new replay-free method, called Consolidation and Decoupling LoRA (CoDe-LoRA), for CL of LLMs. CoDe-LoRA disentangles the learning process into Consolidating Universal Knowledge and Decoupling Task-Specific Knowledge. To achieve this, CoDe-LoRA leverages an adaptive null space projection mechanism and semantic routing to balance knowledge accumulation with task-specific adaptation. Experimental results across four backbones and three CL benchmarks show that CoDe-LoRA achieves the best average accuracy. Our code is available at https://github.com/Estrellajer/CoDe-LoRA.

Momentum

Related topics

Latest papers 103