Continual LoRA

LoRA: Low-Rank Adaptation

Follow this topic Ask about this topic

Latest papers

27 papers4 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Modulating Merging Strengths via Joint Loss Estimation for LoRA-based Continual Learning
Nov 3, 2026Kun Gu, De Cheng, Zhipeng Xu +3Continual LoRA Model Merging
Xidian University · Huawei Technologies Co., Ltd.
BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning
Nov 2, 2026Tom Saliencro, Wen-Hsuan Liu, Rohan Desai +2Exemplar-Free Class-Incremental Learning Continual LoRA
University of California, Irvine · University of Washington
RASP: Read-All, Sparse-Plasticity Continual Learning in a Stationary-Size LoRA
Nov 2, 2026Wen-Hsuan Liu, Amara Okonkwo, Tom Saliencro +2Continual LoRA Storage-Efficient Continual Learning
University of California, Irvine · University of Washington
APRIL: Ambiguity-Aware Expert Routing for Respiratory Sound Domain-Incremental Learning
Oct 20, 2026Xinhong Li, Weiping TuRespiratory Sound Classification Domain-Incremental Learning
School of Computer Science, Wuhan University, Wuhan, Hubei, China
LSA-PAST: Layer-Specific Adaptation and Prototype Transport for Compositional Incremental Learning
Oct 19, 2026Kyle Stein, Andrew Arash Mahyari, Guillermo Francia +2Incremental Learning Class-Incremental Learning
University of West Florida, Pensacola, FL, USA · Florida Institute for Human and Machine Cognition, Pensacola, FL, USA · Indiana Univeristy, Bloomington, IN, USA
WeAct: Dynamic Weight Switching as Agent Actions
Oct 19, 2026Rui Shi, Xunjian Yin, Jiacheng Sang +2LLM Agent Training Continual Learning for LLM Agents
Carnegie Mellon University · City University of Hong Kong · Duke University +1
Making LoRA Identifiable: Orthogonal Alignment for Continual Learning
Oct 12, 2026Pratik Rakesh Singh, Mohammadi Zaki, Akash Saha +2Orthogonal Procrustes Alignment Continual LoRA
Sony Research India
PaLoRA: Paced Low-Rank Adaptation for Continual Learning
Oct 3, 2026Yuxuan Li, Fanhu Zeng, Hao TangContinual LoRA Continual Learning
School of Computer Science Peking University · Institute of Automation Chinese Academy of Sciences
MuLoRA: Spectrally Balanced Low-Rank Adaptation for Continual Learning
Oct 1, 2026Junkang LiuContinual Learning Low-Rank Adaptation
Tianjin University
A Dynamical Theory of LoRA in Continual Learning
Sep 30, 2026Théo Marchetta, Filippo Alessandroni, Alessandro Breccia +2Continual Learning Low-Rank Adaptation
Department of Mathematics, Alma Mater Studiorum – Università di Bologna, Piazza di Porta San Donato 5, 40126 Bologna, Italy · Gatsby Computational Neuroscience Unit, University College London · Donders Centre for Neuroscience, Radboud University, Nijmegen, The Netherlands
Muon Can Outperform Dedicated Continual Learning Methods
Sep 21, 2026Sebastian George Sincari, Bogdan Alexandru Gheorghe, Antonio BarbalauContinual Learning Muon Optimizer
Faculty of Mathematics and Computer Science, University of Bucharest, Bucharest, Romania · Bitdefender, Bucharest, Romania
Bilinear Optimization Divergence: Diagnosing Factor-Constrained LoRA Continual Learning
Sep 20, 2026YongShun Wang, JianLin Su, Yong MaContinual Learning for LLMs Low-Rank Adaptation
Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents
Sep 11, 2026Pengyang Zhou, Xiaobin Tu, Zhengxi Liu +7Low-Rank Adaptation Continual LoRA
Alibaba Cloud
Geo-LoRA: Geometry-Aware Subspace Evolution for Low-Rank Adaptation in Continual Learning
Aug 27, 2026Yibo FengContinual Learning Continual LoRA
University of Electronic Science and Technology of China Chengdu, China
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
Jul 26, 2026Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli +2Continual Learning for LLMs Continual LoRA
College of Health Solutions, Arizona State University, Phoenix, Arizona, USA · Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA +1
ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning
Jul 4, 2026Wentao LuContinual Learning for LLMs Low-Rank Adaptation
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning
Jun 26, 2026Tanguy Dieudonné, Giulia Lanzillotta, Enis Simsar +2Continual Learning Low-Rank Adaptation
Department of Computer Science, ETH Zurich, Zurich, Switzerland.
TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
Jun 4, 2026Marius Dragoi, Ioana Pintilie, Alexandra Dragomir +2Continual Learning Low-Rank Adaptation
Bitdefender, Romania
Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
May 27, 2026Cheng Chen, Pengpeng Zeng, Yuyu Guo +3Stability-Plasticity Dilemma Continual Learning
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Computer Science and Technology, Tongji University, Shanghai, China · Independent Researcher +1
Energy-Structured Low-Rank Adaptation for Continual Learning
May 26, 2026Longhua Li, Lei Qi, Qi Tian +1Continual Learning Low-Rank Adaptation
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China · Huawei Technologies, Shenzhen, China
LoRA vs. Full Fine-Tuning: A Theoretical Perspective
May 18, 2026Ali Zindari, Rotem Mulayoff, Sebastian U. StichFine-Tuning Low-Rank Adaptation
Universität des Saarlandes · CISPA Helmholtz Center for Information Security
Continual Fine-Tuning of Large Language Models via Program Memory
May 13, 2026Hung Le, Svetha VenkateshContinual Learning for LLMs Memory-Augmented Language Models
Deakin Applied AI Initiative, Deakin University, Australia
Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
May 12, 2026Joana Pasquali, Ramiro N. Barros, Arthur S. Bianchessi +7Gradient Interference Catastrophic Forgetting
MALTA, Machine Learning Theory and Applications Lab, PUCRS, Porto Alegre, Brazil · Kunumi Institute, Brazil
SR $^2$ -LoRA: Self-Rectifying Inter-layer Relations in Low-Rank Adaptation for Class-Incremental Learning
May 8, 2026Fengqiang Wan, Yipeng Lin, Kan Lv +1Representation Drift Low-Rank Adaptation
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
Apr 17, 2026Alexandra Dragomir, Ioana Pintilie, Antonio Barbalau +6Parameter-Efficient Fine-Tuning Low-Rank Adaptation
Bitdefender · ETH Zurich · University of Bucharest
Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation
Jan 31, 2026Hao Gu, Mao-Lin Luo, Zi-Hao Zhou +3VLM Adaptation Continual Learning
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China
C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models
Feb 25, 2025Xin Zhang, Liang Bai, Xian YangContinual LoRA Low-Rank Adaptation
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, 030006, China · Alliance Manchester Business School, The University of Manchester, Manchester, M13 9PL, UK

Nov 3, 2026J/K move · Enter open · S save

Modulating Merging Strengths via Joint Loss Estimation for LoRA-based Continual Learning

Kun Gu, De Cheng, Zhipeng Xu, Lingfeng He+2

Xidian University · Huawei Technologies Co., Ltd.

Continual LoRA Model Merging Adapter-Based Continual Learning

Open paper AskPDF arXiv

Continual learning (CL) aims to adapt models to new tasks while preserving previously learned knowledge. Recently, Low-Rank Adaptation (LoRA), a representative Parameter-Efficient Fine-Tuning (PEFT) method, has gained increasing attention in CL due to its efficiency and scalability. Several LoRA-based merging methods maintain a single inference model by scaling the newly learned LoRA update with one coefficient and merging it into the previous weights after each task. However, they usually control the whole update with a shared scalar factor, although different update elements may affect previous and current tasks in different ways. To address this issue, we propose Merging Strength Modulation via Joint Loss Estimation (M2LE). It consists of Hessian Information-guided Adaptive Merging (HIM) and Task Simulation Perturbation for Merging (TSP). Instead of searching for a global merge coefficient, HIM directly solves for the target merged update under a joint objective over previous and current tasks. Solving this objective gives an analytic solution that naturally takes the form of element-wise modulation over the current task LoRA update. To improve the stability of the learned update under subsequent merging, we introduce TSP which simulates different tasks interference. Experiments on multiple benchmarks show that M2LE consistently outperforms existing PEFT-based continual learning methods. Code is available at https://github.com/bbdeeer/M2LE_NeurIPS2026.