Continual Learning for VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 51

All topics

KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptation
Jun 15, 2026Mao-Lin Luo, Yi-Lin Zhang, Zi-Hao Zhou +5VLM Adaptation Continual Learning for VLMs
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China, and the Key Laboratory of Computer Network and Information Integration, Southeast University, Ministry of Education, China · Huawei Noah’s Ark Lab
Understanding Cross-Modal Contributions in Continual Vision-Language Models: A Theoretical Perspective
Jun 12, 2026Salimeh Sekeh, Mary WisellContinual Learning Large Vision-Language Models
San Diego State University
ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation
Jun 10, 2026Jiangtao Kong, Peijun Zhao, Chun-Fu Chen +4Large Vision-Language Models Incremental Learning
Department of Computer Science, William & Mary, Virginia, USA · Global Technology Applied Research, JPMorgan-Chase, USA · Mohamed bin Zayed University of Artificial Intelligence, Masdar City, Abu Dhabi
CL-CLIP: CLIP-Based Continual Learning Framework with Cost-Volume Category Decoupling for Object Detection
Jun 5, 2026Zihan Liu, Yuguang Yang, Shengjie Su +5VLM Adaptation Object Detection
National College for Excellent Engineers, Beihang University · School of Electronic Information Engineering, Beihang University · AI Research, Qihoo 360 +5
PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Models
Jun 2, 2026Ziyang Chen, Shaoguang Wang, Weiyu Guo +5Continual Learning for VLA Models Robotic Manipulation
Thrust of AI, HKUST(Guangzhou), Guangzhou, China · AI2 Robotics, Shenzhen, China
ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning
Jun 1, 2026Yu-Cheng Shi, Zhen-Hao Xie, Jun-Tao Tang +1Large Vision-Language Models Adapter Tuning
School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval
May 29, 2026Alicja Dobrzeniecka, Filip Szatkowski, Sebastian Cygert +2Multimodal IR VLM Adaptation
NASK National Research Institute · IDEAS Research Institute Warsaw University of Technology · Warsaw University of Technology +2
AREA: Attribute Extraction and Aggregation for CLIP-Based Class-Incremental Learning
May 27, 2026Zhen-Hao Xie, Yu-Cheng Shi, Da-Wei ZhouClass-Incremental Learning Vision-Language Models
School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
HyLoVQA: Dynamic Hypernetwork-Generated Low-Rank Adaptation for Continual Visual Question Answering
May 21, 2026Yiran Wang, Chenyi Xiong, Ziyue Qin +3Visual Question Answering Vision-Language Models
School of Computer Science, Hubei University, Wuhan 430062, China · Hubei Key Laboratory of Big Data Intelligent Analysis and Application (Hubei University), Wuhan 430062, China · Key Laboratory of Intelligent Sensing System and Security (Hubei University), Ministry of Education, Wuhan 430062, China
iGSP:Implicit Gradient Subspace Projection for Efficient Continual Learning of Vision-Language Models
May 19, 2026Xuezhi Cui, Dongbo Zhou, Wang Guo +8Vision-Language Models VLM Adaptation
School of Geosciences and Info-Physics, Central South University, Changsha 410083, China · Faculty of Artifi cial Intelligence in Education in Central China Normal University, Wuhan 430079, China · School of Earth Sciences and Spatial Information Engineering, Hunan University of Science and Technology, Xiangtan 411201, China
CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
May 18, 2026Yang Liu, Toan Nguyen, Flora D. SalimContinual Learning for LLMs Vision-Language Models
School of Computer Science and Engineering University of New South Wales Australia
Reasoning Portability: Guiding Continual Learning for MLLMs in the RLVR Era
May 17, 2026Qiuhe Hong, Yuyang Liu, Shuo Yang +3KL-Regularized RL Continual Learning
1Shenzhen Graduate School of Peking University · Centre for Artificial Intelligence and Robotics, HKISI, CAS · 3Peng Cheng Laboratory
Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models
May 14, 2026Yuehao Liu, Shanyan Guan, Weijia Zhang +4Continual Learning for LLMs Continual Learning for VLMs
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University · vivo Mobile Communication Co., Ltd.
Lifelong Learning in Vision-Language Models: Enhanced EWC with Cross-Modal Knowledge Retention
May 12, 2026Hamza Ahmed Durrani, Rafay Suleman DurraniVision-Language Models Cross-Modal Alignment
Sejong University, Computer Science Engineering · Technische Universität Ilmenau, Computer Engineering
Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models
May 8, 2026Mengxin Qin, Xiang Zhang, Kun Wei +2Class-Incremental Learning Vision-Language Models
School of Electronic Engineering, Xidian University Xi’an 710071, China
DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
May 8, 2026Mengxin Qin, Xiang Zhang, Xi Wang +3Vision-Language Models Fine-Tuning
School of Electronic Engineering, Xidian University Xi’an 710071, China
StrLoRA: Towards Streaming Continual Visual Instruction Tuning for MLLMs
May 8, 2026Chang Che, Ziqi Wang, Hui Ma +2VLM Adaptation Low-Rank Adaptation
Hefei University of Technology · Tsinghua University
Memory-Efficient Continual Learning with CLIP Models
May 5, 2026Ryan King, Gang Li, Bobak Mortazavi +1Class-Incremental Learning Contrastive Learning
Department of Computer Science Texas A&M University College Station, TX 77843 · Texas A&M University College Station, TX 77843
Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weighting
Apr 20, 2026Hyeonseo Jang, Hyuk Kwon, Kibok LeeClass-Incremental Learning Vision-Language Models
Yonsei University
AIM: Asymmetric Information Masking for Visual Question Answering Continual Learning
Apr 16, 2026Peifeng Zhang, Zice Qiu, Donghua Yu +4Visual Question Answering Vision-Language Models
Sun Yat-Sen University · National Supercomputing Center in Shenzhen · Tsinghua University
HERMAN: Hierarchical Representation Matching for CLIP-based Class-Incremental Learning
Sep 26, 2025Zhen-Hao Xie, Yan Wang, Lan Li +3Class-Incremental Learning Vision-Language Models
School of Artificial Intelligence and the State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

Jun 15, 2026, cs.CVJ/K move · Enter open · S save

KeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptation

Mao-Lin Luo, Yi-Lin Zhang, Zi-Hao Zhou, Yankun Hong+4

School of Computer Science and Engineering, Southeast University, Nanjing 210096, China, and the Key Laboratory of Computer Network and Information Integration, Southeast University, Ministry of Education, China · Huawei Noah’s Ark Lab

VLM Adaptation Continual Learning for VLMs Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Continual learning for pre-trained vision-language models requires balancing three competing objectives: retaining pre-trained knowledge, preserving knowledge from a sequence of learned tasks, and maintaining the plasticity to acquire new knowledge. This paper presents KeepLoRA++, balancing these objectives through a unified dual-dimensional knowledge retention mechanism. We analyze knowledge distribution of Transformer architecture from both inter-layer and intra-layer perspectives. The inter-layer perspective examines how retention is distributed across layers, while the intra-layer perspective focuses on the parameter space within each layer. Our analysis reveals a structural property: general transferable knowledge is mainly encoded in the shallow layers and the principal subspace of the parameters, while task-specific adaptations are localized in the deep layers and the residual subspace. Motivated by this insight, KeepLoRA++ introduces a layer-scaled residual gradient adaptation method. New tasks are learned by restricting LoRA parameter updates to the residual subspace, combined with a shallow-to-deep layer scaling, to prevent interference with previously acquired capabilities. Specifically, the gradient of a new task is projected onto a subspace orthogonal to both the principal subspace of the pre-trained model and the dominant directions of previous task features, while simultaneously assigning smaller update magnitudes to shallow layers and larger ones to deeper layers. Our theoretical analysis and empirical evaluations confirm that KeepLoRA++ successfully balances these three competing objectives, consistently outperforming representative baselines across image classification, visual question answering, and video understanding tasks.

Momentum

Related topics

Latest papers 51