Continual Learning for VLMs

VLM: Vision-Language Model

Overview

This field studies how vision-language models such as CLIP and multimodal large language models can learn new tasks, domains, or modalities while retaining earlier knowledge and cross-modal alignment. Common approaches include replaying examples or compact memories, regularizing parameter and representation changes, and using parameter-efficient prompts, adapters, low-rank updates, or routed experts. Researchers evaluate these methods in settings such as class- and domain-incremental recognition, visual question answering, multimodal instruction tuning, retrieval, and video understanding. A central tension is balancing learning new distributions against forgetting and preserving generalization, while benchmarks increasingly test task order, realistic distribution shifts, transfer, memory, and computational cost.

Latest papers

CardsList

87 papers7 papers in the last four weeks, down 12% on the four weeks before. 0.1% of all new papers.

  1. [CLS] and [EOS] Tokens are not Enough for CLIP-based Class-Incremental Learning

    Nov 16, 2026Hao Sun, Tao Hu, Shixiong Kai +4Incremental LearningContinual Learning for VLMs

  2. Incremental Learning of Vision-Language Models via Task Subspace Projection and Dynamic LoRA

    Nov 14, 2026Hao Fu, Qian Feng, Hanbin Zhao +5Task-Incremental LearningContinual Learning for VLMs

  3. CoPre-MoE: Consolidating and Preserving MoE for Multimodal Continual Instruction Tuning

    Nov 13, 2026Tao Hu, Ling-Xiang Zhang, Da-Wei ZhouMixture-of-Experts Fine-TuningContinual Learning for VLMs

  4. STABLE: A Continual Learning Optimizer with Adaptive Drift Control

    Nov 7, 2026Xiaoyan Li, Lanpei Li, Massimo Coppola +2Continual LearningIncremental Learning

  5. ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models

    Nov 6, 2026Lingyun Song, Ziyao Chen, Kang Pan +5Subspace-Based Continual LearningKnowledge Editing

  6. Vision-Language Agents Can Learn in Their Own Latent Worlds

    Oct 27, 2026Haoxi Li, Sikai Bai, Jianfei Ma +5RL for VLMsWorld Models for VLA Models

  7. Expandable Modality Experts for Medical Vision-Language Learning

    Oct 21, 2026DM BAPPY, Eunsu Kim, Hanjoo Lee +2Medical VLMsContinual Learning for VLMs

  8. CART: Compatible Routing for Multimodal Continual Instruction Tuning

    Oct 15, 2026Ling-Xiang Zhang, Tao Hu, Da-Wei ZhouContinual Learning for VLMsMultimodal Large Language Models

  9. Monarch-MoE: Structured Experts for Multimodal Continual Instruction Tuning

    Oct 9, 2026Xi Zhang, Boyu Diao, Zesen Chen +3Continual Learning for VLMsMultimodal Instruction Tuning

  10. ReCAP: Retrieval-Guided Capability Reuse for Multimodal Continual Instruction Tuning

    Sep 29, 2026Tao Hu, Zhinuo Zhou, Xialiang Tong +2Subspace-Based Continual LearningContinual Learning

  11. Visual Branch is What You Need for CLIP-based Class-Incremental Learning

    Sep 29, 2026Tao Hu, Zhen-Hao Xie, Jingcai Guo +2Class-Incremental LearningModality Gap in VLMs

  12. Dual-Mode Low-Rank Learner with Bridge-Prototype Ensemble for Vision-Language Class-Incremental Learning

    Sep 29, 2026Chiyuan He, Zihuan Qiu, Fanman Meng +5Prototype-Based LearningLow-Rank Adaptation

  13. Brain-Inspired Hierarchical Modularity for General Continual Learning

    Sep 21, 2026Hongwei Yan, Kanglei Zhou, Qi Cheng +7Continual Learning for LLMsContinual Learning for VLA Models

  14. ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training

    Sep 15, 2026Zhihao Zhang, Mingqi Wu, Qiaole Dong +15Fine-TuningMultimodal Large Language Models

  15. AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation

    Sep 14, 2026Shengjie Jin, Zelong Sun, Hengbo Xu +2Memory-Augmented VLMsContinual Learning for VLMs

  16. Towards Continual Test-Time Adaptation of Vision-Language Models in Open-Vocabulary Semantic Segmentation

    Aug 30, 2026Chandler Timm C. Doloriel, Yunbei Zhang, Sarthak Kumar Maharana +5Test-Time Adaptation of VLMsOpen-Vocabulary Semantic Segmentation

  17. CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning

    Aug 28, 2026Runze Liu, Naibin Gu, Mingxu Ai +4Continual Learning for VLMsMultimodal Instruction Tuning

  18. LLaVAFlow: Preserving Latent Alignment Flow for Parameter-Efficient Multimodal Fine-Tuning

    Aug 27, 2026Muyao Yuan, Muyan Jiao, Jiangyong Ying +5Multimodal Instruction TuningCross-Modal Alignment

  19. A Visual Dependence-Aware Framework for Multimodal Unsupervised Continual Post-Training

    Aug 26, 2026Kaichen Li, Zhilin Zhu, Jianhao Huang +7Continual Learning for VLMsVLM Adaptation

  20. TASSO: TAsk-Specific Subspace Optimization for Continual Learning of Vision-Language Models

    Aug 21, 2026Chang Sun, Francesco Barbato, Matteo Caligiuri +1Continual Learning for VLMsCatastrophic Forgetting

  21. Information-Theoretic Decoupled Prompt Tuning for Continual Learning

    Aug 13, 2026Yunfei Zhang, Wen Wen, Tieliang Gong +1Continual Learning for VLMsContinual Learning for LLMs

  22. DARAD: Dual Adapters and Ranking-Aware Distillation for Continual Remote Sensing Image-Text Retrieval

    Aug 6, 2026Xi Chen, Xu Chen, Xiangyang Jia +3VLM AdaptationImage-Text Retrieval

  23. Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Training

    Aug 4, 2026Yibei Liu, Jiajun Chen, Qianle Zhang +4Multimodal Large Language ModelsRisk-Sensitive RL

  24. SAFE-Merge: Data-Free Continual Model Merging with General Knowledge Preservation

    Aug 2, 2026Zihuan Qiu, Zhiyang Liao, Chiyuan He +5Language Model MergingModel Merging

  25. Progressive Multimodal Alignment for Continual Instruction Tuning

    Jul 29, 2026Duzhen Zhang, Yahan Yu, Qiaoyi Su +2Cross-Modal AlignmentMultimodal Large Language Models

  26. Continual Video-MLLM Adaptation over Evolving Domains

    Jul 21, 2026Rui Cheng, Meixing Shi, Yuxiang Cai +3Video Question AnsweringDomain-Incremental Learning